One thing I really believe and it is that affairs are emotional abuse. We tend to see an affair quite wrongly.
As Frank Pittman said, it is not a sign that there is something wrong with the marriage, but that something is wrong with the person having the affair.
Financially can you separate from your h and still be OK? (I seem to remember that you are not married, and live somewhere where a long term relationship confers no legal rights.)
Only you can answer the question 'Am I better off emotionally with this guy or without him' There have been times when I have queried this but in my case I have no real doubt, the price would be too high, but we are all different.
As an observer I do not think you value yourself enough, and people dump on you. I hate to know that you are unhappy, and this jerk isn't mature enough to stop behaving in this way.