Not sure I have any advice, but here are some (((hugs)))!
One question tho, do you think H could be a sex addict? If he's having sex with you 3-4x/week, plus swinging, plus anything you might not know about - that's kind of a lot of sex, no? Is there porn, too?
FWIW, this sounds very mid-life crisis as well.
The trouble with having an open mind is that people put things in it.
My sitch - Divorce Busted! http://www.divorcebusting.com/forums/ubbthreads.php?ubb=showflat&Number=1804137#Post1804137