I guess what I am having a VERY hard time understanding is how can say there are not any other affairs, when in fact, there certainly was and it was you!
You sort of gloss over how you met your H as if it's *kinda* wrong but really deep down it all worked out for you so who cares!
What sort of therapy are you getting as an individual? It's sort of shocking to me (which I realize means very little to you) that you would be surprised your H would have an affair.
What are you wanting the end result to be both for yourself and for your H?
I understand these are hard observations to read but hard or not they are factual as per your post and IMO are significant.