Are you sure he gave up instantly? Maybe he thought he should do the last resort technique - pull back and let you miss him once you were forced to rely on OM to meet all your emotional needs. I kept trying to do the same with my W and it worked a little bit, but I couldn't keep it up. I enjoy being with her too much. I guess if I was your H and you said you weren't sure about OM, I would have told you that we couldn't work on transforming our marriage with three people involved and I'd be prepared to work hard to make the M be supportive and fulfilling for both of us but there'd have to be no contact with OM. He didn't do anything like this at all? I can understand being hurt, but not doing ANYTHING isn't good.
My wife wrote me that part of the reason she never talked to me was because she was trying to figure out what she was missing from me that would make her interested in OM. Said she also felt confused because we were always discussing problems while everything with OM was new, exciting, and promising. She didn't want to admit the latter feelings were more compelling. Sounds like you might feel something similar?
You do have some thinking to do, but you aren't alone. You should do some reading, definitely counseling if you can, and keep posting to the people here - they have good insight.