I hear you. That nagging doubt about being #2 is really you thinking you're not good enough though. If you were convinced you were good enough then you wouldn't be threatened by H's fantasy, you'd either know you were better than the fantasy woman, or you'd know it was H's issue and not yours and it wouldn't bother you at all.
I also hear you about the sadness.
Yes, I struggled for years to get more affection from W. Eventually I concluded that she just didn't love me. I was angry at her for marrying me in the first place. I thought she married me because I was "good enough" versus someone she was passionate about, and I felt that had prevented me from marrying someone with whom I could have been truly happy. I didn't voice any of this, but it fueled both quiet resentment as well as convinced me that I wasn't good enough. Nothing I did could get an affectionate response from W, so I must not be good enough.
Maybe as a defense mechanism I then convinced myself that W was incapable of being loving with anyone. Maybe it wasn't me, maybe she was just depressed and therefore incapable of happiness. That formed a kind of protective shell for me for several years.
When she then had the EA and I saw all the emotion, enthusiasm and love she lavished on OM, how she sought to meet up with him, go for walks, planned a picnic at the beach, etc., it was very hard. It took away my conclusion that my W was "just an unloving person" and doubled-down on the "you're not good enough". I then got the "oh, she's capable of it alright, you just don't do that for her"
MC told me that what W was bringing to the table in that EA is not who she is, that I have the real W. That she would not be capable of sustaining that, nor would she be happy if she tried. He said that an affair is all about two people telling each other how wonderful they are -- and that is of course unsustainable. Sooner or later that glass is going to break. W says she realized that while she was in it -- that what she was living was not real, but it felt really good. She says she knew it wasn't going anywhere, and she had no illusions about a long term relationship with OM. That said, she was no less crushed and angry when he chose to end it.
One of my best learnings through this is that I am good enough, and it's my job to convince myself of that. It's not W's job to tell me that I'm worthy or show me that I'm worthy, it's my job to BE worthy, and at that point I don't care if W appreciates that or not.
I discovered the EA through snooping, and for a long time thereafter I was addicted to snooping. I eventually decided that snooping is a defensive life, and I didn't want to live it. My current attitude is that if W wants to pursue another EA (or PA), then go for it. It's a reflection on her, not me. I *know* I'm a good husband and a good father. My W's issues with me are more a reflection of her issues than places I fall short.
The challenge, however, is that it's always been important for me to be acknowledged, and to feel wanted and needed. W doesn't naturally do that. Some of the things that W says or does continues to prey upon this weakness of mine, not necessarily through any fault of her own. What I have learned, and am still learning, is that my challenge in this relationship is not to let that bother me, and to find comfort in my own successes.
I'm a work in progress for sure but I "get it" now. I'm not worried about competing with the fantasy, and if W wants to pursue one, have at it, I'm gone -- and I have no doubts that I can find someone who appreciates what I bring to the table.
Accuray
Married 18, Together 20, Now Divorced M: 48, W: 50, D: 18, S: 16, D: 12 Bomb Dropped (EA, D): 7/13/11 Start Reconcile: 8/15/11 Bomb Dropped (EA, D): 5/1/2014 (Divorced) In a New Relationship: 3/2015