When you went to her house and the OM was there, describe the interaction between everyone. You said you didn't say anything about him to her, but tell us more.
Let me share with you what I see as a big mistake newcomer H's make. He steps back and stops engaging in arguments. Suddenly, the tension is broken and he and his WW are on more friendly grounds again. He sees this as great progress, to the point of thinking everything is back to going uphill again. He gets all excited and thinks, "I can't believe it was so simple. This is great!" It usually takes less than 48 hrs for him to discover how wrong he was.
I will try to save time by not repeating myself, if you'll read the thread, "For Newcomer LBH with Wayward Wife". I believe I touched on this particular action that you've experienced.
It is not about what you feel should work in your M. It is about doing the work that gets the right results. Do what works!
