I think your last post says it all. You know that your wife is going to keep this up, but you are fearful that she is doing this to "keep" you and not out of a desire to grow the sexual relationship, or increase intimacy with you.
In addition, the EA can't be helping matters. If your heart belongs to another, then you can't expect it to feel "right" with your wife. My advice is to just give it time. Time will tell whether your wife's personality really does bother you too much for long-term happiness, or whether you are just missing the OW. Time will also tell whether your wife wants more intimacy or just wants to hang on to you.
