Think about how you felt back when you did have a connection to her and a good rapport versus today. These things are an accumulation, and if you tolerate it for long enough will cause your mental health to get poor.
Its not about what they say, its about how they act around you as well, it takes alot of the spirit out of you.
I am glad that you have the amount of fight and self-dignity that you have left.
Whatever you do, ensure it is not a reaction - but planned action.
One of the best things I did for my self-esteem in my messed up, leveraged against me situation was to go on ahead and spend time with a couple of other ladies. Before that point, I knew in my mind that women would be attracted to me, even as I am, but once I took action in it - it really was apparent that I am being a fool, and putting up with it for even one day is one day too many.
I know this goes against the advice on this board, but it is helpful for some. I consider it along the lines of GAL.
Spend time with people who want to spend time with you. Do things that you want to do. Handle your personal responsibility.
You need to spend as little time "Looking back" as you can. In this case "looking back" is thinking about the relationship with your wife. Define your new direction, and if she can convince you through love and dedication then perhaps you will allow her to join you for your future journey. If not, then fine - you are going to be happier anyway.