Hi Dawgy -- I think what you are seeing are lonely women looking to take advantage of a bad situation with a weakened man who they believe needs to be cuddled. Because your confidence is shot (mine too), you are attaching importance to these meetings when they mean nothing.
Unless, of course, you want to have an affair because your needs are not being met -- and then these meetings mean everything. That should sound familiar to everyone on this thread ...
If you believe in your marriage and your wife, then stand up for them, accept what you did to hurt the person you love, and do the work to change those things if you think they make you better.
While you are doing what needs doing, stop confiding in bimbos. Consider the character of a woman who is willing to take advantage of another person who feels like we do right now. They must have the moral compass of one of those green flies that eats shXX all day ...