I always view things as moving FORWARD instead of moving ON. Moving "on" is what the WAS do - they decide for multiple people this is for the best and they move on. Your life will be far better if you decide to move FORWARD.
Your W was probably the biggest and certainly the most important part of your life for a very, very long time. And like you said, you now realize she isn't the sort of person you ever really liked or enjoyed anyway. As you stated above the rejection you feel eats at your self esteem and that really isn't so much about your W but more about you.
But really, you can't bank on your W's actions to fix your self esteem issues. Rejection by somebody who was "front and center" in your life is very painful... I think we can all agree on that but only you can decide that she will not dictate the strength of your self esteem.
Does it every go away? I really can't answer about "forever" but I do know the longer you let the actions of a WAS to keep you fragile the longer you will suffer.
My H has been in his affair for now 3 years. I read and read and read some more about how affairs never last and blah blah blah. I question those statistics because the few people I know who had affairs IRL are all now married to their affair partner and have been for some time. My H goes home every night to a very fly, very expensive townhouse to the love of his life (apparently) without a care in the world. Does it make me feel bad? Sometimes. But I also know that my H has always been a very weak man, has not a clue how to lead in a R, lives his life on the surface and needs way too much coddling, touching, reassurance and ego boosting. So you weigh the good with the bad.
I'm glad you see progress. It's very important. But keep the progress about you. I always find it very questionable when WAS are "depressed" yet they manage to vacation and become deeply involved in other enjoyable activities. I'm not sure if you are aware but last year I was diagnosed with clinical depression and a situational panic disorder stemming from tremendous anxiety about my H's verbal/emotional abuse, his infidelity and the extreme legal situation I was thrust in. Completing basic tasks like showering or walking my dog became monumental. That is depression. WAS (IMO) use 'depression' as their crutch because if you can vacation for 10 days you aren't depressed. It's an easy "WAS diagnosis" instead of forcing them to see their actual disorder is entitlement, laziness, lack of maturity and possession of tools to make long term R's work.