Part of the problem you're having is physiological. Have you read up about PEAs at all, and affair withdrawal?
The concept is the same, and it's one of the reasons why many of us are stressing NO CONTACT for you. Each and every time you have contact with your ex-husband, endorphines (PEAs) are released in your brain, and it sets your withdrawal "clock" back to 0:00:00 with your ex-husband. And you start reeling, and the cycle repeats.
If you would stop your hyper-focus on him, and "what could have been" (and what might still be, someday), and STOP CONTACT, your brain and emotions would begin the withdrawal process and -- if allowed to continue, unfettered by more contact -- you will eventually be in a better place, emotionally.
I don't know any other way to put it. Your continued contact with your ex-husband is setting you back in your recovery, in my opinion.
