Maybe it's just competition with the OW that's wearing you out. From your description, it sounds like the act of your H leaving wasn't such a big deal when he wasn't dating anyone. You don't necessarily want him, but you don't want anyone else to have him either. That's an unrealistic expectation you need to resolve.
Maybe a IC can help you sort these feelings out and give you a more positive image of yourself. When you feel secure in yourself, you can overcome fear, jealousy, and anger.
"It is curious that physical courage should be so common in the world and moral courage so rare." -Mark Twain
