Sorry to hear of the loss of your mother. Mine died a few years back, so I know what you are going through.
Ok, here is my take. First peel yourself off the walls. And let's get down to what is really going on in your situation. H has a fantasy and he can't get out of the loop. Why? Who knows. One thought that I have is that you drive yourself nuts calling this thing an EA. Maybe you should look at it differently. An affair takes two willing parties. Maybe call her FC (fantasy chick, or maybe you can find another name using those initials). FCs sooner or later always show kinks in their armor. But you are fretting over something that IMO will never happen.
Tree, things seemed to be improving with H. Is that still going on? I think the big thing you have to ask yourself in all of this is whether or not you are contributing to this ongoing situation. DBing tells us when something doesn't work to do something different. What can you do differently? Do you fear his reaction if you do? Is it time to rock the boat? Are you two even communicating?
Let me tell what I am doing in my situation. My divorce is over a year old already. Little by little, I put a more of myself out. I take it very slowly and really don't know what the end of all of it will be. Is there a way to put more of yourself out there? Are you flapping in the wind or being too guarded with your feelings and emotions around the H? After all, this really isn't about dinner with a group of colleagues.