From this guy's perspective--and don't take it the wrong way--There are two things going on here-- OW and you.
First, with regard to the OW, as you know, there is a lot of healing to do, especially in your heart. For H, it's pressure and a reminder; he simply wishes the issue would go away and that the two of you can go forward. Obviously, that is not going to be possible if your heart feels the way it does. Thus, in the meantime, you have to let your head take over--and give him a little bit of space and let him feel comfortable about discussing the subject with you. One of the 8,000 books I've read is "Not Just Friends," by Shirley Glass--which is a setep by step analysis of affiars from the perspective of all sides. I'd get my hands on it. I also would suggest consulting a good C. Ultimately, it seems the issue must be addressed head-on and if H is serious about the matter he will have to come to the table, but not right now.
Second, when I read your thought process, which contantly analyzes every detail--it reminds of me what drove a wedge between my W and me. I simply couldn't take it any more. She would see things that weren't there and it drove me nuts. Eventually, I just tuned her out to the point where she allowed someone else to tune in. In any event, all your H said in this e-mail is CHILL!!!! And when you don't chill, you put him on edge. There has to be a middle ground somewhere, but for now, you really have to dig deep to better relax and thought-stop.
{{{{{{{{{{{{{Zoo}}}}}}}}}}}}}}}}}}}. You're doing well. Relax and be good.