I'm not sure if this is "advice" per se, because these are your feelings that you're dealing with and they are perfectly valid, but I'll tell you what I think.
I try to maintain a personal philosophy that I decide to do things not because of external simuli (reactive) but rather because of who I am and what I believe, it comes from inside. I'm not actually successful at this most of the time , but something I like to remind myself of.
If you're committed to your part of saving the marriage (and I know you are), I think it would be consistant with that to drop the D. Focus on yourself and what you want - don't allow yourself to be controlled by H, OW, and the situation. (I know I haven't been there and may be talking out of my a$$).
Secondly, it seems to me that you're kind of in an arms race with H. You want something - him to get rid of OW - and he presumably wants something - dismissal of the D. It might be seen as an act of trust - of unconditional love - to drop it. Yes, you're making yourself vunerable, and it's really uncomfortable. You're giving up your power a little - but you're also in a way taking control of the situation, You're showing strength - "I am committed to doing this. Now, how about you?"
Imagine that you and your H have guns pointed at each other, and you're both waiting for the other to blink. You're thinking, as long as he's got that gun, I'm not putting mine down. But what would happen if you did? Nobody's going to shoot an unarmed person, right? You show strength and actually take the lead by making yourself more vunerable.
Just my two cents - again, I don't mean to minimize your valid feelings on the subject - just providing my POW.
, but something I like to remind myself of.