While I agree with NNP's first point, I do not agree with this.
Yes, it is very likely that contacting or starting any 'drama' with OW will cause more trouble between you and H. However, I do not believe for one bit that he "will forever blame you".
It might feel that way at first, but when and if he comes out of his 'fog', I seriously doubt he will continue to feel this way.
Getting over a painful experience is much like crossing monkey bars. You have to let go at some point in order to move forward. ~ Joseph Campbell