You may miss your H because you are choosing to no longer respond to him. When the two of you were in contact (even if the contact was not healthy) at least it was *something*. It was familiar and while not always pleasant the connection was still somewhat there.
As far as the OW's H not responding to your message it may very well be because they are already separated. I know in our Separation Agreement I had a clause added that my exH was NOT to interfere in any outside relationships I had under any condition be it a romantic, platonic or family R.
My in laws and grand parent in laws really wanted to keep in touch with me as we were all VERY close. My policy remained the same... anybody who supported my H's affair had no place in my life. And his family provided him with money and free housing so that was that. His mom did contact me a while back just asking how I was but I never responded to her e-mail. I was kind with my goodbye but I meant what I said. Of course there was some confusion as my H told his family our split was mutual and hid his affair for a very long time. Blood is blood no matter what words were said when things were decent.