When I was married, a good part of the problems with xH sound similar to what you are saying about your H... except the opposite. Meaning, I was your H and you were my xH.
There were things, sexually, my xH wanted to do, that I just could not do and had no interest whatsoever in doing (bufu, for one). He also wanted me to be assertive. And some other stuff...
It wasn't so much the 'acts' of what he was asking me that made me reluctant... it was the fact that there was no emotional trust between us, and I could only go so far in taking risks, without him taking a few of his own (and I didn't want to 'cop' or 'mother,' him about it). I was learning to come out of my sexual shell, and I think that he was 1) out of patience and 2) put to a few test of his own that he didn't want to face.
What I hear from you is something very similar. You say you are not angry at your H, but that is all I hear from you. Your respect for him is gone as well, and right about now, you just want what you want, and to he!! with waiting. (Which is completely understandable, btw). And I think sometimes hearing about Mojo's and maybe some of my 'dating' escapades are NOT helping you, either. It feeds the 'restlessness.'
But if you think he isn't picking up on that, don't kid yourself. What are you expecting of H, really? And then on top of all that you tell him you are fantasizing about another man and you were ready to go meet him... but the guy canceled on you, so now H gets to hear the confession?
I think you need to sh!t or get off the pot, and stop punishing your H for what he isn't. I'm not saying that anything that you want is bad or wrong... but to stay in that M, and continue to be angry and resentful of him for what he isn't... and to hold that up to him and even TELL him... wow. If you don't want to be there, leave. (He is doing some hostage holding of his own that isn't cool either). But this 'covert' emotional warfare the two of you are doing to each other really needs to stop.