I know that they say that the OP is like an addiction and hard to let go of but I think your H took it to a whole new level.
Personally, if I had been in your shoes and my H had humiliated me and basically called me a liar in front of someone who he supposedly did not love, I would have turned around, walked out the door and kept driving. But that's just me... I personally hope and expect that someone who claim's that they loved me would also have my back, so to speak. I actually experienced something like this last year when my ex's GF started an arguement with me for no reason and I refused her attempts at drawing me in, my ex actually came to my defense and told her to shut the F up. It was a beautiful moment and I got to watch GF go into absolute psycho mode that he stood up for me. That was being respectful.
I'm a firm believer in actions speak louder than words. His actions are dumbfounding to me. He's gone to all the trouble of pursuing you and then here you are and he asks you to leave? Wow talk about wanting his cake and eating it too.
Some of the most important components of a loving relationship are trust, respect, admiration...
Actually under the circumstances as unfortunate as they were, I think you did handle yourself quite lovely and admirably. Akward situation no doubt.
Wonder what the OW is thinking right about now? Wonder if she told him off and high tailed it out of there.
I think the main thought here is what do YOU want. You sound quite level headed and realistic here. And regarding boundaries... what boundaries do you have in place if he makes contact and wishes to come back? What will you deem repsectable and acceptable behaviour from him?