I have dealt with these same issue in my sitch. It makes you feel almost like your S is leaving YOU, based on OTHER PEOPLES actions! There was a period of time, in the beginning of our split, when my wife would segway into how angry she was with her best friend. I kept thinking, "uhhh shouldn't this separation be in regards to US?" We would literally be discussing the demise of our marriage and she would go off on a rant about how mad she was with BF.....makes no sense to me, and I learned to just let it go and validate her.
I can understand your H feelings, if possibly the signals or 180's you are doing aren't perfectly clear to him. If you are "acting as if" and detaching too coldly, it could easily be misconstrude as you no longer being interested in the M. You have to detach and act as if, but in a way that still allows him to feel that he can engage you. You have to keep that road home clearly paved and accessible. Take a moment and consider how much of an effort that will be for him....the guilt, the embarassment, the anger, the fear. It is a lot to step through. It will be exteremely tough for him. Also, do you think you have completely owned up to your own shortcomings? Have you actually verbalized that to your H? Have you made honest efforts to 180 them and be consistent?