My H ALSO tried to use the excuse that the women from his past would use info against him...and therefore he wouldn't let himself trust me. I also had to point out to him, pretty much the same way you did...that he was punishing ME because of other women's past behaviors and judging how I would behave/react on them...with no basis for it from my own behaviors.
You ARE breaking down that resolve of his. He IS seeing now that you are SERIOUS. As I told my H, and I told you....they sabotage a good thing, and it's their own doing. You have to point it out to them though and let them see that you are SERIOUS about the consequences and really put them at the brink of it (with you being serious about following-through) before they see it.
He may not have picked up the phone then and there (he may not have been able to) BUT...he did let that wall down some and that's a victory for you. Just be aware (and I believe you are) that this is probably not going to be the last time you will have to address this. Hopefully Monday he won't come up with some "excuse" and will follow through, just don't be surprised if he doesn't.