Wow I really don't know what to say except "this too shall pass" I have been there. I NEVER really "dated" but I was really close to lowering myself to the WAS level. WAAAY back I had a chance to "cheat" and the last possible minute I "chickened out" Now I am glad I did. I have kept my vows. I have nothing to appoligise for.
Yes you can't keep him from seeing your\his son BUT..he as well as you need to be reasonable. He needs to give you a date and time. If it fits your schedule great if not he needs to "rebook". Another way is if YOU need to go somewhere tell him in advance and see it he can watch your\his son on that day. Sorry I could not be more help Doc
And if I claim to be a wise man, well It surely means that I don't know