I just read your post and wonder what it is you are trying to achieve with your h. Is he asking for the kids overnight? I would suggest that you allow him to have them overnight if possible. This can do 2 things: 1. Add mystery to your life as he gets a chance to wonder what you are about and 2. give him a taste of single parenting. Kids might put a damper on his activities.
This is only my advice. My boys were 11 and 12 when their dad left us. I also had a very hard time with it but my boys were able to vocalize how disgusted they were with their father which made my h's activities very limited (he couldn't have gal friends over cause my boys hated him and didn't want to come back). Whereas your children are too small, they may be able to remind him how important family is and may also give you more of a reason to call him and ask how they are.
I would suggest that you begin to add more mystery to yourself by maybe doing some changes to the house that would surprise him when he drops in. Or change your clothes to a different style, change your hair, get a nose piercing, take up a sport. Anything that will get him wondering what's up would be helpful. I remember clearing out all the 'extras' from our home when the h moved out. He came around a bit more to find out what else was going out on the rubbish heap...gave me an opportunity to wear something provocative on those occassions !
Don't give in yet. My h and were separated for 2 years and we've been back together now for 2 years. Keep your chin up and stay very busy.
!