The thing is...H doesn't have an ounce of empathy for me. Not a bit. He can't see how this is making me feel. He's only concerned about this own selfish motives.
I'm curious - How is he around your kids? You mentioned he has had D a couple of nights, but not S yet. Does he seem like he is spending time with them out of obligation, rather than because he really desires it? Maybe his parents are kicking him in the ass to be a more responsible father (which, I realize isn't perfect, but it's better than nothing, I guess).