It will get easier you just have to keep those boundries in place.
H needs to work on himself right now and you can not fix this for him. By you letting him come and go you were letting him cake-eat and not allowing him to work on his issues. He needs to figure it out on his own.
You are doing well and day by day you will feel more at ease (at least i did).
NOW GO DO SOMETHING JUST FOR YOU!!!!
JAK
You don't get to choose how you're going to die. Or when, you can only decide how you're going to live now. ~Joan Baez