I don't think you loved your H too much, I think you loved yourself too little. And you excused all his crap behavior because you're confrontation-avoidant and he took advantage of it. I think that's human nature--people will take all you give them if it's offered at no cost. (FWIW, both BF and I did that too. Didn't want to rock the boat. Now I speak up immediately when something bothers me and we deal with it then. Because everyone is responsible for his/her own happiness.)
So why are you still allowing him to come and go at the house as he pleases? Because he doesn't have internet and you're still making his life easier? What is life going to look like if you divorce? If that's what he wants then you should show him what that will entail. Set up a custody schedule so he has specific times to be with the children. Then make other plans and leave them to have time together. It will be good for you to be able to have GAL time, good for the children to have time with their father, and good for H to allow him to step up and be a responsible parent.
If you love somebody, set them free. http://tinyurl.com/2empx2g