If you can't leave house when he comes over to be with son, can you have a friend over to visit and hang out in your room or have a board game night or something corny like that? This will help force you to ignore him if you HAVE to be there until you build up the willpower and detachment to do so.