Do what you feel is right for you. If you want to visit them, then do it. It isn't his business. He divorced you, his family didn't. At least that is the way it seems from your post. For me, I am learning that her family wants to remain in contact with me. I thought that they all disowned me. I am happy that I was wrong.
You know, your EH might be going through a MLC. The more I think about my sitch lately, I think that is what my EW is going through. One of the signs is no communication with their family.
When he says that it is over, to him it is -- for now. Remember this phrase "for now". I was told this by my therapist in the beginning, and she is turning out to be right. It seems that my EW is not feeling as strong about her decision to D me. Only time will tell for all of us....
M: 35; Together 14 years - married 5.5 no children Bomb 22-JAN-2005; D a few months later First real relationship post D just ended ( 7 months ):(