You're right it would be letting W's family have too much power over you if you were angry about the way the treat you. Its that old saying I suppose that just because you love your W/H it doesn't mean you will love or even get on with their family. It sounds an awful lot to deal with but I think you handle it correctly by not reacting.