IMHOP I think that you should keep in contact with your in-laws no matter what. This is not about your EX It is about a woman who is not doing well and who would like to see you. Keep your head high and don't listen to your EX. YOu are a person with feelings and his family did not divorce you. The obviously enjoy your company or they would not ask you back. Listen to them.
My in-laws have been my biggest support through this whole mess. I talk to them more than my Ex H, He sees them at Christmas only. I see them all year. I'm sure he is carrying alot of guilt around after what he has done. That's his problem. We were toegther for 32 years and I will not give that up , his family is and has been my family for a long time.