"I appreciate your invitations but I find myself struggling after spending time with you so I'm going to step back for a time to take care of myself." Or something along those lines.
I agree with this approach, or perhaps an even stronger statement. Don't go to him with your feelings about the family events until he asks to be a part of another one or invites you to one. Also, I am not sure I would mention that you are struggling...this could come across as needy.
Maybe even a simple "no thank you, but you have fun" or "no thank you, i already have plans" when he invites you. Your actions will speak louder than your words. Or if he asks you, just say "I don't believe that that would be appropriate at this time".
Me: 40, W: 40 M: 15, T: 18 D - 10, S - 7 D announcement 6/7/2014 A discovered 7/20/2014 (but denied by W) Still living together and sharing same bed
