Karen,

I think that is just AWFUL that he puts on your shoulders that he feels you are sacrificing your relationship with him by seeing your parents 4 times a year. Family is SO important. He sounds like he is very selfish. He should never ask you not to spend time with your family. As I see it that will only lead you to lots of resentment because one day that won't be here anymore.
Next time you want to go visit them I would leave him home and let him sleep the weekend away.

I also think his sleep excuse is just another way to avoid intimacy. Yeah again he uses an excuse and tries to put blame on you for sacrificing your sleep. Because why you want to spend a little time being intimate with your husband.

Karen I think you need to just try and put this issue aside maybe and just really begin to work on yourself. That was one of the best things I ever did. I never thought I could ever say these words but my marriage has really been turning around. My husband who I thought was the most LL man out there in the world now has a higher drive then me. Is now telling me I had better not fall asleep again. Sad thing is through the toll this takes I have become LL.