It sounds as though you and your H are just wired differently.
I, too, grew up as the oldest of 3 with no time to be by myself, but crave it now. My H also MUST have alone time to decompress. It's become imperative in taking care of a little girl who is developmentally disabled.
From my helicopter, I just see that your forest is riddled with trees that represent misunderstandings.
Michele makes some great points on this very topic in her books and the video tapes... about the fact that we each have unique needs and deserve to have them met.
It's easy to feel resentful of those needs when we're not taking care of ourselves first.
I'll bet if you were to ask him if it was okay to go hang out with a bunch of chatty girlfriends while he watched your S or made arrangements for childcare, you would be a little more amenable to seeing things from his side.
Have you read Mars/Venus by John Gray yet? If not, I think it would really help clarify some of this stuff for you.
Relax... no deal breakers here.
Bets
"There are only 2 ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle."