Well done. You need a break too. I use to hate it when H took them overnight (he sent me pictures the first night - I think he thought he was being kind) but now, I love having the time to myself.
Don't think of it as building memories without you. He is just the babysitter.
