You don't suck, JustD. Though, you are right, this whole thing does suck!
I've written a couple different replies to you, and none of them really sound right. I do think having time away is important ... and the challenge is the trust to let DH go away alone too (wouldn't that be a major 180 for you?). But did DH say he wanted to go away, or did he just say he knew already it wasn't a possibility? Are you jumping into the guilty frustrated thinking when you don't really need to?
Sorry, this doesn't quite sound right either. I hope you are finding some energy to forgive yourself tonight. (Something I suck at big time!)