You can call it quits if you want, that's your choice and I know it seems like he already has...sometimes...well, maybe not.
Anyway, I noticed something in your conversation with him. He came home and was talking (and in some ways probably thinking out loud) about what he wanted to do. Then, you said something like "Can't we....".
I'm not necessarily saying that's bad, but with a WAS, I think it's usually better to ask them more of what's on their mind instead of telling them what's on yours.
