Don't make the clothes a deal breaker...could be he's just procrastinating, lazy...oops, that's MY H!
(Sorry CJ!)
Would it help you to feel better if you just mentioned to your H that you'd really like to sit down and sort some things through this weekend? Set a time, time limit...write out SOME of your key issues, ask him to do the same?
Just thinking that SOME perception of control would really help these anxious feelings you're having.