To my mind threats to leave when you are miserable are just childish and passive aggressive... I take it as a statement of his mood more than any plan he has in the works...
When people are feeling miserable an hopless they want to express that... He will say the marriage is over, its failed, he's done, he's gonig to leave etc...
You just have to remember what he's TELLING you when he does these things... and ignore the words he's using to express it