Just a question. If it wouldn't be an ultimatum, then what if he said he wanted to continue to do what he is doing, and didn't want you to sit around waiting for him?
I think when you get to the point that you want to say that to him,....then maybe it should be an ultimatum. Then you would have to be prepared to follow through...otherwise you wouldn't be taken seriously.