Cat, I just want to throw this out there to you. It is not about the financial burden, it is about doing things the way that are most comfortable for YOU, not him.
Take all the time YOU need, do it the way that you are comfortable with and forget about the rest. It's not about whether he squirms or is uncomfortable with the pace, in fact he has no bearing on how you proceed at all or what so ever. There is no timeline and this is not a race.
It is amazing how they try and turn it to guilt for us in order to push us forward with what THEY want to do. Don't respond to his nonsense, don't get into these silly conversations with him, and do not allow him to vent his own frustrations on you.
It is time for you to be courageous and strong for yourself.