If you feel that your conversations are taking you backwards versus forwards, then maybe you might consider reducing the number of conversations that you are having w/him. You can be civil and return one or two calls and if he asks why you didn't answer/return his calls, you can always say that you've been busy...which you have been.
From what you've posted, he reminds me of someone traveling down memory lane and is reflecting back on his life and the home he had w/you.
Your h senses you are pulling away and creating a new life w/o him in it. They have excellent sensory skills and when they sense that we are pulling away, they tend to step up their game to pull us back in. Unless your conversations are about finances and selling the home, try to minimize those conversations.
At the end of the day, only you can decide what is right for you.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.