I think you handled the situation very well. Sometimes, things do pop up and we have to step back and assess them before going off half cocked on the spouse, i.e., just like this situation. You stepped back, thought about it and then advised your h of the situation. He didn't feel any pressure nor that you were accusing him of anything.
Great job!
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.