WCW, It takes a while for things to become "normal" again and I can understand why your walls are still up. You and your h have come a long way and there is still some growing to do.
Listen to what your h is telling you about the situation w/the female and her partner. Maybe he is testing you to see how you will respond to the comments. He very well could be trying to see what your thoughts are about the subject and this is the only way that he can see what you are thinking. They can't come right out and ask you about such things while they are on the final lag of this long trip. They hint, they suggest, etc., because they are still "afraid" of what our responses may be.
Be gentle w/him just as you would a young colt. He's still insecure and has some growing up yet to do. One step at a time.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.