I would much prefer if it didn't happen, but I'm trying to be realistic and look at it if the roles were inversed: if he said something that unsettled/upset me I could possibly want to drop the subject and take a break in talking altogether until I get over that feeling.
Is that a wrong way to look at it?
Accept that there is only one thing you can change in life and that is you