I think it's OK to think of possible future events, like your H deciding to go back to OW, however, don't get caught in the 'What If' mode and second guess him. I had a bad habit of doing that, so now whenever I see my H looking sad, or uneasy, or get "the look" I occupy my mind with anything to NOT second guess that he's thinking about her and wondering if he was feeling that he made a mistake coming back to me.
For me, if it ever were to happen, I would not know what to do, I do not hold myself to expectations like that anymore-no ultimatums of that nature anymore, (at least not yet). I always said I'd leave him if he cheated...he did...and here I am, so what would I do? NO CLUE...but like you said, I'd probably tell him I was sorry he felt that way.
I have thought it through as logically as I can to be prepared IF it ever comes. But I have long since stuffed those thoughts-they were not helping me heal myself and our marriage. I still struggle with acting 'as if'.
You are smart enough NOT to dwell on this, as it is a waste of time. Listen to yourself, and focus on the good things.