I'm serious about that - IF you think he's taking it for granted that he could just waltz back into your life if he wanted, and IF you don't have any history of infidelity yourself.
Just make sure that everything is potentially deniable or explainable - so maybe no note on the flowers - just a dozen red roses