When you say it is common, how do people put the newcomer out of their mind?
I think it's common in a sense that when your M is on the rocks there is a lot of void that is not filled within your M and it's natural to want it filled somewhere else.
You need to realize that that's what's happening and stick to your boundaries.
As far as getting them off your mind... why bother. You can entertain yourself with whatever your imagination can offer, as long as you have no intention of acting it out.
