First off...thank you for your honesty about the grinin' guy.
Smiley dude is not your answer. Just like the OW was not the answer for your H. You are not getting what you want from your marriage niether is your H. That's how affairs start.
What are your needs? What are your H needs? If those needs are met affairs will not start.