If your husband is having an affair.. how would you be the Walk Away Wife? Someone having an affair.. has kind of already left you. Basically what you have allowed him to do is have his cake and eat it too. You will have to make the choice on whether you want to let it continue or not.