Setting boundaries with your husband that you will not share him with OW is not "giving up". In fact, by attempting to be the better option you are giving him more leeway than ever. You are giving him the luxury of casual conversation that is light and airy and the knowledge that you are still around and accepting of this situation for now. Your actions say to a WAS in an affair that it is okay to take some time and choose.
Affairs are VERY powerful and rarely is the LBS seen as the better option. I know some will disagree but IMO that is the wrong way to go.