Sound like your doing well (all things considered). I see the communication between the two of you as a positive (regardless of OW). You seem to be his safe place and this is good.
You do seem to have a good attitude and are well informed. That helps so much.
As far as the ML Was2sad is right, you have to choose. If you feel you are not being true to yourself then by all means set the boundary.
