Like the others, I think that YOU have to make this decision, and you need to have a peace about it. I will only add this:
I cannot think of a single instance where I ever felt like I should have taken a SOFTER stand. Most of my regrets in my marriage, in dealing with my kids, and in my life in general have been that I wished I had been FIRMER and drawn a stronger boundary.