For me the main reason to ask them to leave is if it is harming you emotionally more than you can handle or that they are in someway violating your boundaries.
I would agree with this ^^^^^.
I am in the same basic situation as both of you. But now there is a clear violation of my boundaries that has to be addressed. I only see this being addressed with a physical separation, which to me is not a tool for R, but a step toward D. (sadly)
If you are unable to detach while he's at home, then it seems to be a perpetual cycle of negativity. Again - this is my opinion and my reality.
Me-45 W-44 S21, S18, D15 T-27, M-21 BD Jan 2014 PA revealed March 2014 In-house separation - April 2015 I filed - Aug 2015 She moved out Oct 2015