I may be off the mark here, but in reading your posts of recent I get a sense that you feel some threat to your emotional well being with the bullying from your H. I can totally understand this. I feel this way too right now, although for different reasons. I really don't feel "safe" emotionally right now with H especially on a one on one basis. It's hard for me to explain. And has nothing to do with physical safety what so ever. It's maybe feeling too vulnerable. So with that in mind, I have been working on some new boundaries with respect to my H. I have really benefited from the discussion on these boards about how and when to set boundaries. Why they are for us and not to be used as punishment or for manipulation.
In my case, I no longer do the "family dinners" we once had with the five of us. I encourage our boys to spend time with H but I don't participate. My boys are older 19,21,23. If H plans to be at the house, I make sure to be gone. I only answer texts or emails relative to the boys or the house. My H all along has wanted to be "friends". I told him I could not promise that now. (I know our circumstances are very different) But my point is that setting stricter boundaries for myself helped me feel more protected emotionally. And in your case whatever boundaries you can set for yourself and enforce I think would help you to feel less vulnerable and more empowered. So you don't feel at the mercy of every whim of his bullying. I dont' know if this makes sense or not, but for some reason your emotional place resonated with me.
And now, I can kind of see that a physical move but actually be very helpful for you. The physical distancing might be good. Maybe it will also be easier for your boys, to be farther away from him. It will be a natural reason for if and why they are not seeing him much. May be harder if your H was close by and did not see them.
Stay strong and beautiful. My own adopted motto. Best,