Hmm, I may have missed something but I think addie you did make changes and you are not complaining about your life (apart about your H and not even complain anymore). You moved right? You made all changes necessary to get to a stable and good environment for you and your kid? Dont I remember correct?
I dont think you sound controlling, you do sound in control of your life and only your life which I think is great and the right thing to do.
I am glad you H was caring and loving. When you said you set the boundaries while he was there, were those about the future plans or about your daily contacts during his visit? K