Yes! JT- setting some rules and boundaries to protect you and your state of mind is very necessary. You sound like you are moving in a good direction. Just heads up though, when you do lay down some rules, it will make him angry, just like most things do right now. That's okay. Let him deal with it.
"Success is the sum of small efforts, repeated day in and day out." Robert Collier
"One's best success often comes after their greatest disappointments." Henry Ward Beecher