That's easier said than done with 3 kids and me not working...and money being tight... but yes, I know - no excuses for not standing up to him.
Stop making excuses for yourself, stand up for yourself, get a job, research your job options, possibly jobs where you can work from home or part-time when some family & friends may be able to watch the kids for a few hours if they even need to be watched, if they're old enough to watch themselves, what's holding you back?
Excuses don't explain and explanations don't excuse.
You want your life to be different, it's up to you do the things that are required to make it different.