I was about to say the same thing. The sentence formation you use when talking about what is essentially really your own decisions and behavior is often passive and places the verb on the wrong person. How you communicate...how you think...what you believe are inter-related.
HE is not holding you back. Your decisions and behavior with regard to him are holding you back.
HE doesn't bait you. You react to him in one way but could react to him in any other way that you choose.
It's good that you're taking ownership of your life, keep it up. Even when you were in a relationship, and when you are in future ones, that is still what you need to do.
Adinva 51, S20, S18 M24 total 6/15/11-12/1/12 From IDLY to H moving out 9/15/15-3/7/17 From negotiating SA to final D at age 50 5/8/17-now: New relationship with an old friend __ Happiness is a warm puppy.