Growing is good. I don't know if you need the letter for him as much as you do for you. Writing down what you want to say to him will keep you and track and make sure you don't forget what you want him to know.
Yes... those are flags. Maybe yellow and not red, but flags none the less.
Regardless of whether or not you give him a letter or just talk to him you need to do something for yourself. Remember, you deserve to be happy. Not just him. This is a two way street. You can't make him happy. You can only find ways to make yourself happy. Get back on the road BBJ. You know the direction to take. I know you do.