Abbey, All the feelings you are feeling, I have too. I walked a tightrope...never knowing how much to ask or how much time to spend with him in the beginning.
Our daughters are young so it was a little different, when he started to come around again he had to learn to be a Dad again and become part of the family. I would leave him alone with the girls and go about chores, errands etc. This kept him "wanting" to spend time with me...he would always aske when I would be back and if we were going to eat together, etc...
This is a risk on your emotions but it is a risk worth taking...continue to do what you are doing!