Not sure what to do at this point

[Tina825]

H and I started spending more time with each other and staying the night together and talking ALOT, about everything and he was so sweet and everything had been progressing well until two days ago. All of a sudden, H is withdrawing. I ask what's wrong and he claims that since my D(18) hasn't changed (Mind you she doesn't live here anymore) that he knows he will have to live with her actions (Not going to the college he wants her to and having the job he thinks she should have) will affect him for the rest of his life and he doesn't want to be unhappy for the rest of what life he has left. I don't get it. I asked him if he missed the OW? He says not at all. We have both been very open and honest (he leaves his phone for me to see) the last month, but it sure seems to have taken a few steps back over the weekend. I asked him this morning if we were splitting up again and he said "Why are you talking like that?" The only time we have actually had any arguments was about my children, so to me this is another excuse to argue. We did have a argument last night, he doesn't seem to understand, no matter what she will always be my D. Even though I don't agree with alot of things she does, it's out of my control. I don't get it!