I'm guessing the whole thing with the son will be an adjustment for the dr and the son as well. If things do go to a more permanent (for lack of a better word) level with you and the dr, it might do his son some good to be in a house with other kids and another strong, stable influence who doesn't think he is the be all, end all (not that you don't like the kid, mind you, but you aren't and won't likely be as enamored of him as his mother and grandmother are and that is a good thing).
Me 52, H53 Bomb drop 9/29/2014 Divorce from XH final 12/17/2014 Marriage #2 12/31/2019 5 adult (step)daughters (3 from XH's first marriage, 2 from current H's previous relationships) 6 grandkids