Have you asked for space? Have you said that you feel very wronged and just up and being friends feels like your perspective isn't being honored at all. Would it be a horrible idea to express that sentiment in a kind and loving way. Tell him you appreciate him being a good daddy but as far as being buddies, it's just a big thing to ask of you right now.
