If you feel that you won't be happy on your trip with him there then maybe you should ask him not to go with you. I know this may sound drastic But you need to think about you and DD. Let him deal with the consequences of his attitude toward you.

JAK

You don't get to choose how you're going to die. Or when, you can only decide how you're going to live now. ~Joan Baez