"I agree, they will be very disappointed. You have more say in this than you let on, in fact you have ALL the say in this. Your affair is the only thing standing in our way, but I can't control you nor do I have any desire to. [(W Friend) doesn't want you at the place because of your choice and] We can't afford this trip and go through a divorce. I would love nothing more than for you to stop the affair, work on the marriage, and continue fun trips with the family."
That's the best, IMO, yet it is a very long text to have. Perhaps, wait until you see her face to face. When will that be?