The only advantage to giving him this letter, was to SEND it to him, to spoil the illicit couple's trip. Affairus interruptus.
Now that you're waiting until he's back, I think it's all hand-tipping, and not necessary. Just withdraw the funds you need, and when he asks you about it, say "It's only about a tenth of what you've taken out to spend on your affair, so please don't lecture me on the subject of squandering marital assets," and leave it at that.
What's the upside to informing him of all of us? It also makes it look like you intentionally waited until he got back, which is WAY too much deference and respect than he deserves!