I agree with the others, he is projecting his guilt on you. I don't know what talking or letter writing would accomplish, sounds like it would be more of the same.
I like the simple statement, don't defend yourself or explain. It is your actions that will show him what he needs to know. How you talk about him to others, to the children, that will filter back.
Keep it simple, conversations light hearted, and stop his train of thought with the I'm sorry you feel that way, you are the only one I love. I think some time will let him work through this negative wave of thoughts he is having.
And Ellie's idea of a visit to Seattle seems like a good one, it sounds like when you have the weekends away it always goes well.
