I wish I had the answer for how to deal with all of our ongoing worry. One thing you need to do is to remember that as far as you know, your H has not done anything in the last while (one month?) that indicates that he is still involved with the OW. Think about all of the times you've worried, only to find out later that you were wrong, and there was a completely innocent explanation. Also, notice that your H did not change his passcode. I think that is a very good sign, even if he is mad about the erased message at the moment.
Trust can be rebuilt, but it's going to take time, I'm sure. I've read 2 to 5 years. It's only been, what, 2 or 3 months for you? Realize that you are not alone. We are all going through these feelings. It can be so weird. Yesterday my W did something really sweet and loving, and then went upstairs alone. In about the space of 20 seconds, I went from feeling great to having evil thoughts that she may be going to call him. No, of course she didn't, and it was stupid of me to even think that she might. But that's just the way it's going to be for awhile. Be patient, it's going to get better.
When these panic attacks come, focus on what you know, not on what you don't know. I know you try to be positive when you are around him, but from my own experience, it's very hard when you are still trying to recover from the latest panic attack.
I don't think writing him a letter is a bad idea. I think it can be helpful as a way of organizing your thoughts and being able to say things to him that might not come out right if you just start talking. If you do write a letter, the important thing will be to not accuse him or blame him anymore. He knows what he did. No point in rehashing any of it. But you can gently share with him how fearful you still are. Let him know if there are specific things that he can do to help reassure you (for example, call home more often, tell you specifically where he's going, etc). Ask him for his help and try to make it a team effort, so he doesn't feel like you think of him as the enemy.
I don't know if this is helpful; I obviously struggle with the same stuff, and I feel like a bit of a hypocrite giving you advice about it.
Yesterday was another good day. Lot's of affection and togetherness. Today is a rare day in which both my W and I are working, so we drove down together and I dropped her off at work (we live north of Los Angeles and our commutes are to roughly the same area south of LA).
I hope you have a good day and evening. Remember all the good things that are happening between you and your H.