Didn't you know ahead of time that him going out was going to cause you distress? If you did then it is your responsiblity to bring it up.
You need to tell him your feelings, what would make you more comfortable, and why. You let him make his own decision but his response would be telling. How hard does he want to work on the marriage?
M22,H45,W45 S21/18D12 Retain faith that you will prevail in the end, regardless of the difficulties and at the same time confront the most brutal facts of your current reality, whatever they might be.