Ok...well that definitely does add to your trigger today.
Since this IS for a Dr. appt., is there a reason you cannot accompany him? If not, then ask him flat-out how he plans on spending his time, aside from the Dr. office...and be honest with him. This is making you trigger, tell him that up-front (I think you already did that), but let him know some things that will help you through this day too...and bring you some comfort. Like phoning YOU and talking to you....IM'ing you whatever.
If he really does seem sincere in his effort then as tough as it's going to be for you (and I know how tough this is after infidelity) you are going to have to try to trust him today. So...what activities can you do to occupy your mind. Something that won't let you mind wander, something you have to concentrate on? BTW...are you a SAHM, or do you work outside the home?