I look at it this way (about not wanting to be at work), if you were home, what would you be doing? I tend to wander around the house, can't really relax, etc, so may as well be at work, you know? As long as you can peek in on us, then you are good to go.
Its a low day for you, and I'm sorry. Hold off on calling anyone (OW and OW's H) for 48 hours, so you can think about it. Remember, usually confronting the OP doesn't bring the confronter any relief, especially if the OP is emotionless. Now, the OW's H? That's another thing. I am still grateful OW's H contacted me, I wish he had done it sooner. But in my case, I knew something physical and emotional was actually happening. Your situation is probably EA only, and who knows where your H is in his fog, he might be trying to space himself from her, and you contacting anyone might really rock that boat.