You don't have to tell her anything. "Going out with a friend see you later". That's it. Be polite and respectful. She might press you. Just go. Don't let her drag into a confrontation.
As far as the coworker who cares if he's the office good guy. He's a problem in your marriage. Who cares what people think. Find your value. Find your confidence. If he asks you what's up just smile and say "nothing evreythings great". It seems you're struggling with your confidence bud. That's going to keep pushing your wife away. Are you more interested in solving you're marriage problems or worried about what people at the office think? As long as you're courteous and respectful and in good spirits no one will think anything less of you.
