Ok...lunches were just a thought if doable. Just out of curiosity how did you hear the rumor that he'd been having an affair for possibly a year? Was it from the annonymous phone call or from someone directly?
His moodyness may possibly be coming from the mere fact that he isn't contacting this OW anymore, that is always possible....so don't be surprised if he starts blaming you for things either, small things...insignificant things. I'm only guessing here, but it would seem reasonable to me that if he isn't in contact with her...that he may be missing that contact, and well....you would be the reason he's missing that contact with her...therefore he might form resentment towards you on that end.
Don't despair, these are things that can be worked on in your therapists office. Things can and most likely will get better for you.