If you are convinced that he is having an affair (and I agree with you that the odds are very good that he is, based upon his behavior), why are you still having sex with him?
I hope you are at least having him use protection. I'm sorry to be so blunt and personal, but this is really no different advice than your doctor would give you. Please be careful!