So so new to this :(

[Grace21]

I'm sorry you are here and going through this unwanted journey. You are in good hands here, as there are kind, considerate people that have successfully navigated this journey that will give you good advise. Here's a few things:

Now I want to call him up and confront him, or at least let him know that EVERYONE knows now too.

This is useless. I did this to my H when I found out he hooked up with his OW again after he moved out. They will downplay and lie, so don't bother. He certainly knows that people are aware of what's going on. He probably doesn't care.

So before acting I'm asking you all, should I say something or just not react? It does bother me because I was going for a visit to my hometown in June, but now I really don't feel like going. I don't want to answer questions about my marriage.

You have nothing to be hiding from. This behavior is on HIM, not you. Further, you don't have to answer any questions about your marriage. It's nobody's business except yours. If questions arise, you can just say "it's a private matter", and change the subject. Don't let your H's behavior dictate your actions or plans. Go on your trip, and have a terrific time!