Got to agree with 5 here. He blew it, probably some good excuses - got to drinking whatever. However, he should have come home after a couple of rounds. It is really easy to separate at that point without sounding whipped. "Guys, haven't seen the wife in two weeks. Going home to nail that woman!" Pretty crude, but he could have exited with cheers, no face lost and built time with you.
Hopefully this is an oversite on his part and a calm and rational (better for men than emotional) discussion about how this made you feel (and what he missed, had the whipped cream out) will probably have the best effect.