For starters, it treats them as equals, and that shouldn't be done. Would POTUS sit down with some third-world terrorist band leader? It confers credibility upon them that is not deserved.
Also, the betrayed spouses in the equation inevitably treat the info they gather from talking to OM/OW as at least partial truth, and try to base decision-making on it. It is spin and obfuscation and deception at best, and COMPLETE LIES at worst.
Such a meeting will make you look weak and appeasing.