Just tell him if he cant' respect you and your family enough to be fully honest and committed there's the door.
And let him go.
He is THEN forced to CHOOSE.
This "don't ask dont' tell" thing does not work I'm afraid. First it's chit chat, then eventually one hook up, then another...
He must be committed and transparent or he has to leave.
Stop talking about going to jail and saying he's not worth it. Stop inviting him to punch you for goodness sakes. THAT is not going to improve this situation.
Let me tell you this plain and simple.
It is YOUR JOB to do everything you can to STABILIZE this while maintaining dignity and respect for yourself and your family.
You have every right to confront him.
You have every right to expect 100% commitment.
You inviting him to punch you does not result in you being more respectful or dignified to you or your kids.
Just calmly, quietly, and resolutely TELL him to LEAVE.
Let him know he is NOT welcome in the family home again unless he commits to you AND your kids to be 100% committed and to END ALL CONTACT and BE FULLY TRANSPARENT.
It's great that you confronted him. It's terrible how it went down.
I am so sorry you are hurt, betrayed, deceived, and manipulated. But you have to get a hold of yourself.
Inviting violence into this is NOT an improvement.
Tell him to leave BEFORE it comes to violence. Tell him his behavior and infidelity is toxic to you and the kids. Tell him until that stops he needs to stay away.
Calmly, quietly, and resolutely. And then end the conversation.