Please do not feel guilty for having finally found your voice. You have been so patient and have been putting up with so much for so long.
I am amazed that you have been so quiet for so long.
It's OK to rock the boat.........just don't tip the boat over!!
I do agree that your Husband needs to be shaken in some way. His self entitlement is making too many people suffer. Not only you and the children, but even the OW and her family.
He has built a house of cards, and eventually they will fall. This is probably something along the lines of what he is talking about. Someone is going to get hurt.
I know you want your home to be a place of sanctuary for him, BUT there are absolutely no boundaries for this Man. He may come and go whenever he pleases.
There are no consequences for him, no rules.
It is NOT for you to punish him, but he does need to know that you are no longer going to tolerate being treated like second best.
I have no doubt that the Man loves you, and that he has "issues" right now which prevent him from seeing things logically.
But even a dog knows not to [censored] in his own house.
There can be no testimony without a test. I am praying to go through this test and come out the other end with a new and better marriage then before.