SA, What an @ss! He's trying to bully and scare you into calling off your lawyer. Trust me, his threats will only hurt him in the long run. Let him stew in his pot for a while.
Just remember, his issues are not about you...he's got to learn to face the consequences of his actions. Stay the course and whatever you do, do not waffle when it comes to your lawyer and what you need to live on for both you and your children.
I'm very sorry about his behavior, but I'm not surprised.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.
