It is a personal decision. But you need to know and accept the consequences of whatever decision you make.
Start by seeking advice from another lawyer--letting her/him know you do not want a divorce.
But if you Stand and do not file, you may sacrifice financial security. That is a fine thing to do if you are willing...but kids make that a tough choice. I didn't have kids and was willing to lose anything financial. Without kids, the level of risk wasn't so great.
But the answer isn't so easy...yes or no. There are consequences to both. Can you file for separation instead? And if he can remove the money from the account, why can't you?
I don't know, but consider the risk of not filing and not reacting...not fearing his angry outbursts, but being kind to all shows of meanness and anger. SOME will cease the anger because it was a test. But this is not true of all.