Normally I would advise you to close all joint accounts and come up with a monthly figure that he will transfer to you (by check or EFT) for you and the kids to live on. If he agrees to this, it sets a pretty clear precedent for child and spousal support in the future.
In your case though, since he hasn't filed for D or anything, and he is being financially responsible, it may not be worth it to rock the boat. That is, assuming you didn't find anything funky when you ran his credit before.
My guess would be that he is skimming cash to use with OW-- cashing odd checks instead of depositing them, getting cash back at the grocery store, etc. But for the most part he is trying to do "the right thing" by his wife and kids financially, to ease his guilt about the affair and what he is doing to you.
You should probably still open your own checking account.
