You can always start with mediation (let him pay the fees) and then resort to L if it doesn't work out?
If you truly don't think it's best for the children to spend half their time with him, then document how much (little) time he has spent with them now, what you have done to facilitate them seeing him, and the manner in which he left them behind with you. This could help you show in court (if you end there) that his wish for 50/50 is based on financial considerations and not a genuine desire to maximize time with his children.
Also, if you are in a state that allows it, you could record his statements about threatening to go for 50/50 unless you do what he says. Also make sure you tell your L about it. This kind of documentation can be crucial later, because he is really digging his own grave here.
M 16 yrs, WH62, P54 3 adult blended kids EA 11/13, BD1 6/14 PA fall 14, BD2 2/15 Piecing 2015, BD3 12/15 Separated 4/16 WH moved OW in 5/16 Divorced 6/15/17