Don't worry. It's a ploy. You sound like a great Mom and there is noway a judge is going to hand you daughter over to him fulltime.. I had 2 friends that went through this. In both cases it was bull and never went anywhere. He's just angry angry man and for your daughters sake I hope someday he understands this.
My neighbor is a great guy and father. His spouse did the samething. We all wrote letters and he went in there being able to refute all her outlandish charges. The bottom line was that I don't even know that they were needed. Whoever did the evaluation had it all figured out before they went to court. These people see this stuff everyday and they know what's what.
They don't necessarily even have to have charges here. All a lawyer has to do is request it and cough up the money.