You've allowed your H to get you right where he wants you. My W has done this to me before - I found out about an EA, and confronted her with the phone calls and texts. She then insisted that she had done nothing wrong, they were just friends, etc, and she used my outburst/anger to file divorce. She was the same way - very matter of fact, very calm; because I think they enjoy blaming us for something even though they're the ones in an affair and destroying a marriage.

First of all, unless your state is not like mine, it's VERY rare that an order like that is issued; or if it is, it's short-lived. My W has said the same thing, and I've heard it said to many others. It is mostly gobbledy-gook meant to control and scare you. My W's attorney filed on ridiculous grounds and asked for 60% of my gross income - it was a joke, just meant to scare me. It made me actually both get POed and laugh at the same time.
The courts are much more likely to take into account an affair - it's VERY heavily frowned on in my county.

So don't freak out. You messed up. It happens. Take a deep breath. Personally, I would wait a couple of days to respond. H now thinks he has you running scared, and is gonna push it hard. If you calmly respond with the appropriate information, but wait a couple of days, you take a little wind out of his sails, regain your composure, and get back in the saddle!