I know you are scared. I truly believe all that you had said that he is. His actions show that you have every reason to be concerned. That being said the judge will not care how you feel emotionally about all of this. He will slap H on the wrist for non-payment (have you filed for non payment with the courts?) Failure to pay child support is not always means to deny visitation. Remember that the judge will look at this in the best interests of K and typically having a dad, whether or not he pays child support in better than not having one....It could simply come down to who he likes better in the court room. If your H manages to come across as a normal dad just down on his luck right now and you are percieved and the wronged wife, he will get more time with K. That is why you need a lawyer to speak for you, so it is all business and not emotional.
Did you take pics of K's flea bites as well as document he isnt feeding her properly? Was there anyone besides you there when he returned her to you this way???
if you throw all of this at a judge you may get lucky and he/she will wade thru it all. But you may not and he may disregard all of it. It is the luck of the draw when you are assigned a judge.
Please do yourself some good and get some peace of mind. Find someone who can help you sort thru all the garbage and figure out the best way to present the absolute worse parts of your H to the court so that you can maintain the agreement you have. You need to show your H now that when he pulls this crap you will fight back with everything you have available to you so he knows he OW and their "paralegal" cant win against you.
Otherwise, he will be dragging you into court every 6 months trying to get more visitation so he can lower his cs payments. Use the anger you are feeling now to propel you forward and stop thinking that if you play nice so will he. He wont. He is backed into a corner with this new baby and OW and because he isnt contributing to the household income he is trying to lower the bills he does have which include K's child support.