This is probably the 'toughest' 1st post I've ever written. I'd do the childish "I'm #1!!", but your post really made me want to get to the point.
You didn't DB well at all in that visit.
First, you went over there to clean, but keep your expectations high about it. Why expect more of a guy who has had 2 OW cleaning/watching his kids for him? Yet, he was able to thank you for your efforts finally, after you communicated the need to, and then you're rejecting his gesture of appropriate response? Come on now! If you won't lead, why get into any interaction for him.
Second, to go into another's place, and lay claim (a cat) to anything is dangerous ground. I can see why he bristled.
Third, the two of you have such sizable hurts related to your children and the miscarriage. IMHO, you're going to have to stay amazingly positive about this topic, and let the past go. Let it go, Jo. If not, the sitch is dead in the water. And you're children will suffer from even a lack of friendship or decency b/t their 2 parents.
Fourth, avoid the comparisons - e.g., I keep house so much cleaner. Either call the authorities about his filthy place out of concern for your DDs, or accept this as his way of being. I'd avoid cleaning for him, as that's not an avenue for respect, and he needs to grow out of his way of manipulating others, facing his reality himself.
Fifth, your DDs are very hungry for you. Their argument over a dress was about having you. It seems like they don't quite believe that you have enough love for all of them. Its not about the dress, Jo. Its about you. They love and want you.
Finally, I don't read that Andy's shunning you sexually. My take is he likely doesn't want you holding your possible pregnancy over his head, especially with the battleground your children (as a topic) have proven to be.
Move back toward a quiet, peaceful stance, Jo. If you're not ready to be supportive to him with no expectations, move away and give him space so that he has more energy and focus to deal with his father's grave illness. This is not a time to be looking for him to address any of your needs or to reassure (about OW2, his attraction to you, etc...).
I'm sorry if this post is rough. I've just seen you take steps that have reduced the progress you've made, right when Andy is showing glimmers of appropriate behavior and is finally alone (perhaps) to begin his own growth work. Try not to undercut this work of his with combativeness, Jo. Be a positive peaceful presence in his life. You'll best be able to do so if you work hard at self-care.