1. YOu need to find a good friend or family member who is marriage friend to act as a mediator.. that friend can allow your H to see your Daughter and you don't have to be exposed to him at all... This also presses the attachment fix higher as well.
2. You don't need to talk to him at all if you are with friends or family... let him send messages through THEM... Did you read the protection phase ebook I posted earlier?
3. Long term you may haev to go home yes... But for a few weeks you can leave H sweat.. I don't reccomend even talking to him.. let him contact you through an intermediary... it will reduce your tension AND put a lot of pressure on him which is what you want.. you want him in a panic and sweating bullets. Don't use phone or text message, strictly email him... ONLY email.. no phone or texts. And you can always get your friend or family member to screen the email for abuse first... whcih again is something I reccomend.
4. Start putting together a plan of how you want to attack this long term... Family Therapy, a Transparency Contract, Living arrangements etc... just write down how YOU want things to proceed long term here to deal with the situation. I assume he hasn't agreed to end the affair yet?