OMG girl - please stop this! You are blaming yourself because in your magical thinking, if only you had done X,Y, or Z he would have come home.
Truth is, this is about HIM! This is about a guy who cannot spend time with his daughter and actually told you he wished she hadn't been born! This is about a guy who is shirking his financial duty to his wife and child and has something seriously going wrong with him judging by the state of the house (my bet is still on an addiction of some sort).
If he had trouble with the guilt after returning home he could have said "hey I really want this to work, let's go to counseling together, let's go away on s romantic vacation, etc etc etc. instead you got a guy who came home but didn't make a sincere effort to make it up to you and isn't self aware. And is either gambling or doing drugs or spending money on some other addiction by the sound of it.
Legitimately remorseful guts who just can't live with your disapproval of their affair do NOT treat their daughters like this. Pull off the freaking rose colored glasses. You've been too much of a doormat with him already. He's not who you think he is. Let go and get an attorney to look after your financial rights.
