You have likely exposed him as a mama's boy already.. And no one likes those guys. I don't think anyone worth your time will believe him... If they do, you can find supportive friends instead...

Everyone knows you are working at this and he's not...

But ya, you didn't become part of his family so to his mind you were the bad apple so to speak... he likley feels similarly about his sister