Sorry, just rereading and I have to say one more thing: finances. This is different from his interest in personal involvement with her because it can be legally enforced. Somehow you two will need to figure this part out. I know it's touchy because he's been unemployed, but he has temp work now so he should offer something.
My only advice for it: get your agreement in writing. A contract, as it were. At least then you have some legal standing for whatever amount you guys come up with.