I'm so sorry about your weekend. I'm reeling myself from mine.
I'm not sure I'd agree to the two of you constantly switching houses either. Could you work out some other sort of 50/50 where you live in your house and he lives in the apt., but he comes over certain days during the kids waking hours while you go do something out of the house?
I don't have much advice I'm afraid. I feel like I will soon be trying to figure out the same things you are.
Nature Girl M 40 H 40 M 15, T 19 D11 S9 bomb 3/07 (MOW)