I'm sorry to read about your situation Yanni. But you're right this all boils down to what is most important here....the answer is you and your son.

I don't know what's going on in your h's mind, but he's not very grown up is he? It strikes me that he wants you and son at home waiting and he wants OW to go out and have fun with. Like I say I don't know just a hunch.

Perhaps you need to have a good think and set some boundaries. You need to decide what's best for you and get out there and do it! With or without your h.