I agree with Woog. You have to keep doing things with your H because of the kids (at least for now). But make it different, don't be mean or anything, make him feel like he can "look but he can't touch". Distant is the right word. See if you can do that, if you can't, we will figure out something next time...