(Also check with your lawyer, does having him living there set back hour separation date for divorce? Is there any other negative legal effect? )
Yes, this might be a possibility. I checked with a lawyer friend and he seems to think that because the separation agreement clearly mentions separation date it will not be affected. If H argues differently, at least the fact I have communicated the separation to my friends and family will be able to support my case.