He comes in and makes himself at home? Lays on your bed? Oh heck no. He CHOSE to move in with OW. He LOST his privileges to the family home. Legally he has abandoned the family home. He has ZERO right to do any of these things that he did today. YOU have to establish boundaries and hold him accountable. He's got the run of your place and his OW on the side. No incentive for him to man up and be a decent human being. You're now enabling his A. Please don't do that to yourself. You deserve better. He has to call/ask before he comes to YOUR house and when he does you meet him at the door and ask him what he wants. If it's some clothes, or whatever, you tell him to wait there, YOU go get them, hand them to him at the porch and ask him if that's it. If it is then tell him you need to go take care of YOUR family and close the door.
Do that and you might save your marriage (if you still want to, I wouldn't).