Your husband is in big time withdrawal. Keep doing the things that you are doing because it is helping him to get through it.
He is going to take his anger out on you because he is going through withdrawal/OW withdrawal at the same time. That is the reason for him being in the lower bedroom. All I can suggest is once in a while do invite him up. If he says no, then leave it alone for a while and then ask again.
It helps whether you think it does or not for him to know that the door is open for him to return, but there is no pressure from you. That you are okay, but yet you are still there.
You are handling things well and this is going to take some time for him to get through. This is much harder on him then you will realize at this time. There is so much going on in his head and that is why you don't get responses. Not because he is trying to ignore you, but because he is so wrapped up in his thoughts.
Let him be to get through this. You can not make it go any faster then what it is. Everything is being brought up that he did by the Lord. I mean everything. No holds barred. He is seeing everything that he did and that is alot to deal with.
I remember going through mine, I didn't want to speak to anyone or be around anyone. I wanted to hide in my room and just think and cry. I had to face things that I didn't want to face and it did not give me a good feeling. I snapped at people for the dumbest things as well. It took me a good 3 months to get through the hard stuff before I started coming out of it.
There is no telling how long it is going to take your husband because there was OW involved. It is very hard to face the real you and all that you did. He is not ready to go to speak to a counselor either. He has to deal with this on his own and in his own way. If hunting/fishing is what he has to do, he will do it. It gives him more alone time where no one is there looking at him or watching him. It is him by himself looking at himself with no destractions.
I would not want to be in your husband's shoes right now and my heart goes out to him for everything that he has to deal with. Keep lifting him up in prayer asking the Lord to give him the strength and the courage to face himself and give him the guidance to make it through.