What you are saying to Heather, is IMPO...correct. Heather, it's true...you made a mistake in your M, you know it...your H knows it, no need to rehash right? Your H has also made some big mistakes in the M.
BOTH of you in many ways are the offended spouses (as Chrome put it) in this M....BOTH of you need to commit to making this work and BOTH of you need to be pro-active in moving things forward. I don't believe one spouse can mend a marriage, it's going to take both.
I get this feeling that both of you are waiting for the other to somehow make this grand gesture towards the other that things WILL get better and that you are whole-heartedly committing to your M, but neither of you are seeing that. So, neither of you are wholeheartedly committing. You have done what you feel you can I believe but your H still holds you at arms length, that's frustrating for you (and of course it would be.), so you get tired of trying when it doesn't appear he's doing anything. Your H...seems to be waiting for something from you to show him that you have true remorse and will NEVER do this to him again. However, he's not telling you what it is that he needs you to do, so you have little chance of giving him what he needs (just as frustrating and scary for him.) It's really hard (speaking from my own experience here) to put your trust back in someone and be vulnerable enough to have a good relationship with them...when you are still afraid that you are going to have to endure that horrible pain again in the future. This I think adds to his sense of entitlement of "you hurt me, now you owe me." and it doesn't help.
Just my 2-cents here....but you two are planning on building a house together, big step right? Personally, I believe you need to sit down and have a real hear-to-heart with him and work out some type of a pro-active plan towards coming back to each other instead of pulling away (as you are both doing.) Date nights (regular ones), sleeping in the same bed, common courtesy, spending time together...etc, whatever it is. I believe you need to agree to these things before you agree to move into that new house. But that's just me.