ugh.. I feel the same way even though my H's EA is still going strong! I feel like I will push him back into PA if I do anything wrong! It's horrible walking on eggshells like that.
I think getting rid of that feeling just takes time when you are piecing your M back together like you are. When you build up more trust in each, other you stop waiting for the hammer to fall because you are more confident in your M.