I would never punish your D because of your W. So by all means go see your D.
" Am I supposed to just stay married to someone who doesn’t want to see or talk to me."
No. You aren't. But don't make a decision when you are emotional based on something your W said. Do you know how many times my W flipped-flopped between staying and going?!? I had to be balanced, steady and sure through her flip-flopping.
Read sandi's rules again. Institute them.
Most of all, do not let emotions govern your actions.
M(53), W(54),D(19) M-23, T-25 Bomb Drop - Dec.23, 2017 Ring and Piecing since March 2018