I can only imagine the pain of infertility. A friend of mine just did her first ivf and is waiting for the pg test. It has been a miserable journey. For the record - she NEVER sounds as if she enjoys sex for its own sake. It is awfully sad. It is sad to want a baby so much and to have it be so difficult and it is equally sad that the experience can take love away from the very people who want a child to express that love.
All of that being said, for the good of the M you need to look into your heart, your emotions etc... to find and remove the blocks to enjoying whole hearted sexuality in your M. Do it for you. Sometimes it is painful to look at the things that may be impeding the sexual R but if we plan to live a lifetime with our spouses that is the way the cookie crumbles.
Yes, most on this board are HD but I also experienced LD when I was in a very troubled R. It is difficult in its own way.