Diane...between two people who have any sex drive at all...then NO it is not difficult at all to have a sexual relationship, and it seems totally normal and natural.
To a person such as your H, they can convince themselves that a sex drive is totally unnecessary.
So your question to all, will be answered differently by every one, based on their own sex drive.
However...those of us who are or who have ever been sex-starved in our lives DO understand your question.