Have patience. Maybe, like you said, you'll get a little more time together when your kids get a little more independent. I've decided that my wife and I need to go out once every week without the kids. It is helping.
The most important thing a father can do for his children is to love their mother.