I feel like I am missing out on some secret club or society thing, myself.

Ian, have a GREAT time with your girlfriend, and know that you are an awesome father with very normal feelings. Use them to guide the amount of time and attention you give each facet of your life. It is all about balance.