He's feeding you a line and putting the blame on you. Don't buy in to it! It was always a fear of mine not to pursue a career or life outside of the home because that would cause my husband to feel neglected. Joke was on me because he cheated anyway.
Ignore his lines and get to working on you and what you want.