It's very normal, BigMac. I am consumed by those same fears.
What I've been doing to cope is to already assume divorce. For example, start googling some articles on how to raise kids after a divorce. Once you accept that reality and get a handle on the logical and tactical things you can do to make the best of life post-divorce, then you set your mind to be better prepared for the worst case scenario.
Prepare for the worst. Hope for the best.
M: 15 years BD: 6/25/14 EA/PA: starts 5/14/14 11/30/14 - A ends 5/15/15 - D is finalized. 11/28/15 - Start of new LTR with a wonderful woman (and still going strong)!