Hope ~ Those times away from the kids are so hard for me too. I fear the worst. I feel guilty and at fault for putting them through D. But... I choose to believe that whether their parents are M'd or D'd is waaaaaay less important than showing them I love them and trying to be a great dad for them, imperfections and all. Your love for D4 will always be there no matter what your marital status is. The quality of your time with her is way more important than the quantity.
I think everything you describe -- difficulty being present, the fear and guilt -- it's all normal and natural given what you are going through. Hang in there.