Quite honestly, this sounds to me like unresolved feelings for his ex-wife. I'm sure he loves you, but my guess is he still loves his ex-wife. You may try to blame her for the whole thing, but she didn't force him to do this. It is a choise he made.
Because they have children together, she's in your lives for good. I'm wondering.... if he can flirt with her and have sex with her while your relationship is still in the "honeymoon" stage, how tempted will he be when things settle down and you enter the normal ups and downs and times of boredom that's typical in married life?
I would imagine the temptation later would be stronger. Unfortunately, when people cheat, there are usually a lot of lies involved. (i.e. trying to protect the betrayed person, self, affair partner, etc...).
Personally, if I encountered something like this early on in a marriage I'd probably leave.
There is no arriving, ever. It is all a continual becoming.