The infatuation WILL wear off and you can bet your bottom dollar OW's son will really start to play up and annoy your H as wel....and when that happens that can only help. OW will side with her child and your H will compare her son in an unfavourable way to your children - blood IS thicker than water.
You just need to get through this awful painful stage. Look at my time line....it took a fair while.
Saffie me 46 H 46 M in 1986 D20,D18,S16,D13 H's A 01/05 to 07/06 H recommitted to M 07/06 renewed vows 09/06 Going from strength to strength
