I feel for you and I am incredibly angry at your wife for snatching up your children from you, exposing them to her active affair, and not providing them the privacy and space they deserve and are used to. She is incredibly selfish. She wasn't running from an abusive father/husband for heaven's sake, she was unhappy and left.
Ok. Had to get that part out.
You are doing everything you can. I agree, you need legal advice with regards to visitation with your own children. Hope you can keep up your positive attitude, W will notice.