I didn't say that he'll stop seeing the kids, but he'll stop doing a lot of those fun things that he may be doing w/them right now. He may have the kids over and he could very well sit and watch TV and leave the kids to figure things out in the way of entertaining themselves or he'll plop them down in the front of the TV and do something else while they sit there watching TV.
You need to put a proper visitation schedule in place or this type of jerking your around will continue.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.