You might want to pick up a copy of "Not Just Friends" by Dr. Shirley Glass. It will help explain all of the crazy feelings, which are all very normal, that you are going through and how to better cope with them. She also gives thorough steps in ways to rebuild trust. The need to ask questions is the betrayed spouse's way of picking up the pieces to get to the whole truth. Your H will get annoyed, especially if he feels badgered or interrogated, or even if he still feels some connection or loyalty to the OW. You have to realize that he will be going through his own inner turmoil as well. As unfair as it seems, your H lost someone he felt he cared deeply for. He will need time to grieve, just like you.
When you feel the need to ask, do it calmly. Don't act hateful, don't be sarcastic, and don't accuse. Realize he may not want to answer your questions for fear of hurting you even more, and try to explain that although the pain of hearing the truth will be short-lived, the long-term effect will be to heal.
Get the book, if you possibly can. It's excellent.
Take care.
Getting over a painful experience is much like crossing monkey bars. You have to let go at some point in order to move forward. ~ Joseph Campbell