You need to give it time and he needs to prove she really is out of the picture. He needs to earn your trust again. Maybe the fact that you reacted the way you did (not insanely happy) will get through to your H. Try to keep as detached as you can still and let him to the work he needs to do to come back.
