Like anything, set your boundaries. In a simple sentence tell him what you've said here.
Have you been paying/billed for the interim sessions, emails, phone calls after the initial consultation? Ask him how much you owe him, how much he estimates reviewing the final agreement would be.
He knows the case, the particulars and could wrap it up faster than another lawyer who jumps in cold. That you want to wrap it up and only litigate if the agreement warrants it.
He's there to advise, not hold your hand. Take it or leave it.
Just make sure a qualified lawyer reviews the final agreement.