You are doing great with your 180s, keep it up! I agree about telling him your plans for Saratoga Springs. If he does hint about coming along, tell him he's welcome.
X and I agree that any time outside our hometown we need to inform in advance, but not ask permission.
Also re therapy, you need to set a boundary there and tell H that it is between you and your therapist. Tell him you'd be glad to go to marriage counseling with him, if he's so interested.