You are doing a wonderful job and getting some GREAT baby--actually not so baby--steps! You should be proud of yourself.
Backing off on talks is the right thing to do. They probably feel like 'more of the same' in relation to the controlling thing as well. Let HIM lead the way here.
Concentrate on when you do have time together, making it fun, like you did on Tuesday. In other words--just go with it--for many couples this is the glue that cements the relationship.
Congratulations on your weight loss! How did you do it?