You said you would like her to wait until you get back from work. Maybe phrase it more like. "Sure. You can come by at x o'clock"...instead of emphasizing that you want to be there.