Walking away when someone is yelling at you is a perfectly fine boundary.