There is no objective reference frame in the universe, and it makes much more sense to assume you’d stay on earth, bound by its gravity while time-traveling, than to magically lose all your momentum during the transition and stop in space.