Good one. I am the frame of reference myself… And we rewind time around me, kind of.

Okay, that at least does not sound so wrong like the “teleportation-style” approach!

What you say is like the extreme form of how time travel worked in Primer (the movie), which in fact is an interesting variation.