Comment on Why don't we have motion smoothing on current consoles?

• jarfil@beehaw.org ⁨2⁩ ⁨days⁩ ago

  Hm… good point… but… let’s see, assuming full parallel processing:

  • […]
  • Frame -2 ready
  • Frame -1 ready
    • Show frame -2
    • Start interpolating -2|-1 (should take less than 16ms)
    • Start rendering Frame 0 (will take 33ms)
    • User input 0 (will be received in 20ms if wired)
  • Wait 16ms
    • Frame -2|-1 ready
  • Show Frame -2|-1
  • Wait 4ms
    • Process User input 0 (max 12ms to get into next frame)
    • User input 1 (will be received in 20ms if wired)
  • Wait 12ms
  • Frame 0 ready
    • Show Frame -1
    • Start interpolating -1|0 (should take less than 16ms)
    • Start rendering Frame 1 [includes User input 0] (will take 33ms)
  • Wait 8ms
    • Process User input 1 (…won’t make it into a frame before User input 2 is received)
    • User input 2 (will be received in 20ms if wired)
  • Wait 8ms
    • Frame -1|0 ready
  • Show Frame -1|0
  • Wait 12ms
    • Process User Input 1+2 (…will it take less than 4ms?)
  • Wait 4ms
  • Frame 1 ready [includes user input 0]
    • Show Frame 0
    • Start interpolating 0|1 (should take less than 16ms)
    • Start rendering Frame 2 [includes user input 1+2… maybe] (will take 33ms)
  • Wait 16ms
    • Frame 0|1 ready [includes partial user input 0]
  • Show Frame 0|1 [includes partial user input 0]
  • Wait 16ms
  • Frame 2 ready […hopefully includes user input 1+2]
    • Show Frame 1 [includes user input 0]
  • […]

  So…

  • From user input to partial display: 66ms
  • From user input to full display: 83ms
  • Some user inputs will be bundled up
  • Some user inputs will take some extra 33ms to get displayed

  Effectively, an input-to-render equivalent of between a blurry 15fps, and an abysmal 8.6fps.

  source

  • Boomkop3@reddthat.com ⁨2⁩ ⁨days⁩ ago

    You’ve just invented time travel.

    The basic flow is
    [user input -> render 33ms -> frame available]
    It is impossible to have a latency lower than this, a newer frame simply does not exist yet.

    But with interpolation you also need consistent time between frames. You can’t just present a new frame and the interpolated frame instantly after each other. First you present the interpolated frame, then you want half a frame and present the new frame it was interpolated to.

    So your minimum possible latency is 1.5 frames, or 33+16=59ms (which is horrible)

    One thing I wonder tho… could you use the motion vectors from the game engine that are available before a frame even exists?

    source

    • jarfil@beehaw.org ⁨2⁩ ⁨days⁩ ago

      You’ve just invented time travel.

      Oops, you’re right. Got carried away 😅

      could you use the motion vectors from the game engine that are available before a frame even exists?

      Hm… you mean like what video compression algorithms do? I don’t know of any game doing that, but it could be interesting to explore.

      source

      • Boomkop3@reddthat.com ⁨2⁩ ⁨days⁩ ago

        No, modern game engines produce a whole lot more than the necessary information to generate a frame. Like a depth map and such. One of those is a map of where everything is going and how fast.

        It wouldn’t include movement produced by shaders, but it should include all polygons on screen. which would allow you to just warp the previous frame, no next frame required

        source