I know I’m late. But realistically this is what we would need to do proper video generation.

Instead of the user input of the game, you would use the script from a tv series linked with the video… Or just the subtitles, but the actual script describing the scenes would be better… That’s the equivalent user input. Couple that with an llm and bam! Endless episodes of I love Lucy and Seinfeld.