For just simple and guided actions and dialogue, local would be fine for this. Or better, have AI design the large database of what to say or do when a player does things. No need to reinvent what players are going to duplicate. The token cost was worse when you consider all the same stuff being prompted over and over.
Ledivin@lemmy.world 17 hours ago
If they’re actually hitting a cloud LLM, there’s no good reason not to wrap that and memoize the results… it would probably be hard to generalize the inputs (it could just be the whole game state for all we know 🤷♂️) but youd build up that database of all the most common input/outputs in real-time
Rhaedas@fedia.io 16 hours ago
Exactly, that's just smart programming (maybe that's one of the issues... but I can't say that without evidence).
I saw someone years ago develop that concept for a text to speech idea for NPC responses, and for the same reasons. Why repay for something when you can just capture the sound file for each new phrase? The cost is in the creation and addition of things, not in the actual running of every person's game (or app in this case).