If they’re actually hitting a cloud LLM, there’s no good reason not to wrap that and memoize the results… it would probably be hard to generalize the inputs (it could just be the whole game state for all we know 🤷♂️) but youd build up that database of all the most common input/outputs in real-time
Rhaedas@fedia.io 16 hours ago
Exactly, that's just smart programming (maybe that's one of the issues... but I can't say that without evidence).
I saw someone years ago develop that concept for a text to speech idea for NPC responses, and for the same reasons. Why repay for something when you can just capture the sound file for each new phrase? The cost is in the creation and addition of things, not in the actual running of every person's game (or app in this case).