That’s very interesting and sounds cool - though might be above my paygrade.
What kind of delay does the double prompting incur? I can’t imagine it would be nothing, especially for this use case of calling external LLMs via a proxy.
If anything a RAG for certain game related information might be a nice addition to the current system, rather than replacement. I think for a roleplay and storytelling focus there is a benefit of having a persistent character “life story” of long term memories fed into every prompt, creating character throughlines and potentially even character development.
lemmeLurk@lemmy.zip 1 week ago
I haven’t tried with API yet. With local one it’s super quick though, it adds maybe a couple of seconds at most.
If you have a GPU and want to try it with local one there is a plugin for Godot and unity called “Nobody who” they have implemented the RAG approach out of the box with examples as well. So it wasn’t something I came up with.