[deleted]
You can try setting up Ollama on your RPi, then use a highly-optimized variant of the Mistral model (or quantize it yourself with GGUF/llama.cpp). You can do some very heavy quantization (2-bit), which will increase the error rate. But if you are only planning to use the generated text as a starting point, it might be useful nevertheless. Also see: github.com/ollama/ollama/blob/main/…/import.md#im…
Hjalamanger@feddit.nu 9 months ago
I found this blog post were the author tries to use chat GPT to generate theatre manuscript/narrative. It’s based on the paper “Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals”. In the blog post they outline their narrative generation procedure in this chart:
Fig. 1. Dramatron’s Hierarchical Coherent Story Generation. Dramatron starts from a log line to generate a title and characters Characters generated are used as prompts to generate a sequence of scene summaries in the plot. Descriptions are subsequently generated for each unique location. Finally, these elements are all combined to generate dialogue for each scene. The arrows in the figure indicate how text generated is used to construct prompts for further LLM text generation.
I also found this GitHub repo with links to more resources on this topic.