Oh my god I feel so stupid. I’ve been arguing back and forth whether it was worth de-atomizing my steam deck to spin up alpaca in docker. I forgot they have a flatpak
Bazzite also has podman, though not specifically docker, in the core OS.
So… I have spun up one local LLM in Alpaca, told it what hardware, OS, and environment it is, told it to generate a context prompt to inform itself of all that… and its now helping me try to figure out how/if it is possible to set up a podman container/environment… for LLMs that either Alpaca does not yet support, or I am too stupid to figure out.
Alpaca even has tools. You can give an LLM the ability to search the web for something, and find some info or what not.
Presumably you could tell it to do a lot of things but that seems like a bad idea lol, anyway yeah, I was able to just tell it ‘go online and lookup bazzite, familiarize yourself with pertinent details, reformulate context prompt.’
JustAnotherKay@lemmy.world 1 week ago
Oh my god I feel so stupid. I’ve been arguing back and forth whether it was worth de-atomizing my steam deck to spin up alpaca in docker. I forgot they have a flatpak
sp3ctr4l@lemmy.dbzer0.com 1 week ago
Bazzite also has podman, though not specifically docker, in the core OS.
So… I have spun up one local LLM in Alpaca, told it what hardware, OS, and environment it is, told it to generate a context prompt to inform itself of all that… and its now helping me try to figure out how/if it is possible to set up a podman container/environment… for LLMs that either Alpaca does not yet support, or I am too stupid to figure out.
Alpaca even has tools. You can give an LLM the ability to search the web for something, and find some info or what not.
Presumably you could tell it to do a lot of things but that seems like a bad idea lol, anyway yeah, I was able to just tell it ‘go online and lookup bazzite, familiarize yourself with pertinent details, reformulate context prompt.’