Comment on Seems legit

DarkCloud@lemmy.world ⁨9⁩ ⁨months⁩ ago

• criss_cross@lemmy.world ⁨9⁩ ⁨months⁩ ago

  And gpt-oss is an offline version of chatgpt

  source
• utopianfiat@lemmy.world ⁨9⁩ ⁨months⁩ ago

  Indeed huggingface.co/openai-community

  source
• linkinkampf19@lemmy.world ⁨9⁩ ⁨months⁩ ago

  First thing that came to mind: GPT4All

  source
• sp3ctr4l@lemmy.dbzer0.com ⁨9⁩ ⁨months⁩ ago

  I’ve been toying with Qwen3.

  Opensource too!

  source

  • JustAnotherKay@lemmy.world ⁨9⁩ ⁨months⁩ ago

    Oh my god I feel so stupid. I’ve been arguing back and forth whether it was worth de-atomizing my steam deck to spin up alpaca in docker. I forgot they have a flatpak

    source

    • sp3ctr4l@lemmy.dbzer0.com ⁨9⁩ ⁨months⁩ ago

      Bazzite also has podman, though not specifically docker, in the core OS.

      So… I have spun up one local LLM in Alpaca, told it what hardware, OS, and environment it is, told it to generate a context prompt to inform itself of all that… and its now helping me try to figure out how/if it is possible to set up a podman container/environment… for LLMs that either Alpaca does not yet support, or I am too stupid to figure out.

      Alpaca even has tools. You can give an LLM the ability to search the web for something, and find some info or what not.

      Presumably you could tell it to do a lot of things but that seems like a bad idea lol, anyway yeah, I was able to just tell it ‘go online and lookup bazzite, familiarize yourself with pertinent details, reformulate context prompt.’

      source
• Ghostalmedia@lemmy.world ⁨9⁩ ⁨months⁩ ago

  I mean, most people have a local LLM in their pocket right now.

  source

  • sp3ctr4l@lemmy.dbzer0.com ⁨9⁩ ⁨months⁩ ago

    Unless I am missing something:

    Most people do not have a local LLM in their pocket right now.

    Most people have a client app that talks to a remote LLM, which ‘lives’ in an ecologically and economically dubious mega-datacenter, in their pocket right now.

    source

    • GamingChairModel@lemmy.world ⁨9⁩ ⁨months⁩ ago

      Plenty of the AI functions on phones are on-device. I know the iPhone is capable of several text-based processing (summarizing, translating) offline, and they have an API for third party developers to use on-device models. And the Pixels have Gemini Nano on-device for certain offline functions.

      source

      • tetris11@feddit.uk ⁨9⁩ ⁨months⁩ ago

        My phone does speech-to-text flawlessly offline, it’s a crazy useful little LLM tool

        source
      • sp3ctr4l@lemmy.dbzer0.com ⁨9⁩ ⁨months⁩ ago

        Oh!

        Well, I didn’t know that.

        I’m too poor to be able to afford such fancy phones.

        source
    • Ghostalmedia@lemmy.world ⁨9⁩ ⁨months⁩ ago

      Gemini nano, Apple Intelligence On-device, etc.

      source
• SubArcticTundra@lemmy.ml ⁨9⁩ ⁨months⁩ ago

  ollama.org

  source