OpenAI releases a free GPT model that can run on your laptop

Comments

• nathan@piefed.alphapuggle.dev ⁨1⁩ ⁨month⁩ ago

  *if you have a laptop with 16gb of vram. Otherwise you'll be watching ollama hit your CPU for 5 minutes with no output

  source

  • sefra1@lemmy.zip ⁨1⁩ ⁨month⁩ ago

    Isn’t that true for most models until someone destiles and quantises them so they can run on common hardware?

    source

    • fuckwit_mcbumcrumble@lemmy.dbzer0.com ⁨1⁩ ⁨month⁩ ago

      This is the internet, we’re only allowed to be snarky here.

      source
    • CyberSeeker@discuss.tchncs.de ⁨1⁩ ⁨month⁩ ago

      Yes, but 20 billion parameters is too much for most GPUs, regardless of quantization. You would need at least 14GB, and even that’s unlikely without offloading major parts to the CPU and system RAM (which kills the token rate).

      source

      • -> View More Comments
    • Ghoelian@lemmy.dbzer0.com ⁨1⁩ ⁨month⁩ ago

      I mean yeah, but that doesn’t make the title any lore true.

      source
• Bebopalouie@lemmy.ca ⁨1⁩ ⁨month⁩ ago

  No thanks.

  source

  • SweetCitrusBuzz@beehaw.org ⁨1⁩ ⁨month⁩ ago

    Agreed.

    source
• PoisonedPrisonPanda@discuss.tchncs.de ⁨1⁩ ⁨month⁩ ago

  paywall free version?

  source