Comment on the myth of consensual pc building

jpablo68@infosec.pub ⁨2⁩ ⁨days⁩ ago

• Gyrcas@piefed.zip ⁨2⁩ ⁨days⁩ ago

  Please don’t run LLMs at all…

  source

  • db2@lemmy.world ⁨2⁩ ⁨days⁩ ago

    They have uses, most of which aren’t what’s being pushed right now and none of which are what the tech bro sociopaths are pushing.

    source

    • jpablo68@infosec.pub ⁨1⁩ ⁨day⁩ ago

      I mainly use it to practice german.

      source
• DaTingGoBrrr@lemmy.ml ⁨2⁩ ⁨days⁩ ago

  If you have enough regular ram and an nvme ssd you can run huge models on modest hardware using Colibrí.

  I got a Lenovo mini PC with 64GB of ram and a 500GB nvme and a 38B model runs with 11 tokens per second on my machine.

  The 38B model fits in my ram but if I had used a 120B parameter model then it would stream the rest of the model from my nvme disk.

  source
• jh29a@lemmy.blahaj.zone ⁨2⁩ ⁨days⁩ ago

  Now you can ask the PC for consent

  source