Comment on Gemini wont talk about Bernie Sanders
blazeknave@lemmy.world 1 day agoMy primary desktop has a typical gaming GPU from 4 yrs ago Primary fuck around box is an old Dell w onboard GPU running proxmox NAS has like no gpu Also have a mini PC running HAOS And a couple of unused pi’s Can I do anything with any of that?
Smokeydope@lemmy.world 1 day ago
Your primary gaming desktop gpu will be best bt for running llms first check your card for exact information more vram the better. Nvidia is preferred but AMD cards work.
First you can play with llamafiles to just get started no fuss no muss download them and follow the quickstart to run as app.
Once you get it running and want better performance or latest models then you can spend some time installing and running kobold.cpp with cublas or vulcan and GPU offloading.