Comment on Advice - Getting started with LLMs

<- View Parent
Zworf@beehaw.org ⁨5⁩ ⁨months⁩ ago

Hmmm weird. I have a 4090 / Ryzen 5800X3D and 64GB and it runs really well. Admittedly it’s the 8B model because the intermediate sizes aren’t out yet and 70B simply won’t fly on a single GPU.

But it really screams. Much faster than I can read.

PS: Ollama is just llama.cpp under the hood.

source
Sort:hotnewtop