Comment on Advice - Getting started with LLMs

<- View Parent
xcjs@programming.dev ⁨5⁩ ⁨months⁩ ago

No offense intended, but are you sure it’s using your GPU? Twenty minutes is about how long my CPU-locked instance takes to run some 70B parameter models.

On my RTX 3060, I generally get responses in seconds.

source
Sort:hotnewtop