I run deepseek-r1:14b locally, though it needs to go into RAM and runs slower its still a reasonably good speed. Keeps up with reading it. Should try a larger one at some point, but its quite a bit to download when you get to the larger ones. Usually run ~7b size as that can fit in VRAM and runs way faster.
Comment on DeepSeek-V3 now runs at 20 tokens per second on Mac Studio, and that’s a nightmare for OpenAI
ParetoOptimalDev@lemmy.today 1 week agoNo, people who want something approaching chatgpt but local want to run at least deepseek V3 32B.
Qwen at least fares much worse for my usage as do deepseek V3 under 32B.
Korhaka@sopuli.xyz 1 week ago
morrowind@lemmy.ml 1 week ago
The he’ll is v3 32b. Are you talking about a distill
vintageballs@feddit.org 3 days ago
They probably confused the R1 Qwen distill with something else. Afaik there is no 32b model from DeepSeek directly.