Comment on I'm in the future and it's stupid
DarkCloud@lemmy.world 1 day agoLocally run AI should be the standard. All you have to do is download a .gguf file from hugging face (around 8 gig) and load it into Koboldcpp’s GUI… Or AnythingLLM if Kobolcpp is too technical for you.
Sone models run faster, others run slower (depending on hardware, model size, and structure). Models like LFM2.5 for instance run pretty fast because they’re more like 3 small models cowriting responses.
Mika@piefed.ca 21 hours ago
Minimal viable local AI that can actually solve problems in agentic mode is qwen 3.8 dense/flash. You are looking at fat ass pc with 16 gb vram 64 ram for flash or 32vram for dense.
Maybe if not for AI bubble prices on hardware, that would be affordable. But today it costs a whole lot for a machine that ideally shouldn’t be the same you are working on - cause whatever you use will eat vram/ram.
DarkCloud@lemmy.world 17 hours ago
If by “solving problems” you mean - you yell your criteria at it enough times that it actually listens and does what you asked for an hour ago… then sure.
Let’s face it, most of the time that’s what interacting with AI is like. It feels like repeatly asking for the same thing and getting variations that you didn’t really ask for… Then eventually settling for what it’s done.