- ollama is bad at everything they do. 2. They can run a really small llm, as the requierements are pretty limited.
Not everyone has a GPU with enough RAM. I have a 3060 with 12GB, and I cannot run ollama and a 3D game at the same time. As soon as ollama starts processing, the GPU is at 100% and game and ollama start swapping data in and out of the GPU RAM, which means the whole computer becomes unresponsive for a long time. I had to set up a script that disables ollama if steam is running.
esc@piefed.social 1 week ago
OrteilGenou@lemmy.world 1 week ago
Sorry, all of your what?
iceberg314@slrpnk.net 1 week ago
12GB is a lot of ram. By small local models I am talking about fine tuning something like < 2GB LLM