Comment on Advice - Getting started with LLMs
BaroqueInMind@lemmy.one 6 months agoWhat is the appropriate size for 10Gb VRAM?
Comment on Advice - Getting started with LLMs
BaroqueInMind@lemmy.one 6 months agoWhat is the appropriate size for 10Gb VRAM?
Zworf@beehaw.org 6 months ago
It depends on your prompt/context size too. The more you have the more memory you need. Try to check the memory usage of your GPU with GPU-Z with different models and scenarios.