Comment on [deleted]
What is the appropriate size for 10Gb VRAM?
It depends on your prompt/context size too. The more you have the more memory you need. Try to check the memory usage of your GPU with GPU-Z with different models and scenarios.
Zworf@beehaw.org 2 years ago
It depends on your prompt/context size too. The more you have the more memory you need. Try to check the memory usage of your GPU with GPU-Z with different models and scenarios.