Comment on Advice - Getting started with LLMs
xcjs@programming.dev 5 months agoGood luck! I’m definitely willing to spend a few minutes offering advice/double checking some configuration settings if things go awry again. Let me know how things go. :-)
BaroqueInMind@lemmy.one 5 months ago
My setup is Win 11 Pro ➡️ WSL2 / Debian ➡️ Docker Desktop (for windows)
Should I install the nvidia drivers within Debian even though the host OS already has drivers?
xcjs@programming.dev 5 months ago
I think there was a special process to get Nvidia working in WSL. Let me check… (I’m running natively on Linux, so my experience doing it with WSL is limited.)
docs.nvidia.com/cuda/wsl-user-guide/index.html - I’m sure you’ve followed this already, but according to this. It looks like you don’t want to install the Nvidia drivers, and only want to install the cuda-toolkit metapackage. I’d follow the instructions from that link closely.
You may also run into performance issues within WSL due to the virtual machine overhead.
BaroqueInMind@lemmy.one 5 months ago
I did indeed follow that guide already, thank you for the respect; I am an idiot and installed both the nvidia WSL driver on top of the host OS driver _as well as the Cuda driver. So I’ll try again with only that guide and see what breaks.
xcjs@programming.dev 5 months ago
We all mess up! I hope that helps - let me know if you see improvements!