-
-
Notifications
You must be signed in to change notification settings - Fork 13.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
local-ai: ggml_cuda_init: failed to initialize CUDA: CUDA driver is a stub library #320145
Labels
Comments
using llama-cpp directly works fine so I suspect a problem in local-ai building. I will stick to llama-cpp for now |
This is |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
Not sure if it's a local-ai (or rather one of its dependency llama-cpp/gpt4all) but I can't leverage GPU inference of my nvidia RTX3060 because of
ggml_cuda_init: failed to initialize CUDA: CUDA driver is a stub library
error I think.nvidia-smi output looks ok:
Following an advice on the nixpkgs cuda matrix, I tried to run the previous command with
LD_DEBUG=libs
and ended up withif that helps
Steps To Reproduce
Steps to reproduce the behavior:
I've pusehd master with one commit to enable cuda and unfree by default so one can test with
nix run github:teto/nixpkgs/local-ai-with-cuda#local-ai -- --debug
on a cuda machinelocalhost:8080
)Expected behavior
Have GPU inference. It used to work btw but I would have hard time tracking down, nvidia drivers are hit and miss and there seems to have been a lot of nvidia related changes recently in nixpkgs.
Additional context
I found this related issue but not sure what to do with it horovod/horovod#3831 I think we want to keep using shared libraries
Feel free to close if this is the wrong place to submit but I would appreciate any workaround/tip.
Notify maintainers
Metadata
Please run
nix-shell -p nix-info --run "nix-info -m"
and paste the result.Add a 👍 reaction to issues you find important.
The text was updated successfully, but these errors were encountered: