-
Notifications
You must be signed in to change notification settings - Fork 959
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Question] What is the status of Vulkan backend? #542
Comments
there are ggerganov/llama.cpp#2059 and ggerganov/llama.cpp#2039 |
Yeah, I'm working on it. Let me know if you have any questions. It's a big project, but I'm making progress. |
Is there any low hanging fruit a newcomer to the project could help with? |
@Calandiel If you have experience with Vulkan, maybe. Otherwise probably not. |
I have. I've written vulkan based render pipelines professionally and made toy neural networks in vulkan trained with sgd. Been working with it at least in some capacity for the last 4 years or so. |
Oh cool, I'd be glad to work something out. If you have Discord, send me a message ( |
Will do, see you on Discord! |
I think nomic-ai have functional kompute of llama.cpp right now |
* Revert 7e53955 (ggerganov#542) Still needs to be fixed properly * Fix linking on mingw32
The vulkan and kompute backends have been merged in llama.cpp, all that is left is update the cmake build files to be able to use them in other ggml projects. |
does this affect anything using ggml? whisper.cpp stabledifussion.cpp? etc? |
@Kreijstal backends are usually upstreamed to ggml, but ggml-api consumers need to use them explicitly. eg new-ish backend code here in ggml: |
Vulkan may be not the the best/fastest/easiest or so solution for inference, but is probably most portable GPU acceleration approach.
Is anyone working actively to add support for it? And if so what is status/progress? If not, is it planned?
The text was updated successfully, but these errors were encountered: