-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support GGUF #16
Comments
chatllm.cpp is not down-stream app of llama.cpp, but an app based on ggml just as llama.cpp. It supports some models that are not supported by llama.cpp, I won't wait for llama.cpp to support it and then port to chatllm.cpp. So, I need to maintain my own set of supported models. Further more, since the implementation of some models is developed independently from llama.cpp, some tensors (k/v/q specifically) might use different formats/shapes, which makes them incompatible with each other. Anyway, it seems possible to support GGUF for some models (e.g. LlaMA models). I will look into it later. |
GGML is kind of not supported anymore and all models have moved to GGUF as a standard a year ago. Are there any plans to support it here? I'm wondering what are the limitations to handle sliding window in gguf compared to ggml if that's the problem
The text was updated successfully, but these errors were encountered: