Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

quantization func test failed with GGML_QKK_64 #739

Closed
winice-test opened this issue Feb 19, 2024 · 2 comments
Closed

quantization func test failed with GGML_QKK_64 #739

winice-test opened this issue Feb 19, 2024 · 2 comments

Comments

@winice-test
Copy link

I tried to compile the code with GGML_QKK_64 defined so that I can verify the code when QK_K != 256. Then I found test-quantize-fns failed in iq3_xxs.

Steps to reproduce on Ubuntu 22.04.3 with gnu compiler 11.4.0 :

cmake -DCMAKE_C_FLAGS="-DGGML_QKK_64" ..
cmake --build . --config Release
bin/test-quantize-fns

Then it reports the following in the bottom

Testing iq3_xxs
================================================================= iq3xs_init_impl(grid_size = 256)
iq3xs_init_impl: 18985 neighbours in total
iq3_xxs absolute quantization error:    FAILED (0.019304)
iq3_xxs dot product error:              FAILED (0.825301)
@ggerganov
Copy link
Owner

I think it works now

@winice-test
Copy link
Author

No failure now if compile with GGML_QKK_64 defined. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants