-
Notifications
You must be signed in to change notification settings - Fork 966
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"gpt_tokenize: unknown token" running RedPajama #163
Comments
The same failure occurs for the Mosaic model. However, I think I've found the problem. The highlighted line here defines |
So This line casts away this constness before writing to the underlying storage, so on my compiler replacing |
I'm hitting an error while running RedPajama. It's likely the result of a misunderstanding on my part, so I'm hoping somebody can shed some light on what I'm doing wrong.
To begin with, I've cloned ggml from commit
74705055853f7922e9622bdd0a1ebde2b8f57431
. I build with gcc 9.4.0 on Linux x86:This completes without error. I've already cloned
https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1
, so proceed to ggml conversion:Next, I quantize the model:
And finally attempt inference:
As you can see, errors of the form
gpt_tokenize: unknown token 'I'
appear and the output text is nonsensical. I seem to get the same problem whether I use a 32-bit, 16-bit, or 4-bit model.Does anything look amiss in the steps that I've performed or the logs which are generated from conversion/quantization? Any help at all would be appreciated!
The text was updated successfully, but these errors were encountered: