-
Notifications
You must be signed in to change notification settings - Fork 83
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ValueError: Cannot find the config file for gptq #22
Comments
this is the config file |
Hmm that's a strange one... do you have a traceback I can peek at? Like, the full error? And is the config file still named config.yaml? Also this config looks a bit broken, you're making requests to together.ai but there's no API key, meaning the requests will be incorrectly formatted. IDK if that's the cause of your problem but you should probably fix that. In the most recent version of Augmentoolkit, aphrodite is supported only by running the aphrodite enigne in server mode (this is to make things easier to use, settings-wise). So, the problem you describe may have been patched out, since Augmentoolkit no longer has a quantization mode setting. |
The error is because you're trying to use GPTQ with a non-GPTQ model. The model you linked is a full precision transformers model. You want something more like this: https://huggingface.co/qeternity/Nous-Hermes-2-Mistral-7B-DPO-GPTQ-4bit-128g-actorder_False-Marlin/tree/main (except this is not hermes pro 2). Look for a model that has GPTQ in the name on Huggingface (or convert your Hermes-Pro model to GPTQ). |
LoopControl is right. Closing this since it's not an issue with the project itself, thanks for your report! |
ValueError: Cannot find the config file for gptq
The text was updated successfully, but these errors were encountered: