-
Notifications
You must be signed in to change notification settings - Fork 966
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for bigcode/starcoder #188
Comments
Converted and quantized models can be found here: |
@s-kostyaev I don't think you need this patch - the correct parameters are loaded from the model file |
@ggerganov Ok. Why it doesn't work for @seyyedaliayati ? |
@ggerganov you are right. Without patch all works fine. Thank you for information. |
So, why I get this error? If you need more information, please let me know. Thanks. |
I just re-run and now I get this: (base) ali@host:~/ggml$ python examples/starcoder/convert-hf-to-ggml.py bigcode/starcoder
Loading model: bigcode/starcoder
Downloading (…)l-00001-of-00007.bin: 100%|█████████████████████████████████████████| 9.90G/9.90G [05:09<00:00, 32.0MB/s]
Downloading (…)l-00002-of-00007.bin: 100%|█████████████████████████████████████████| 9.86G/9.86G [04:51<00:00, 33.8MB/s]
Downloading (…)l-00003-of-00007.bin: 100%|█████████████████████████████████████████| 9.85G/9.85G [04:59<00:00, 32.9MB/s]
Downloading (…)l-00004-of-00007.bin: 100%|█████████████████████████████████████████| 9.86G/9.86G [04:53<00:00, 33.6MB/s]
Downloading (…)l-00005-of-00007.bin: 100%|█████████████████████████████████████████| 9.85G/9.85G [04:55<00:00, 33.3MB/s]
Downloading (…)l-00006-of-00007.bin: 100%|█████████████████████████████████████████| 9.86G/9.86G [04:58<00:00, 33.1MB/s]
Downloading (…)l-00007-of-00007.bin: 100%|█████████████████████████████████████████| 4.08G/4.08G [01:56<00:00, 34.9MB/s]
Downloading shards: 100%|████████████████████████████████████████████████████████████████| 7/7 [31:46<00:00, 272.30s/it]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████| 7/7 [03:42<00:00, 31.77s/it]
Killed
(base) ali@host:~/ggml$ python examples/starcoder/convert-hf-to-ggml.py bigcode/starcoder
Loading model: bigcode/starcoder
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████| 7/7 [03:38<00:00, 31.20s/it]
Killed
|
I'm afraid you may be lacking system memory for the conversion. |
You are right. I have increased my RAM and issue solved! |
Hi!
I saw the example for the
bigcode/gpt_bigcode-santacoder
model. I am wondering how I can run thebigcode/starcoder
model on CPU with a similar approach.When I run the following command:
I encountered this error:
Any ideas or help would be greatly appreciated.
The text was updated successfully, but these errors were encountered: