Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Support PyTorch GPT-2 Models #76

Open
nomyTx opened this issue Apr 10, 2023 · 1 comment
Open

[Feature Request] Support PyTorch GPT-2 Models #76

nomyTx opened this issue Apr 10, 2023 · 1 comment

Comments

@nomyTx
Copy link

nomyTx commented Apr 10, 2023

It would be very useful if there was a script to convert a PyTorch GPT-2 model to ggml.

@appvoid
Copy link
Contributor

appvoid commented May 25, 2023

It would be my world if this is achieved since it seems to be simple with cerebras models. I'm not so good at the core of this library but the minimal approach llms it's one core philosophy of mine. Having said that I do have trained this model recently, but I'm not capable of make it work with ggml yet.

Having gpt2 quantized models "packaged" into a binary ready to use makes it pretty easy and actually usable for realtime scenarios on cpu. So please, someone who knows how to do this, help!

Edit:
Oh also, not forget to mention that gpt2 models perform better than cerebra's ones which I see has a proper convertion!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants