Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

readme : add link to python bindings #181

Merged
merged 1 commit into from
May 21, 2023

Conversation

marella
Copy link
Contributor

@marella marella commented May 21, 2023

Hi,

I created Python bindings for the GGML models: https://github.com/marella/ctransformers

Currently it supports GPT-2, GPT-J, GPT-NeoX, LLaMA, MPT, etc. See Supported Models.

It provides a unified interface for all models:

from ctransformers import AutoModelForCausalLM

llm = AutoModelForCausalLM.from_pretrained('/path/to/ggml-gpt-2.bin', model_type='gpt2')

print(llm('AI is going to'))

It can be used with models hosted on the Hugging Face Hub:

llm = AutoModelForCausalLM.from_pretrained('marella/gpt-2-ggml')

It has built-in support for LangChain:

from ctransformers.langchain import CTransformers

llm = CTransformers(model='/path/to/ggml-gpt-2.bin', model_type='gpt2')

print(llm('AI is going to'))

Please see README for more details.

@ggerganov ggerganov merged commit 35a78bd into ggerganov:master May 21, 2023
@marella marella deleted the ctransformers branch May 21, 2023 12:37
CCLDArjun pushed a commit to CCLDArjun/ggml that referenced this pull request Dec 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants