-
Notifications
You must be signed in to change notification settings - Fork 780
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to handle embedding model download yourself with ChromaDB client #222
Comments
@zainhoda I could attempt to make a PR to address both these features. As they both impact how the ChromaDB is initialized and setup, it makes sense to do this in the same PR. Thoughts? Not sure what the best way to do this is though.
I will wait with making the PR, until we have agreed on a solution :] |
just use this: from chromadb.utils import embedding_functions
# device = "cuda" if torch.cuda.is_available() else "cpu"
bge_embeddingFunction = embedding_functions.SentenceTransformerEmbeddingFunction("/AI-ModelScope/bge-large-zh-v1.5","cuda",normalize_embeddings=True)
config = {"embedding_function":bge_embeddingFunction, "path": "/root/chatdb/chromadb"} |
in /src/vanna/chromadb/chromadb_vector.py if config is not None:
path = config.get("path", ".")
self.embedding_function = config.get("embedding_function", default_ef)
else:
path = "."
self.embedding_function = default_ef |
Sure, I can give it a go and see if it resolves the issue I was observing. But this needs to be added to Vanna itself, as I do not control this logic when using the library. Thanks for the suggestion, @relic-yuexi :] |
actually i had make a pr for it |
@relic-yuexi I do not see a PR for this yet. Were you going to make one? If not, I can give it a go, as I have a specific use case I can test it for directly. |
There. you can use this code to load the embedding model as you download. from chromadb.utils import embedding_functions
bge_embeddingFunction = embedding_functions.SentenceTransformerEmbeddingFunction("/AI-ModelScope/bge-large-zh-v1.5","cuda",normalize_embeddings=True) then use the config to chromadbvector config = {"embedding_function":bge_embeddingFunction, "path": "/root/chatdb/chromadb"}
vs = ChromaDB_VectorStore(config) then it will all use the bge_embedding or you have downloaded |
@relic-yuexi Sorry, I misunderstood your first reply. I can run a test to see if this addresses issue (2), which is the most critical one to fix. Cheers! :] |
is there an expected configuration pattern for I'd love to use an in-memory version as I intend to primarily load DDL from the database anyway and would reload from scratch every time |
maybe you can use faiss , it is just a vector search tool as you want in memory |
@roshanr10 ChromaDB supports in-memory storage but the Chroma instance setup by Vanna does not. There is no API in Vanna to use in-memory instead of persistent store for Chroma: @zainhoda I can make a PR now to address this as I need this myself. EDIT: I made a separate issue about this, @roshanr10. Then I can tag a PR to it if I make one. |
@roshanr10 I have now made a PR to add in-memory support. PR: #250 |
Merged in #250. Your feature request will be part of the upcoming release, @roshanr10 :] As my original question has been resolved. I am closing this issue. If relevant, feel free to open a new issue and tag this issue, if you have issues/requests related to this issue. |
This line of code is causing some headache:
vanna/src/vanna/chromadb/chromadb_vector.py
Line 13 in 8c5e05a
The problem is that if you want to store ChromaDB in a docker container, you most likely want to handle all downloads yourself, and give very limited rights to ChromaDB and Vanna, in terms of what is allowed to write on disk and where. If you try to download the model yourself, at the appropriate place (after reading some ChromaDB source code I found where it was trying to save the model), it will give an error stating that the model is already on disk (this I expected would be fine but this yields an error).
Another problem is that ChromaDB currently also uses PersistentClient, meaning that an
.sqlite3
file will be saved on disk.Hence, I would like the following features:
Issue was observed on: Ubuntu 22.04 using
vanna==0.0.36
and Python 3.10.8 (after also fixing thesqlite3
issue Chroma has for Python 3.10).The text was updated successfully, but these errors were encountered: