-
Notifications
You must be signed in to change notification settings - Fork 58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Integration with local LLaVA, do we want it? What is the way? #40
Comments
And I just saw that there is work going on at BumbleBee for that: |
Llama CPP support here: |
@thbar
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I've been looking for local-only solutions to reliably extract structured data from invoices/receipts. No API/cloud solutions for obvious privacy reasons (those receipts can sometimes include credentials or account identifiers, and I don't want that data to leave the server in that case).
Thanks to a tweet, I came across this apparently very nice solution:
https://github.com/haotian-liu/LLaVA
A first test via their demo page with a real restaurant receipt worked very nicely (just as nicely as GPT4 currently), see https://twitter.com/thibaut_barrere/status/1773031570259001720 for the input and output.
I see (in #36) that other people are interested to extract data from images.
BumbleBee is also a possibility with a proper choice of model, of course.
Would this have its place in
instructor_ex
to your opinion?If yes, is there a recommended path to integrate a new model? (unsure I'll tackle this, but at least interested to discuss that).
The text was updated successfully, but these errors were encountered: