-
-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: Running LibreChat against LiteLLM backed by Ollama #1270
Labels
bug
Something isn't working
Comments
Thanks for your thorough report, hope to address this soon as I've seen this error in another context, and I can reproduce it now. |
I'm fixing this right now |
FYI I seem to have issues with ollama independent of LibreChat when I don't include |
Confirmed the latest merge works great. Thanks @danny-avila |
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Contact Details
No response
What happened?
I have Ollama/openchat running behind the OpenAPI compatible frontend of LiteLLM.
The chat completion never "finishes" when the bot is responding, appears to be searching for a chatcompletion message
Steps to Reproduce
install librechat via docker,
in .env file set OPENAI_REVERSE_PROXY=http:https://192.168.2.142:8000 (i installed litellm on the docker host)
install ollama via install script: curl https://ollama.ai/install.sh | sh
pip install litellm
ollama pull openchat
pip install async_generator
litellm --model ollama/openchat --api_base http:https://localhost:11434 --drop_params <-- required for presence penalty not being supported in litellm
The setup works for calling a localLLM, but the cursor gets "stuck" and does not return to the user to add additional chats.
What browsers are you seeing the problem on?
Firefox, Microsoft Edge
Relevant log output
Screenshots
Code of Conduct
The text was updated successfully, but these errors were encountered: