You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
FROM nvidia/cuda:12.2.0-runtime-ubuntu20.04
RUN apt-get update -y && apt-get install -y python3.9 python3.9-distutils curl
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
RUN python3.9 get-pip.py
RUN pip3 install fschat vllm
RUN pip3 install fschat[model_worker,webui]
Everything works, but when the prompt length is close to 4000 tokens (that is size of the the model's context window).
When I approach the limit I keep getting the following error back: Error: 400 status code (no body).
Could someone help me debug the issue? The length of the prompt is still under the length of the context window and the error message is not useful. Is there a debug mode I can use to gather more information on what is happening in the backend?
The text was updated successfully, but these errors were encountered:
GianlucaDeStefano
changed the title
Openai_api_server error 400 (no body)
Openai_api_server Error: 400 status code (no body)
Jul 7, 2024
I spawn an openai compatible server using the following docker-compose:
The container instead is:
Everything works, but when the prompt length is close to 4000 tokens (that is size of the the model's context window).
When I approach the limit I keep getting the following error back:
Error: 400 status code (no body)
.Could someone help me debug the issue? The length of the prompt is still under the length of the context window and the error message is not useful. Is there a debug mode I can use to gather more information on what is happening in the backend?
The text was updated successfully, but these errors were encountered: