Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: blank response with ollama backend #2694

Closed
travisgu opened this issue Jun 20, 2024 · 6 comments
Closed

[Bug]: blank response with ollama backend #2694

travisgu opened this issue Jun 20, 2024 · 6 comments
Labels
area: backend Related to backend functionality or under the /backend directory bug Something isn't working Stale

Comments

@travisgu
Copy link

What happened?

When I tried to use Quivr with Ollam qwen:72b model, the chat is already empty. There is no error showing in the log or web page. I checked the superbase chat history table the assistant text is empty:
image

Relevant log output

backend-core  | INFO:     127.0.0.1:51044 - "GET /healthz HTTP/1.1" 200 OK
backend-core  | INFO:     127.0.0.1:51088 - "GET /healthz HTTP/1.1" 200 OK
beat          | [2024-06-20 08:42:00,000: INFO/MainProcess] Scheduler: Sending due task process_premium_users (check_if_is_premium_user)
beat          | [2024-06-20 08:42:00,003: INFO/MainProcess] Scheduler: Sending due task process_sync_active (process_sync_active)
worker        | [2024-06-20 08:42:00,004: INFO/MainProcess] Task check_if_is_premium_user[ed7d1da4-7965-4b03-aa6c-d08f6d1b4953] received
worker        | [2024-06-20 08:42:00,009: INFO/MainProcess] Task process_sync_active[3e087e91-ba04-484a-8f1c-0d363f90dcc2] received
worker        | [2024-06-20 08:42:00,050: INFO/ForkPoolWorker-3] HTTP Request: GET https://host.docker.internal:54321/rest/v1/syncs_active?select=%2A&last_synced=lt.2024-06-20T02%3A42%3A00.012042 "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:42:00,059: INFO/ForkPoolWorker-3] HTTP Request: GET https://host.docker.internal:54321/rest/v1/syncs_active?select=%2A&force_sync=eq.True "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:42:00,062: INFO/ForkPoolWorker-3] Task process_sync_active[3e087e91-ba04-484a-8f1c-0d363f90dcc2] succeeded in 0.05039690800185781s: None
worker        | [2024-06-20 08:42:00,842: INFO/ForkPoolWorker-2] HTTP Request: GET https://host.docker.internal:54321/rest/v1/subscriptions?select=%2A&current_period_end=gt.2024-06-20%2010%3A42%3A00.006442 "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:42:01,493: INFO/ForkPoolWorker-2] HTTP Request: GET https://host.docker.internal:54321/rest/v1/customers?select=%2A&order=created.desc "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:42:01,501: INFO/ForkPoolWorker-2] HTTP Request: GET https://host.docker.internal:54321/rest/v1/product_to_features?select=%2A "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:42:01,508: INFO/ForkPoolWorker-2] HTTP Request: GET https://host.docker.internal:54321/rest/v1/users?select=id&email=eq.admin%40quivr.app "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:42:01,516: INFO/ForkPoolWorker-2] Task check_if_is_premium_user[ed7d1da4-7965-4b03-aa6c-d08f6d1b4953] succeeded in 1.5096762990033312s: True
backend-core  | INFO:     172.26.0.1:56690 - "GET /user HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56692 - "GET /chat HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56696 - "GET /user/identity HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56694 - "GET /onboarding HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56698 - "GET /brains/a0950bc4-0e84-4bb9-aecb-2aac255061da/ HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56694 - "GET /brains/ HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56822 - "GET /onboarding HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56818 - "GET /user HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56828 - "GET /user/identity HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56824 - "GET /chat HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56828 - "GET /brains/ HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56826 - "GET /brains/a0950bc4-0e84-4bb9-aecb-2aac255061da/ HTTP/1.1" 200 OK
backend-core  | INFO:     127.0.0.1:51286 - "GET /healthz HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56826 - "OPTIONS /chat/79148a96-9cb3-490f-ab32-c848b71eda35/question/stream?brain_id=a0950bc4-0e84-4bb9-aecb-2aac255061da HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56824 - "GET /user HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56828 - "GET /onboarding HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56818 - "GET /user/identity HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56826 - "POST /chat/79148a96-9cb3-490f-ab32-c848b71eda35/question/stream?brain_id=a0950bc4-0e84-4bb9-aecb-2aac255061da HTTP/1.1" 200 OK
backend-core  | INFO:     127.0.0.1:51386 - "GET /healthz HTTP/1.1" 200 OK
beat          | [2024-06-20 08:43:00,000: INFO/MainProcess] Scheduler: Sending due task process_sync_active (process_sync_active)
worker        | [2024-06-20 08:43:00,004: INFO/MainProcess] Task process_sync_active[be12573f-e929-4138-ba8a-90ece81ffcd6] received
beat          | [2024-06-20 08:43:00,005: INFO/MainProcess] Scheduler: Sending due task process_premium_users (check_if_is_premium_user)
worker        | [2024-06-20 08:43:00,009: INFO/MainProcess] Task check_if_is_premium_user[7cbdd17c-492e-4c4e-a5cb-fea70cdfd3a1] received
worker        | [2024-06-20 08:43:00,016: INFO/ForkPoolWorker-2] HTTP Request: GET https://host.docker.internal:54321/rest/v1/syncs_active?select=%2A&last_synced=lt.2024-06-20T02%3A43%3A00.006841 "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:43:00,043: INFO/ForkPoolWorker-2] HTTP Request: GET https://host.docker.internal:54321/rest/v1/syncs_active?select=%2A&force_sync=eq.True "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:43:00,045: INFO/ForkPoolWorker-2] Task process_sync_active[be12573f-e929-4138-ba8a-90ece81ffcd6] succeeded in 0.03935996099971817s: None
redis         | 1:M 20 Jun 2024 08:43:00.096 * 100 changes in 300 seconds. Saving...
redis         | 1:M 20 Jun 2024 08:43:00.097 * Background saving started by pid 25
redis         | 25:C 20 Jun 2024 08:43:00.102 * DB saved on disk
redis         | 25:C 20 Jun 2024 08:43:00.102 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
redis         | 1:M 20 Jun 2024 08:43:00.197 * Background saving terminated with success
worker        | [2024-06-20 08:43:00,709: INFO/ForkPoolWorker-3] HTTP Request: GET https://host.docker.internal:54321/rest/v1/subscriptions?select=%2A&current_period_end=gt.2024-06-20%2010%3A43%3A00.012100 "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:43:01,276: INFO/ForkPoolWorker-3] HTTP Request: GET https://host.docker.internal:54321/rest/v1/customers?select=%2A&order=created.desc "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:43:01,283: INFO/ForkPoolWorker-3] HTTP Request: GET https://host.docker.internal:54321/rest/v1/product_to_features?select=%2A "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:43:01,289: INFO/ForkPoolWorker-3] HTTP Request: GET https://host.docker.internal:54321/rest/v1/users?select=id&email=eq.admin%40quivr.app "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:43:01,290: INFO/ForkPoolWorker-3] Task check_if_is_premium_user[7cbdd17c-492e-4c4e-a5cb-fea70cdfd3a1] succeeded in 1.278936637998413s: True
backend-core  | INFO:     172.26.0.1:57000 - "GET /onboarding HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:57004 - "GET /user HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:57006 - "GET /brains/a0950bc4-0e84-4bb9-aecb-2aac255061da/ HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56826 - "GET /user/credits HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:57002 - "GET /user/identity HTTP/1.1" 200 OK
backend-core  | INFO:     127.0.0.1:51460 - "GET /healthz HTTP/1.1" 200 OK
backend-core  | INFO:     127.0.0.1:51522 - "GET /healthz HTTP/1.1" 200 OK
beat          | [2024-06-20 08:44:00,000: INFO/MainProcess] Scheduler: Sending due task process_premium_users (check_if_is_premium_user)
beat          | [2024-06-20 08:44:00,004: INFO/MainProcess] Scheduler: Sending due task process_sync_active (process_sync_active)

Twitter / LinkedIn details

No response
@travisgu travisgu added the bug Something isn't working label Jun 20, 2024
@dosubot dosubot bot added the area: backend Related to backend functionality or under the /backend directory label Jun 20, 2024
@dosubot DosuBot
Copy link

dosubot bot commented Jun 20, 2024

Hey there, @travisgu! I'm here to assist you with any issues you're experiencing while using Quivr. I can help you solve bugs, answer questions, and even guide you on how to become a contributor to the project. Let's get started!

To resolve the issue of blank chat responses when using the Ollam qwen:72b model in Quivr, ensure that the prompt and assistant columns in the Supabase chat history table are being correctly populated. Here are the steps to address this:

  1. Check the generate_stream and generate_answer methods: Ensure these methods are correctly saving the responses to the database.

  2. Update the save_answer method: Ensure this method correctly updates the prompt and assistant columns in the Supabase chat history table.

Here is an example of how you might update the save_answer method:

def save_answer(self, question, response_tokens, streamed_chat_history, save_answer):
    # Combine response tokens into a single response
    response = ''.join(response_tokens)
    
    # Update the chat history with the response
    streamed_chat_history.assistant = response
    
    # Save the chat history to the database
    if save_answer:
        chat_service.save_chat_history(
            chat_id=question.chat_id,
            question=question.question,
            answer=response,
            prompt=streamed_chat_history.prompt,
            assistant=streamed_chat_history.assistant
        )
  1. Verify .env configuration: Ensure the OLLAMA_API_BASE_URL is uncommented and correctly pointing to the local URL for the Ollama API:
# LOCAL
OLLAMA_API_BASE_URL=https://host.docker.internal:11434 # Uncomment to activate ollama. This is the local url for the ollama api
  1. Add additional logging: If the issue persists, add logging to trace where the data might be getting lost or not being saved correctly.

Ensure that the generate_answer and generate_stream methods in the backend/modules/brain/qa_headless.py file are correctly handling the chat history updates and saving the responses to the database [1][2][3].

To continue talking to Dosu, mention @dosu.

@travisgu
Copy link
Author

I didn't change any code and I was running quivr with the docker image. The url configuration is correct. I can see the prompt in the Ollama log file and Ollama return 200 for the API call. @dosu

@mkhludnev
Copy link

what's the model name you use? I find that it should be prefixed with ollama/

@travisgu
Copy link
Author

what's the model name you use? I find that it should be prefixed with ollama/

right，the model name is “ollama/qwen:72b”

@travisgu
Copy link
Author

btw, I am using the quivr with docker-compose on my laptop, I am not sure if this is cause.

@github-actions GitHub Actions
Copy link
Contributor

Thanks for your contributions, we'll be closing this issue as it has gone stale. Feel free to reopen if you'd like to continue the discussion.

@github-actions github-actions bot added the Stale label Sep 22, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area: backend Related to backend functionality or under the /backend directory bug Something isn't working Stale
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mkhludnev @travisgu