Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GoLang Ollama plugin uses a fixed 30s timeout #719

Closed
lesv opened this issue Jul 31, 2024 · 1 comment · Fixed by #1040
Closed

GoLang Ollama plugin uses a fixed 30s timeout #719

lesv opened this issue Jul 31, 2024 · 1 comment · Fixed by #1040
Assignees
Labels
bug Something isn't working

Comments

@lesv
Copy link

lesv commented Jul 31, 2024

Is your feature request related to a problem? Please describe.
Yes - Running Ollama on my M2 Mac it times out.

time=2024-07-30T15:28:09.584-10:00 level=DEBUG msg="span start" name=ollama/llama3.1:70b
time=2024-07-30T15:28:39.592-10:00 level=DEBUG msg="span end" name=ollama/llama3.1:70b
time=2024-07-30T15:28:39.592-10:00 level=DEBUG msg=Action.Run name=0x22e13a0 output=(*ai.GenerateResponse)(nil) err="failed to send request: Post \"https://127.0.0.1:11434/api/generate\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
time=2024-07-30T15:28:39.601-10:00 level=DEBUG msg="span end" name=dev-run-action-wrapper
time=2024-07-30T15:28:39.602-10:00 level=ERROR msg="request end" reqID=9 err="failed to send request: Post \"https://127.0.0.1:11434/api/generate\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"

Describe the solution you'd like
Either a longer timeout - 1000s or better a configurable timeout.

Describe alternatives you've considered
It appears to be set here

Additional context

I'm really enjoying using this. Thanks.

@lesv
Copy link
Author

lesv commented Jul 31, 2024

If you'd be receptive to a PR, I'd propose modifying genkit/go/plugins/ollama/ollama.go:

  • adding Timeout int to type Config L148
  • adding timeout int to var state struct L44
  • add a look at Timeout and default to 30 if zero or set if not in Init L165
  • actually use it in L204

@chrisraygill chrisraygill added the bug Something isn't working label Sep 5, 2024
@apascal07 apascal07 assigned alonsopec89 and unassigned apascal07 Oct 10, 2024
@alonsopec89 alonsopec89 linked a pull request Oct 11, 2024 that will close this issue
2 tasks
@alonsopec89 alonsopec89 moved this to In Progress in Genkit Backlog Oct 15, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in Genkit Backlog Oct 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

4 participants