New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

running into rate limit with OpenAI's GPT-3 API when used during evaluation #3068

Closed

Timoeller opened this issue Aug 19, 2022 · 1 comment

Contributor

Timoeller commented Aug 19, 2022

Is your feature request related to a problem? Please describe.
OpenAI has some rate limits on their API access:

20 requests per minute for free access
60 rpm in pay as you go initially, then it ramps up to 3000 later

see https://help.openai.com/en/articles/5955598-is-api-usage-subject-to-any-rate-limits for details.

Describe the solution you'd like
I see 3 solutions:

get paid plan and wait a bit (3000 requests per minute should be fine for most cases)
hack a time.sleep(X) in here
haystack implements an optional rate limit in the eval methods

Timoeller mentioned this issue

Fix: Add rate limit to OpenAIAnswerGenerator, change token computation #3078

Closed

6 tasks

Collaborator

mayankjobanputra commented Mar 8, 2023

Closing this as this is fixed by #3398

mayankjobanputra closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment