Skip to content

Issues: openai/evals

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
85 Open 118 Closed
85 Open 118 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Cannot pass the check and got this error: KeyError: 'sample' bug Something isn't working
#1012 opened May 23, 2023 by 14H034160212
1
Exact Match template
#1040 opened May 26, 2023 by JasonGross
1
stock picking as model eval
#1131 opened Jun 9, 2023 by qrdlgit
1
oaieval hangs a lot bug Something isn't working
#1292 opened Jul 5, 2023 by shamas-
1
Sample evaluations completing after timeout cause duplicate results bug Something isn't working
#1333 opened Aug 10, 2023 by robatwilliams
1
Having trouble building Evals locally? Try this. bug Something isn't working
#1340 opened Aug 25, 2023 by silverfoxf7
1
How to eval output with ideal_answer directly without having to define the completion_fn ?
#1342 opened Aug 29, 2023 by liuyaox
1
In the task "balance_chemical_equation", many instances have incorrect labels. bug Something isn't working
#1386 opened Oct 19, 2023 by dongZheX
1
What is this
#1527 opened May 3, 2024 by DXv-3
1
Schelling point eval doesn't work
#1533 opened May 22, 2024 by johny-b
1
1
Evaluation on computer vision benchmarks Idea for Eval These issues keep track of requests for different kinds of eval PRs
#235 opened Mar 16, 2023 by finitearth
2
Windows path and unicode decoding
#379 opened Mar 21, 2023 by ulasdilek
2
Idea for Evals: GPT matches lyrics with song name Idea for Eval These issues keep track of requests for different kinds of eval PRs
#390 opened Mar 21, 2023 by Ein-Tim
2
Idea for Evals: Complex, multi-turn instruction-following Evals Idea for Eval These issues keep track of requests for different kinds of eval PRs
#632 opened Apr 11, 2023 by andrew-openai
2
Eval making guide?
#652 opened Apr 12, 2023 by Luka5-A8ram
2
Taxonomy to use for model evaluation?
#706 opened Apr 17, 2023 by qrdlgit
2
Is there a list of tasks that chatgpt fails?
#952 opened May 10, 2023 by irthomasthomas
2
Expose run_id to code being run within an eval
#1264 opened Jun 29, 2023 by robatwilliams
2
Support for Azure OpenAI client
#1469 opened Feb 21, 2024 by pkt1583
2
Evaluate GPT-4 on classical NLP tasks Idea for Eval These issues keep track of requests for different kinds of eval PRs
#246 opened Mar 16, 2023 by LifeIsStrange
3
Update documentation regarding custom evals
#715 opened Apr 18, 2023 by PreciousWarrior
3
Idea for Evals: Count how many numbers are greater than or less than X Idea for Eval These issues keep track of requests for different kinds of eval PRs
#785 opened Apr 23, 2023 by voynow
3
Feature suggestion - Save and Load Conversation History Idea for Eval These issues keep track of requests for different kinds of eval PRs
#907 opened May 3, 2023 by wawryszukd
3
国际化的支持
#1239 opened Jun 26, 2023 by wangkunmin
3
Dataset hosting, data cards and previews
#139 opened Mar 15, 2023 by lhoestq
4
ProTip! Find all open issues with in progress development work with linked:pr.