Skip to content

Issues: openai/evals

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
87 Open 118 Closed
87 Open 118 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Functions: Support for minItems and maxItems for json schema array
#1173 opened Jun 15, 2023 by tjhiggins
👍 15
Publish latest evals framework to PyPI
#1344 opened Aug 31, 2023 by robatwilliams
👍 9
Dataset hosting, data cards and previews
#139 opened Mar 15, 2023 by lhoestq
4 👍 8
Evaluate GPT-4 on classical NLP tasks Idea for Eval These issues keep track of requests for different kinds of eval PRs
#246 opened Mar 16, 2023 by LifeIsStrange
3 👍 8
Add BigBench Tasks for evaluation Idea for Eval These issues keep track of requests for different kinds of eval PRs
#153 opened Mar 15, 2023 by Muhtasham
1 👍 5
Evaluation on computer vision benchmarks Idea for Eval These issues keep track of requests for different kinds of eval PRs
#235 opened Mar 16, 2023 by finitearth
2 👍 5
Any website where I can share evaluation results?
#1166 opened Jun 15, 2023 by pocca2048
7 👍 5
Feature request for evals: Add support for function call.
#1346 opened Sep 7, 2023 by srenault
👍 5
Evaluate the cost of running tests
#1350 opened Sep 15, 2023 by onjas-buidl
👍 4
Are not merged PRs the result of irrelevancy to the model?
#873 opened Apr 30, 2023 by albukirky1
12 👍 4
Overall Report for Eval Set As a Whole
#939 opened May 8, 2023 by chinggg
👍 3
Idea for Evals: Emotion and sentiment analysis Evals Idea for Eval These issues keep track of requests for different kinds of eval PRs
#637 opened Apr 11, 2023 by Pabreetzio
6 👍 2
Update documentation regarding custom evals
#715 opened Apr 18, 2023 by PreciousWarrior
3 👍 1
Make GPT4 aware of the evals format
#143 opened Mar 15, 2023 by bhack
👍 1
Windows path and unicode decoding
#379 opened Mar 21, 2023 by ulasdilek
2 👍 1
Improvements to Match: case insensitive and strip
#1421 opened Nov 30, 2023 by LoryPack
👍 1
oaieval fails with error UnicodeDecodeError: 'charmap' codec can't decode byte bug Something isn't working
#980 opened May 17, 2023 by rslinford
1 👍 1
Is there a list of tasks that chatgpt fails?
#952 opened May 10, 2023 by irthomasthomas
2 👍 1
Add version option to oaieval
#1102 opened Jun 5, 2023 by phelps-sg
👍 1
Idea for Evals: Sorting numbers with repeats and negatives Idea for Eval These issues keep track of requests for different kinds of eval PRs
#782 opened Apr 23, 2023 by voynow
7 👍 1
Proposal for Adding a New Evaluation Metric: Sentiment Analysis Accuracy
#1419 opened Nov 23, 2023 by Sarfaraz021
👍 1
Request for Repo: "Open Source, Daily Auto-Generated SOTA LLM Model Benchmarks"
#505 opened Mar 29, 2023 by Marviel
1
The metrics valus are not right
#414 opened Mar 23, 2023 by lshowway
Information exposure alert through an exception bug Something isn't working
#1543 opened Aug 8, 2024 by arpitjain099
1
I believe that in order for GPT-4 to create by itself many inventions through two month, necessary that...
#685 opened Apr 15, 2023 by 55255ru
ProTip! Find all open issues with in progress development work with linked:pr.