Skip to content

Issues: openai/evals

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
84 Open 117 Closed
84 Open 117 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Make GPT4 aware of the evals format
#143 opened Mar 15, 2023 by bhack
The metrics valus are not right
#414 opened Mar 23, 2023 by lshowway
A way to evaluate GPT models
#542 opened Apr 1, 2023 by qrdlgit
Move/Include directions on how to contribute a simple eval into top-level README.md
#545 opened Apr 1, 2023 by jonathanagustin
Eval requirements when using GPT-4 for renovation price estimations?
#560 opened Apr 2, 2023 by jjyuhub
Data formatting step outputs incorrectly keys for JSONL
#567 opened Apr 3, 2023 by ridoy
How deterministic should evals be?
#629 opened Apr 10, 2023 by garyrob
I believe that in order for GPT-4 to create by itself many inventions through two month, necessary that...
#685 opened Apr 15, 2023 by 55255ru
Mandarin homophones eval 🇨🇳
#689 opened Apr 15, 2023 by ofou
Where to find the experiment comparation: Using the data of training reward model for fine-tuning without reinforcement learning.
#711 opened Apr 18, 2023 by guotong1988
fix error
#765 opened Apr 23, 2023 by Huymoixode
Eval idea: Security code review for unicode attacks on code Idea for Eval These issues keep track of requests for different kinds of eval PRs
#787 opened Apr 24, 2023 by qrdlgit
Please add an option to change language or understand other languages
#795 opened Apr 24, 2023 by 0XxMuzanxX0
Safety Eval Idea: allergen information of different food products in the Israeli market.
#874 opened Apr 30, 2023 by ido777
Inaccuracy of AI-generated responses bug Something isn't working
#906 opened May 3, 2023 by vmn2014
docs out of date bug Something isn't working
#929 opened May 7, 2023 by rustam-e
Overall Report for Eval Set As a Whole
#939 opened May 8, 2023 by chinggg
Remove leading whitespace from sampled answers in Match template
#947 opened May 9, 2023 by keltin13
Private local memory coordination
#1048 opened May 27, 2023 by rpsnoopy
pip package is outdated (1.0.3.post1) - has broken elsuite/basic/includes.py bug Something isn't working
#1098 opened Jun 4, 2023 by phelps-sg
Add version option to oaieval
#1102 opened Jun 5, 2023 by phelps-sg
Extracting and analyzing data from scientific articles
#1126 opened Jun 8, 2023 by QuantumCoderr
Functions: Support for minItems and maxItems for json schema array
#1173 opened Jun 15, 2023 by tjhiggins
closedqa prompt is not adequate for gpt-4-0613
#1228 opened Jun 24, 2023 by JasonGross
1
Code Evals
#1275 opened Jul 3, 2023 by billxbf
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.