Skip to content

Issues: openai/evals

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
85 Open 118 Closed
85 Open 118 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Accuracy Score
#1328 opened Aug 3, 2023 by jeyarajcs
Find claims from research paper
#1338 opened Aug 21, 2023 by ghost
Publish latest evals framework to PyPI
#1344 opened Aug 31, 2023 by robatwilliams
Feature request for evals: Add support for function call.
#1346 opened Sep 7, 2023 by srenault
Evaluate the cost of running tests
#1350 opened Sep 15, 2023 by onjas-buidl
Use github.com/apssouza22/chatflow as a conversational layer. It would enable actual API requests to be carried out from natural language inputs.
#1362 opened Sep 25, 2023 by GiovanniSmokes
Context window of completion functions not accounted for bug Something isn't working
#1377 opened Oct 13, 2023 by pskl
Should random collection of values be supported?
#1382 opened Oct 17, 2023 by assert6
Proposal for Adding a New Evaluation Metric: Sentiment Analysis Accuracy
#1419 opened Nov 23, 2023 by Sarfaraz021
Improvements to Match: case insensitive and strip
#1421 opened Nov 30, 2023 by LoryPack
Request to change arithmetical_puzzles prompting
#1448 opened Dec 24, 2023 by ArcticBeat05
Support multiple completions for ModelbasedClassify
#1484 opened Mar 14, 2024 by tom-christie
OpenAIChatCompletionFn should __init__ should accept **kwargs bug Something isn't working
#1493 opened Mar 15, 2024 by ezraporter
1
Setting completion function args via CLI does not work bug Something isn't working
#1504 opened Mar 27, 2024 by LoryPack
1
Multiple Unit Test Failures Across OpenAI Assistants, Anthropic, and Google Gemini Libraries bug Something isn't working
#1536 opened Jun 24, 2024 by sakher
Music evals Idea for Eval These issues keep track of requests for different kinds of eval PRs
#138 opened Mar 15, 2023 by bhack
1
Add BigBench Tasks for evaluation Idea for Eval These issues keep track of requests for different kinds of eval PRs
#153 opened Mar 15, 2023 by Muhtasham
1
Update Autoflake to Ruff for faster pre-commit
#347 opened Mar 19, 2023 by imjuanleonard
1
Create an evaluation that measures a model's ability to remember specifics about texts in it's dataset? Idea for Eval These issues keep track of requests for different kinds of eval PRs
#383 opened Mar 21, 2023 by mrconter1
1
Suggestion: add git-hook hook for pre-commit
#384 opened Mar 21, 2023 by machinekoder
1
Request for Repo: "Open Source, Daily Auto-Generated SOTA LLM Model Benchmarks"
#505 opened Mar 29, 2023 by Marviel
1
Idea for eval - Bible knowledge as tested in the yearly International Bible youth Contest
#840 opened Apr 27, 2023 by ido777
1
pip install evals throws AssertionError bug Something isn't working
#918 opened May 4, 2023 by CholoTook
1
eval approach: GPT4 provides alternative responses with reasoning and then has to pick the best one
#968 opened May 14, 2023 by qrdlgit
1
oaieval fails with error UnicodeDecodeError: 'charmap' codec can't decode byte bug Something isn't working
#980 opened May 17, 2023 by rslinford
1
ProTip! Follow long discussions with comments:>50.