-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[evals] Refactor evals package to expose completion_fn
.
#515
Merged
Merged
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
d87a056
[evals] Refactor evals package to expose `completion_fn`.
hwchung27 d9c1395
Add `record_raw_samples`
hwchung27 a1c6207
Andrew/evals refactor (#579)
andrew-openai deb29d3
update manifest and pyproject to support fetching data on pip install…
andrew-openai 9b1c350
we need to still use the interop for string/list[dicts] for modelgrad…
andrew-openai c470d52
refactor simple evals to not use result.prompt (#593)
andrew-openai b691cfa
Clean up duplicate recordings
hwchung27 7266049
Replace ModelSpecs with CompletionFn (#594)
jwang47 b2a45cf
Add --registry_path CLI arg (#601)
jwang47 924d2d4
Andrew/langchain llms (#602)
andrew-openai 4401cce
rm sample freeform, some docs (#603)
andrew-openai 013d636
Update completion-fn-protocol.md
andrew-openai 08062bc
some documentation cleanup
joe-at-openai 3367006
some documentation cleanup
joe-at-openai 5e71a76
some documentation cleanup
joe-at-openai e621b6f
inner monologue example (#610)
andrew-openai 49d17ed
Update README.md
andrew-openai 1bfba77
Update run-evals.md
andrew-openai b018aff
cleanup
andrew-openai 5222f2c
Merge branch 'main' into evals_refactor_merge_main
andrew-openai 9db703d
get oaieval to run
andrew-openai 02bc2cb
address comments
andrew-openai 50114a5
bump version
andrew-openai File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Merge branch 'main' into evals_refactor_merge_main
- Loading branch information
commit 5222f2c453f83289fefe095ce660bd5e6ee2b791
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
You are viewing a condensed version of this merge commit. You can view the full changes here.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We probably don't need this, can just keep the previous line:
recursive-include evals/registry/data *.jsonl