Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[evals] Refactor evals package to expose completion_fn. #515

Merged
merged 23 commits into from
Apr 11, 2023
Merged

[evals] Refactor evals package to expose completion_fn. #515

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
d87a056
[evals] Refactor evals package to expose `completion_fn`.
hwchung27 Mar 29, 2023
d9c1395
Add `record_raw_samples`
hwchung27 Apr 2, 2023
a1c6207
Andrew/evals refactor (#579)
andrew-openai Apr 5, 2023
deb29d3
update manifest and pyproject to support fetching data on pip install…
andrew-openai Apr 5, 2023
9b1c350
we need to still use the interop for string/list[dicts] for modelgrad…
andrew-openai Apr 5, 2023
c470d52
refactor simple evals to not use result.prompt (#593)
andrew-openai Apr 5, 2023
b691cfa
Clean up duplicate recordings
hwchung27 Apr 6, 2023
7266049
Replace ModelSpecs with CompletionFn (#594)
jwang47 Apr 6, 2023
b2a45cf
Add --registry_path CLI arg (#601)
jwang47 Apr 6, 2023
924d2d4
Andrew/langchain llms (#602)
andrew-openai Apr 7, 2023
4401cce
rm sample freeform, some docs (#603)
andrew-openai Apr 7, 2023
013d636
Update completion-fn-protocol.md
andrew-openai Apr 7, 2023
08062bc
some documentation cleanup
joe-at-openai Apr 10, 2023
3367006
some documentation cleanup
joe-at-openai Apr 10, 2023
5e71a76
some documentation cleanup
joe-at-openai Apr 10, 2023
e621b6f
inner monologue example (#610)
andrew-openai Apr 10, 2023
49d17ed
Update README.md
andrew-openai Apr 10, 2023
1bfba77
Update run-evals.md
andrew-openai Apr 10, 2023
b018aff
cleanup
andrew-openai Apr 10, 2023
5222f2c
Merge branch 'main' into evals_refactor_merge_main
andrew-openai Apr 10, 2023
9db703d
get oaieval to run
andrew-openai Apr 10, 2023
02bc2cb
address comments
andrew-openai Apr 11, 2023
50114a5
bump version
andrew-openai Apr 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Prev Previous commit
Next Next commit
update manifest and pyproject to support fetching data on pip install (… 
…#592)

This supports downloading all data on pip install from git, e.g. `pip
install git+https://github.com/openai/evals.git@main`
  • Loading branch information
@andrew-openai
andrew-openai committed Apr 5, 2023
commit deb29d3f89169f17a67fbb4075494bf1d7847170
1 change: 1 addition & 0 deletions MANIFEST.in
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
recursive-include evals *.py
recursive-include evals *.yaml
recursive-include evals *.sql
recursive-include evals/registry/data *.jsonl
1 change: 1 addition & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ dependencies = [
"pyyaml",
"sacrebleu",
"matplotlib",
"setuptools_scm",
jwang47 marked this conversation as resolved.
Show resolved Hide resolved
]

[project.scripts]
Expand Down