Commits

Commits on Mar 21, 2024

AnthropicSolver (openai#1498 )
thesofakillers
committedMar 21, 2024
Add Human-Relative MLAgentBench (openai#1496 )
danesherbs
committedMar 21, 2024
Add Multi-Step Web Tasks (openai#1500 )

danesherbs
and
ianmckenzie-oai
committedMar 21, 2024

Commits on Mar 19, 2024

Add In-Context RL eval (openai#1491 )
james-aung
committedMar 19, 2024
Add Function Deduction eval (openai#1492 )

james-aung
and
johny-b
committedMar 19, 2024
Identifying Variables Eval (openai#1488 )
thesofakillers
committedMar 19, 2024
Track the Stat Eval (openai#1489 )
thesofakillers
committedMar 19, 2024
Already Said That Eval (openai#1490 )
thesofakillers
committedMar 19, 2024
Add 20 questions eval (openai#1499 )
inwaves
committedMar 19, 2024
Add skill acquisition eval (openai#1497 )
inwaves
committedMar 19, 2024
Error Recovery Eval (openai#1485 )

ojaffe
and
ianmckenzie-oai
committedMar 19, 2024
Can't Do That Anymore Eval (openai#1487 )
ojaffe
committedMar 19, 2024
Bugged Tools Eval (openai#1486 )
ojaffe
committedMar 19, 2024

Commits on Mar 13, 2024

Updates on existing evals; readmes; solvers (openai#1483 )
ojaffe
committedMar 13, 2024

Commits on Mar 12, 2024

Drop two datasets from steganography (openai#1481 )
thesofakillers
committedMar 12, 2024

Commits on Feb 23, 2024

Suppress 'HTTP/1.1 200 OK' logs from openai library (openai#1468 )
JunShern
committedFeb 23, 2024

Commits on Feb 13, 2024

Fix small typos and inconsistencies in README (openai#1464 )
kwinkunks
committedFeb 13, 2024

Commits on Jan 29, 2024

Updates for Solvers (openai#1461 )

committedJan 29, 2024

Commits on Jan 26, 2024

Logged spec now includes overridden args (openai#1460 )
ojaffe
committedJan 26, 2024
Add run_id to final_report from LocalRecorder (openai#1452 )
ianmckenzie-oai
committedJan 26, 2024

Commits on Jan 10, 2024

Fix formatting/typing so pre-commit hooks pass (openai#1451 )
ianmckenzie-oai
committedJan 10, 2024
icelandic gec eval (openai#1400 )
svanhvitlilja
committedJan 10, 2024

Commits on Jan 9, 2024

Add eval yaml for Theory of Mind eval (openai#1453 )
ojaffe
committedJan 9, 2024

Commits on Jan 3, 2024

Improve MMMU performance with prompt engineering (openai#1450 )
etr2460
committedJan 3, 2024
Add eval japanese prime minister (openai#1422 )
return-nil
committedJan 3, 2024
Solve openai#1394 (openai#1395 )
LoryPack
committedJan 3, 2024
Add a recorder for function calls (openai#1389 )
danesherbs
committedJan 3, 2024
Add gpt-3.5-turbo-16k support to ctx len getter (openai#1388 )

danesherbs
and
ianmckenzie-oai
committedJan 3, 2024
Fixed parameter incorrect (openai#1378 )
assert6
committedJan 3, 2024

Commits on Dec 24, 2023

Randomly select MMMU answer when none is returned from the model (openai#1447 )
etr2460
committedDec 24, 2023

Commits on Dec 21, 2023

Change wrong kwargs name (openai#1435 )
LoryPack
committedDec 21, 2023
Fix Pydantic warning on data_test run (openai#1445 )
inwaves
committedDec 21, 2023
Fix small typo in oaieval run function (openai#1438 )
inwaves
committedDec 21, 2023
Release 2.0.0 (openai#1444 )
etr2460
committedDec 21, 2023
Add MMMU evals and runner (openai#1442 )
etr2460
committedDec 21, 2023