Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

spar mt prompt invar #184

Merged
merged 55 commits into from
Apr 20, 2023
Merged

spar mt prompt invar #184

merged 55 commits into from
Apr 20, 2023

Commits on Mar 14, 2023

  1. add boolq_pt template and christykoh as included user

    @ChristyKoh
    ChristyKoh committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    7b68f5f View commit details
    Browse the repository at this point in the history

  2. add templates.yaml for boolqpt

    @ChristyKoh
    ChristyKoh committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    edfdd6d View commit details
    Browse the repository at this point in the history

Commits on Mar 26, 2023

  1. pt yaml

    @reaganjlee
    reaganjlee committed Mar 26, 2023
    Configuration menu
    Copy the full SHA
    d285798 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2023

  1. Merge remote-tracking branch 'origin/main' into spar_boolq_pt

    @reaganjlee
    reaganjlee committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    6c91be2 View commit details
    Browse the repository at this point in the history

  2. add ag_news template, translated to pt

    @ChristyKoh
    ChristyKoh committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    0ff1609 View commit details
    Browse the repository at this point in the history

  3. add ag_news template, translated to pt

    @ChristyKoh
    ChristyKoh committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    7180a64 View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2023

  1. save eval runs to separate subfolders by target dataset

    @ChristyKoh
    ChristyKoh committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    b8f5e8b View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    51fed5d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into spar_pt

    @ChristyKoh
    ChristyKoh committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    9cebbdd View commit details
    Browse the repository at this point in the history

  4. Merge branch 'spar_boolq_pt' into spar_pt

    @ChristyKoh
    ChristyKoh committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    6be825a View commit details
    Browse the repository at this point in the history

  5. eval multiple datasets

    @ChristyKoh
    ChristyKoh committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    2660844 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'eval_dirs' of github.com:EleutherAI/elk into eval_dirs

    @ChristyKoh
    ChristyKoh committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    f35a6a8 View commit details
    Browse the repository at this point in the history

  7. change prompt answer chouces to portuguese

    @ChristyKoh
    ChristyKoh committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    b230209 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2023

  1. add imdb_pt template

    @ChristyKoh
    ChristyKoh committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    74c9915 View commit details
    Browse the repository at this point in the history

  2. implement prompt sharing, generate combined templates.yaml

    @ChristyKoh
    ChristyKoh committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    85fd9e4 View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2023

  1. fix num templates logic

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    8383f26 View commit details
    Browse the repository at this point in the history

  2. fix pt answer choice

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    df41ab4 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'spar_pt' of github.com:EleutherAI/elk into spar_mt_prom… 

    …pt_invar
    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    c03742d View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' of github.com:EleutherAI/elk into main

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    e536aa6 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into spar_pt

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    208200d View commit details
    Browse the repository at this point in the history

  6. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    a132f40 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'spar_pt' of github.com:EleutherAI/elk into spar_mt_prom… 

    …pt_invar
    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    9f3f218 View commit details
    Browse the repository at this point in the history

  8. remove empty prompt_dataset file

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    c71bf1c View commit details
    Browse the repository at this point in the history

  9. fix empty prompters bug

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    1ec5787 View commit details
    Browse the repository at this point in the history

  10. fix multiclass label bug

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    66c7a6b View commit details
    Browse the repository at this point in the history

  11. move prompt combination to PromptConfig post_init logic

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    d91acac View commit details
    Browse the repository at this point in the history

  12. Merge branch 'main' of github.com:EleutherAI/elk into main

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    89b2346 View commit details
    Browse the repository at this point in the history

  13. Merge branch 'main' into spar_mt_prompt_invar

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    715bba8 View commit details
    Browse the repository at this point in the history

  14. fix refactor bugs, runnable state

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    0c2f5c4 View commit details
    Browse the repository at this point in the history

  15. rewrite template merging, regenerate prompter every run

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    066cd44 View commit details
    Browse the repository at this point in the history

  16. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    846b78c View commit details
    Browse the repository at this point in the history

  17. line len fixes

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    1aecd7e View commit details
    Browse the repository at this point in the history

  18. Merge branch 'spar_mt_prompt_invar' of github.com:EleutherAI/elk into… 

    … spar_mt_prompt_invar
    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    b7bbee0 View commit details
    Browse the repository at this point in the history

  19. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    b0c0f63 View commit details
    Browse the repository at this point in the history

  20. Merge remote-tracking branch 'origin/main' into spar_mt_prompt_invar

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    34bc364 View commit details
    Browse the repository at this point in the history

  21. update README with prompt invariance argument

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    2da069a View commit details
    Browse the repository at this point in the history

  22. fix bugs, add dataset col checks

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    4c6d344 View commit details
    Browse the repository at this point in the history

  23. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    53d186b View commit details
    Browse the repository at this point in the history

  24. fix prompter init typing

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    b78355f View commit details
    Browse the repository at this point in the history

  25. Merge branch 'spar_mt_prompt_invar' of github.com:EleutherAI/elk into… 

    … spar_mt_prompt_invar
    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    3486ded View commit details
    Browse the repository at this point in the history

  26. Update README.md

    @lauritowal
    lauritowal committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    1975410 View commit details
    Browse the repository at this point in the history

  27. try to fix typing again

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    8a5fb0d View commit details
    Browse the repository at this point in the history

  28. Merge branch 'spar_mt_prompt_invar' of github.com:EleutherAI/elk into… 

    … spar_mt_prompt_invar
    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    6d46f7e View commit details
    Browse the repository at this point in the history

  29. assert datasettemplates type

    @ChristyKoh
    ChristyKoh committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    a7f5a8b View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2023

  1. Merge remote-tracking branch 'origin/main' into eval_dirs

    @ChristyKoh
    ChristyKoh committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    ba090b7 View commit details
    Browse the repository at this point in the history

  2. bugfix to run eval separately on each dataset

    @ChristyKoh
    ChristyKoh committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    7af1a1b View commit details
    Browse the repository at this point in the history

  3. add combine_evals flag to differentiate a multi dataset eval from a b… 

    …atch of single-dataset evals
    @ChristyKoh
    ChristyKoh committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    74551fa View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2023

  1. Merge branch 'main' of github.com:EleutherAI/elk into main

    @ChristyKoh
    ChristyKoh committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    0b3d3c9 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into eval_dirs

    @ChristyKoh
    ChristyKoh committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    d8cee8b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'eval_dirs' into spar_mt_prompt_invar, separate combinin… 

    …g logic into functions
    @ChristyKoh
    ChristyKoh committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    54aa710 View commit details
    Browse the repository at this point in the history

  4. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    f7a4713 View commit details
    Browse the repository at this point in the history

  5. define ds_name

    @ChristyKoh
    ChristyKoh committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    6e2f54c View commit details
    Browse the repository at this point in the history

  6. Merge branch 'spar_mt_prompt_invar' of github.com:EleutherAI/elk into… 

    … spar_mt_prompt_invar
    @ChristyKoh
    ChristyKoh committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    3b0765d View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2023

  1. fix ds_name bug

    @ChristyKoh
    ChristyKoh committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    5f0f32a View commit details
    Browse the repository at this point in the history

  2. Update README.md

    @lauritowal
    lauritowal committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    8fa07b4 View commit details
    Browse the repository at this point in the history