General zero and few-shot evaluations on model suite

[StellaAthena]

Collect zero-shot and 5-shot performance on the Pythia model suite across a selected subset of checkpoints over time.

Tasks:

• MMLU (57 tasks)
• BLiMP
• PiQA
• SciQ
• Lambada
• Wikitext
• Winogrande
• WSC
• Arc-challenge and ARC-easy
• Logiqa

[haileyschoelkopf]

@Kyle1668 this is the issue, I can't assign you for some reason. Let's track statuses in the table in the Pythia README! (maybe in an evals branch if needed).

[StellaAthena]

@haileyschoelkopf you can’t assign him because he’s not in the EAI GitHub group. I’ll send him an invite

[Kyle1668]

Sounds good y'all! I've created a branch to track what I'm up to. https://github.com/EleutherAI/pythia/blob/8f4cb1004dab7fb33c3f789fb0b82cf6477a4dee/README.md