Skip to content

Issues: EleutherAI/pythia

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
19 Open 82 Closed
19 Open 82 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Fine-tuning recommendations
#96 by RainIwakura was closed Apr 28, 2023 updated Apr 28, 2023
2
Train/valid/test split
#102 by choidami was closed May 21, 2023 updated May 21, 2023
1
Will memorization experimental codes be released?
#98 by chujiezheng was closed May 22, 2023 updated May 22, 2023
2
Revamp experiment organization and migrate code when necessary documentation Improvements or additions to documentation
#99 by StellaAthena was closed Jun 4, 2023 updated Jun 4, 2023
3 tasks
1
@StellaAthena
pythia-12b checkpoints missing on HuggingFace for step4000 and step32000
#107 by byungdoh was closed Jun 4, 2023 updated Jun 4, 2023
2
Possible error in Pythia-12B-deduped step 32000
#108 by smahdavi4 was closed Jun 15, 2023 updated Jun 15, 2023
2
Multiple training runs of same model with different random seed for weight initialisation
#110 by KarolisRam was closed Jun 22, 2023 updated Jun 22, 2023
1
What tool do you use for your data preprocessing/binarization?
#69 by ajesujoba was closed Apr 18, 2023 updated Jul 3, 2023
9
Can I provide custom data and continue training Pythia on this new data?
#113 by GeorgiAngelov was closed Jul 21, 2023 updated Jul 21, 2023
1
Is there an access to the deduplicated version of the data with meta info?
#92 by Jason3900 was closed Jul 21, 2023 updated Jul 21, 2023
6
Is there a template poilerplate for the prompt used in C.1 gender bias intervention?
#106 by ruyuan-zuo was closed Jul 21, 2023 updated Jul 21, 2023
1
Difference between LFS and HuggingFace datasets?
#112 by eric-mitchell was closed Jul 21, 2023 updated Jul 21, 2023
1
Clarification of Pythia tokenizer(s) at different sizes, steps and data preprocessing?
#115 by RylanSchaeffer was closed Aug 3, 2023 updated Aug 3, 2023
1
Convert the huggingface checkpoint to GPT-Neox checkpoint
#116 by ZhiYuanZeng was closed Aug 13, 2023 updated Aug 13, 2023
2
Weights tying
#117 by link-er was closed Oct 8, 2023 updated Oct 8, 2023
1
The performance about pythia and LLaMA model architecture
#122 by peiyingxin was closed Oct 30, 2023 updated Oct 30, 2023
1
Model Initialization Question
#129 by yanlai00 was closed Nov 4, 2023 updated Nov 4, 2023
1
Error when running unshard_memmap.py
#114 by ShaneeyS was closed Nov 4, 2023 updated Nov 4, 2023
2
The value of weight decay
#132 by yehuitang was closed Nov 9, 2023 updated Nov 9, 2023
1
the loss of pythia training
#97 by Wangpeiyi9979 was closed May 1, 2023 updated Nov 10, 2023
3
Deduplicated Pile dataset with Domain Attribution
#137 by michaelduan8 was closed Nov 21, 2023 updated Nov 21, 2023
Replicating the Training Data Order
#136 by prakharg24 was closed Nov 23, 2023 updated Nov 23, 2023
1
.
#140 by ParthaKrPaul was closed Dec 2, 2023 updated Dec 2, 2023
2
Pytia or GPT-neox?
#138 by borgr was closed Dec 12, 2023 updated Dec 12, 2023
1
Details about "EleutherAI/pythia-160m-seed*" models
#142 by IanMagnusson was closed Dec 12, 2023 updated Jan 4, 2024
3
ProTip! Type g i on any issue or pull request to go back to the issue listing page.