New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert GPT2Dataset back to old working state #57

Merged

lucidrains merged 2 commits into main from sdtblck-patch-2

Jan 13, 2021

Contributor

sdtblck commented Jan 13, 2021

had to decrease batch size a fair bit
also cut out generation in the train script as this doesn't use the deepspeed engine, so can cause ooms
but it finally works again haha. I guess one of the tf2 dataloading classes was having some weird interactions with our GPUs. Who really knows.

sdtblck added 2 commits

January 13, 2021 01:19


revert dataset fn back to working state

3c8a969


Update gpt3_small.json

2ec82d1

sdtblck requested a review from a team as a code owner

January 13, 2021 00:23

sdtblck requested review from lucidrains and leogao2

January 13, 2021 00:23

lucidrains approved these changes

View reviewed changes

lucidrains merged commit f229ab2 into main

lucidrains deleted the sdtblck-patch-2 branch

January 13, 2021 00:35

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment