Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove support for lazy dataset implementation #1033

Merged
merged 4 commits into from
Sep 18, 2023
Merged

Remove support for lazy dataset implementation #1033

merged 4 commits into from
Sep 18, 2023

Conversation

dashstander
Copy link
Contributor

Also specifies the valid data_impl options in neox_args.py

@dashstander dashstander requested a review from a team as a code owner September 15, 2023 19:36
@dashstander dashstander linked an issue Sep 15, 2023 that may be closed by this pull request
when preprocess data and load data using "lazy" mode #904
Closed
@Quentin-Anthony Quentin-Anthony enabled auto-merge (squash) September 18, 2023 13:53
@Quentin-Anthony Quentin-Anthony merged commit fcd5f92 into main Sep 18, 2023
2 checks passed
@Quentin-Anthony Quentin-Anthony deleted the 904-remove-lazy-data-impl branch September 18, 2023 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Quentin-Anthony Quentin-Anthony Quentin-Anthony approved these changes

@ShivanshuPurohit ShivanshuPurohit Awaiting requested review from ShivanshuPurohit ShivanshuPurohit was automatically assigned from EleutherAI/pm-gptneo

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

when preprocess data and load data using "lazy" mode
2 participants
@dashstander @Quentin-Anthony