Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

streaming multipack for pretraining dataset #959

Merged
merged 9 commits into from
Jan 6, 2024
Merged

streaming multipack for pretraining dataset #959

merged 9 commits into from
Jan 6, 2024

Commits on Jan 5, 2024

  1. [Feat] streaming multipack

    @winglian
    [email protected] authored and winglian committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    8ed5bcb View commit details
    Browse the repository at this point in the history

  2. WIP make continued pretraining work w multipack

    @winglian
    winglian committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    da9aee1 View commit details
    Browse the repository at this point in the history

  3. fix up hadrcoding, lint

    @winglian
    winglian committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    36b244d View commit details
    Browse the repository at this point in the history

  4. fix dict check

    @winglian
    winglian committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    680cbe2 View commit details
    Browse the repository at this point in the history

  5. update test for updated pretraining multipack code

    @winglian
    winglian committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    789c972 View commit details
    Browse the repository at this point in the history

  6. fix hardcoded data collator fix for multipack pretraining

    @winglian
    winglian committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    2a49248 View commit details
    Browse the repository at this point in the history

  7. fix the collator to be the max length for multipack pretraining

    @winglian
    winglian committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    7c3be2e View commit details
    Browse the repository at this point in the history

  8. don't bother with latest tag for test

    @winglian
    winglian committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    bea8bee View commit details
    Browse the repository at this point in the history

  9. cleanup docker build/test

    @winglian
    winglian committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    5a321c3 View commit details
    Browse the repository at this point in the history