Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: BlinkDL/RWKV-LM Loading
base: main
Choose a base ref
...
head repository: StellaAthena/RWKV-LM Loading
compare: main
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 3 commits
  • 2 files changed
  • 1 contributor

Commits on May 26, 2023

  1. Set num_tokens to the correct value

    @StellaAthena
    StellaAthena committed May 26, 2023
    Configuration menu
    Copy the full SHA
    b227471 View commit details
    Browse the repository at this point in the history

  2. Update trainer.py 

    Change warm-up to be constant LR
    @StellaAthena
    StellaAthena committed May 26, 2023
    Configuration menu
    Copy the full SHA
    dbdb36e View commit details
    Browse the repository at this point in the history

  3. Update train.py 

    Set warm-up to be a valid number that's approximately 15B tokens
    @StellaAthena
    StellaAthena committed May 26, 2023
    Configuration menu
    Copy the full SHA
    3996538 View commit details
    Browse the repository at this point in the history
Loading