Skip to content

Commit

Permalink
Upgrade to PretrainDataset and FinetuneDataset (#73)
Browse files Browse the repository at this point in the history 
* add pretrain finetune datasets and tests

* update train script and fix datasets/tests

* add finetuning tests and fix resume

* fix tokenizer data aug bug
  • Loading branch information
@loubbrad
loubbrad committed Dec 5, 2023
1 parent 55f5068 commit 3a396b6
Show file tree
Hide file tree
Showing 7 changed files with 942 additions and 505 deletions.
Loading

0 comments on commit 3a396b6

Please sign in to comment.