Commits

Commits on Sep 15, 2023

Have multiple options
Kyle1668
committedSep 15, 2023
Begin focusing just on ppl
Kyle1668
committedSep 15, 2023
Introduce parallel
Kyle1668
committedSep 15, 2023

Commits on Sep 14, 2023

Updated files to handle new schema
Kyle1668
committedSep 14, 2023

Commits on Aug 18, 2023

Fix batch size issue
Kyle1668
committedAug 18, 2023

Commits on Aug 9, 2023

Created configureble batch sizes
Kyle1668
committedAug 9, 2023
Checkpoint
Kyle1668
committedAug 9, 2023

Commits on Aug 7, 2023

Begin making code parralel
Kyle1668
committedAug 7, 2023

Commits on Jul 15, 2023

Add error handling for when there is an exception in the pple code
Kyle1668
committedJul 15, 2023

Commits on Jul 5, 2023

Load LLMs in FP16 for faster inference
Kyle1668
committedJul 5, 2023

Commits on Jul 4, 2023

Merge pull request #9 from jaydeepborkar/attention
uSaiPrashanth
committedJul 4, 2023
Calculate duplicates: Save bucket only at the end
uSaiPrashanth
committedJul 4, 2023

Commits on Jul 3, 2023

Merge branch 'EleutherAI:master' into attention
jaydeepborkar
committedJul 3, 2023
Update inference.py
jaydeepborkar
committedJul 3, 2023
Merge pull request #1 from uSaiPrashanth/attention
jaydeepborkar
committedJul 3, 2023
Fix typo while saving json
uSaiPrashanth
committedJul 3, 2023
MPI C++ based duplicate filters
uSaiPrashanth
committedJul 3, 2023
Add Unigram Perplexity and freq analysis
uSaiPrashanth
committedJul 3, 2023
Modularize save_inference_log()
uSaiPrashanth
committedJul 3, 2023

Commits on Jun 27, 2023

Update inference.py
jaydeepborkar
committedJun 27, 2023
attention entropy and gini coefficient
jaydeepborkar
committedJun 27, 2023

Commits on Jun 25, 2023

Add Full Pile Loading script
uSaiPrashanth
committedJun 25, 2023

Commits on Jun 9, 2023

Commits on Jun 1, 2023

Merge pull request #5 from EleutherAI/alvin/duplicated-filters
Kyle1668
committedJun 1, 2023

Commits on May 29, 2023

update func names
alvin319
committedMay 29, 2023
comments
alvin319
committedMay 29, 2023
update reqs
alvin319
committedMay 29, 2023
add filters
alvin319
committedMay 29, 2023

Commits on May 25, 2023

Get perplexity for prompt, generation, and whole sequence
Kyle1668
committedMay 25, 2023
Add module docstring
Kyle1668
committedMay 25, 2023
Set up first draft of an evaluation pipeline
Kyle1668
committedMay 25, 2023

Commits on May 24, 2023

Added features arg
Kyle1668
committedMay 24, 2023
Add line limits
Kyle1668
committedMay 24, 2023
Formatted and comemnted the pipeline
Kyle1668
committedMay 24, 2023