Commits

History for

Commits on Oct 27, 2023

Add new features for quickly finding potential training data
carlini
committedOct 27, 2023

Commits on Jun 6, 2022

Fix two array out of bounds reads, including google-research#17
carlini
committedJun 6, 2022

Commits on May 1, 2022

Count occurrences fix part 2
carlini
committedMay 1, 2022
Last occurence before EOF is now counted correctly (fixes google-research#15 )
carlini
committedMay 1, 2022
Fix tabs to spaces
carlini
committedMay 1, 2022
Fixes google-research#14 count occurrences EOF error
carlini
committedMay 1, 2022

Commits on Apr 15, 2022

Fail early on crashes; purge cache directory on rerun (Fixes google-research#11 )
carlini
committedApr 15, 2022

Commits on Mar 11, 2022

Merge branch 'dev-v1' of github.com:google-research/deduplicate-text-datasets into dev-v1
carlini
committedMar 11, 2022

Commits on Mar 8, 2022

Silence last warnings
carlini
committedMar 8, 2022

Commits on Mar 7, 2022

Simplify process
carlini
committedMar 7, 2022

Commits on Mar 3, 2022

Commits on Feb 24, 2022

More code commenting and cleanup
carlini
committedFeb 24, 2022
Once upon a time I was told to comment my code
carlini
committedFeb 24, 2022
Better collection
carlini
committedFeb 24, 2022

Commits on Feb 23, 2022

Full pipeline works again
carlini
committedFeb 23, 2022
Fixing across-similar script
carlini
committedFeb 23, 2022
Building suffix array works correctly
carlini
committedFeb 23, 2022
Start a v1 rewrite; make individual functions for each command; new argument parsing.
carlini
committedFeb 23, 2022

Commits on Dec 10, 2021

Clean up warnings and clarify selfsimilar_parallel output
carlini
committedDec 10, 2021

Commits on Jul 16, 2021

Initial commit
carlini
committedJul 16, 2021

Loading