Skip to content
/ qshuf Public

A tool for shuffling rows in file that doesn't load the input into the memory

License

Notifications You must be signed in to change notification settings

wsuchy/qshuf

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Qshuf is a memory efficient replacement for GNU shuf

It might be used by data scientist when one has gigabytes of data in one file and need to have it randomised.

Qshuf will not read the source file into memory. Instead it analyzes the file by looking for places where lines start. Then it randomly reads one line from the input and outputs it to stdout.

The only supported newline format is linux (\n)

About

A tool for shuffling rows in file that doesn't load the input into the memory

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published