Skip to content

Simple rust solution for 1 billion rows challenge

License

Notifications You must be signed in to change notification settings

kuznetsss/1brc_rust

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

brc

Simple solution for 1 billion rows challenge.

I assumed to have a limitation to use only standard library and no crates.

The solution has multithread mode (default, 8 threads) and single thread modes (activated by -s flag). Also printing result could be disabled by -q flag. Because I'm not using any crates cli flags processing is very basic.

On my mac single thread solution takes 80 seconds, multithread solution takes 12 seconds (input file is 14 GB).

What could be improved

  • Map file with data into memory (memmap2 crate is required)
  • Faster hashmap (hashbrown crate is required)

Getting data

File measurements.txt is required to be in the repo to measure performance. It could be generated by steps from 1brc's instruction. I used docker, because I don't have java on my mac:

git clone https://github.com/gunnarmorling/1brc/
cd 1brc
docker run -it --mount source=$(pwd),target=/home eclipse-temurin:21 /bin/bash
# inside docker:
cd home
./mvnw clean verify
./create_measurements.sh 1000000000
exit
# on host
mv ./measurements.txt <1brc_rust path>

Thanks

About

Simple rust solution for 1 billion rows challenge

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages