-
Notifications
You must be signed in to change notification settings - Fork 10
Load testing tools for Flume
License
mpercy/flume-load-gen
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Load testing and validation tools for Flume. At the moment, it caters to TCP-based syslog load testing. Files: bin/start_hammer.sh: start script for the load generator bin/stop_hammer.sh: stop script for the load generator bin/validate.pl: validates that its input appears as sequential, increasing integers bin/pig_validate.sh: runs validate.pl via pig in streaming mode bin/sort_results.pl: helper to sort the output of several validate.pl runs concatenated together (only for human readability) bin/summarize_results.pl: summarizes the output from concatenated validate.pl runs bin/hammer_aggregate.sh: aggregates and prints snapshot from hammer client logs. Best to run this periodically in a loop. src/main/pig/validate.pig: pig script to stream validate.pl src/main/java/org/apache/flume/tcphammer/Hammer.java: load testing tool Notes: To build the load generator, just run mvn clean package; it will leave an executable jar in target/ pig_validate.sh will store a bunch of files on HDFS, then you can just do hadoop fs -cat hdfs:https://output_dir > results.txt If you want it to be pretty (sorted) so you can make sense of it, run ./bin/sort_results.pl < results.txt > results-sorted.txt If you want to get a summary of the results, run ./bin/summarize_results.pl < results.txt | grep -v DEBUG > summary.txt
About
Load testing tools for Flume
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published