Skip to content

Commit

Permalink
readme, man page, changelog updates
Browse files Browse the repository at this point in the history
  • Loading branch information
simonmichael committed Oct 17, 2016
1 parent b49b75e commit ec55f98
Show file tree
Hide file tree
Showing 3 changed files with 57 additions and 14 deletions.
3 changes: 2 additions & 1 deletion CHANGES
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ quickbench changes:

- more robust command execution, signal handling, output, error reporting

- switch to docopt; options must precede args for now
- switch to docopt, options must precede args for now

- new UI: full commands are specified by arguments/file/stdin,
alternate executables can be specified with -w, flags cleanup
Expand All @@ -17,3 +17,4 @@ quickbench changes:

- -N/--cycles repeats the whole test suite

- man page
56 changes: 45 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,31 @@
# quickbench
# quickbench 1.0

Quick & easy benchmarking of command-line programs.

[About](#about)
| [Examples](#examples)
| [Usage](#usage)
| [Related](#related-work)


## About

This is a reboot of the simplebench benchmarking tool from the hledger
project. You can use it as a fancier "time" command for benchmarking
command-line programs, or as a haskell library (eg in package benchmark
suites).
quickbench is an update and repackaging of a benchmarking tool I've been
using in the hledger project since 2008.
Use it like a more powerful "time" command for benchmarking general command-line programs,
or for creating repeatable benchmark scripts for your projects.

quickbench is not smart or complicated like "bench" or criterion; it is
good for quick and dirty, exploratory, comparative measurements
that you can run quickly and understand at a glance.
I find it very useful; patches welcome!
quickbench produces very simple output (elapsed seconds),
as quickly as possible (running commands just once by default),
and tabulates results from multiple executables.
I find it very useful for quick and dirty, exploratory, and comparative measurements
that I (and others) can understand at a glance.

## Examples

Install it easily on most platforms with [stack](https://haskell-lang.org/get-started) (or cabal).
It's not yet on Hackage, so you'll need the source:

```bash
$ git clone https://github.com/simonmichael/quickbench.git
$ cd quickbench
Expand All @@ -35,7 +45,8 @@ Best times:
+---------++------+
```

or in a file.. `bench.sh` will be used by default:
or put them in a file and use `-f FILE`.
A file named `bench.sh` in the current directory will be used automatically:
```
$ echo 'echo 3 * 1000000' > bench.sh
$ quickbench
Expand All @@ -62,7 +73,8 @@ Best times:
+-------------++---------+---------+
```

and repeat tests to reduce and evaluate jitter:
and run tests repeatedly to reduce or observe jitter.
(quickbench assumes the quickest measurement is the truest one):
```
$ quickbench -w echo,expr -p5 -n100 -N2
Running 1 tests 100 times with 2 executables at 2016-10-16 23:57:34.387764 UTC:
Expand Down Expand Up @@ -155,3 +167,25 @@ Options:
-h, --help show this help
```

## Related

[bench](https://github.com/Gabriel439/bench#readme) (Gabriel Gonzalez 2016) is another
command line benchmarking tool written in Haskell.
Use that one if you need detailed statistical analysis and output, or HTML reports.
Here is bench's output for the echo/expr example above:
```
$ bench 'echo 3 * 1000000'; bench 'expr 3 \* 1000000'
benchmarking echo 3 * 1000000
time 2.215 ms (2.173 ms .. 2.250 ms)
0.995 R² (0.989 R² .. 0.998 R²)
mean 2.238 ms (2.203 ms .. 2.296 ms)
std dev 147.6 μs (92.84 μs .. 265.3 μs)
variance introduced by outliers: 48% (moderately inflated)
benchmarking expr 3 \* 1000000
time 3.484 ms (3.405 ms .. 3.556 ms)
0.994 R² (0.987 R² .. 0.998 R²)
mean 3.564 ms (3.497 ms .. 3.684 ms)
std dev 280.3 μs (178.0 μs .. 494.9 μs)
variance introduced by outliers: 52% (severely inflated)
```
12 changes: 10 additions & 2 deletions quickbench.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,14 @@ quickbench - quick & easy benchmarking of command-line programs

# DESCRIPTION

Runs some test commands, possibly with different executables, once or more
and shows their best execution times in tabular format.
quickbench runs some test commands, possibly with different executables,
once or more and shows their best execution times in tabular format.

It produces very simple output (elapsed seconds),
as quickly as possible (running commands just once by default),
and tabulates results from multiple executables.
I find it useful for quick and dirty, exploratory, and comparative
measurements that you (and others) can understand at a glance.

Commands are specified as one or more quote-enclosed arguments,
and/or one per line in CMDSFILE; or read from a default file [./bench.sh].
Expand Down Expand Up @@ -106,6 +112,8 @@ Home: https://github.com/simonmichael/quickbench

bench(1): https://github.com/Gabriel439/bench

time(1)

# COPYRIGHT

Copyright (C) 2008-2016 Simon Michael.
Expand Down

0 comments on commit ec55f98

Please sign in to comment.