Skip to content

R code stress / fuzz tester: tests all combinations of function input parameters enabling 100% code coverage & simplified unit testing

Notifications You must be signed in to change notification settings

cloudcell/fuzztest

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

fuzztest

(Please read a supplementary description here: https://gtdir.wordpress.com/2016/03/02/fuzz-testing/)

Purpose

To isolate errors through identifying groups of parameter combinations that cause them.

What the Package Does

Tests all possible combinations of argument values for any function to produce test statistics and graphs to help identify and fix defects faster than by using more traditional approaches. The key feature of the package is its ability to present multiple test results in a very compact form.

Alternative Use

Every run automatically creates data for unit testing. Simply compare saved "reference" results against the ongoing development results. For now, one can check them by loading data files with test results and checking whether exit status is identical for all combinations of input parameters. (Timestamped data is saved in a current work directory after every test.)

Roadmap

  1. To arrange axes on a parallel plot automatically based on a relative influence of a particular parameter to make the visual analysis faster.
  2. To enable automatic comparison and visualization of a current test against saved reference data.
  3. To report "incremental" "PASS/FAIL" i.e. only differences between the current test and previously recorded reference data.

Demo Visual Output

Below are a several demo test graphs showing parameter settings leading to function failures (in color). Axes are used only to order and group tests by cardinal_numbers of argument parameters.

demo

demo

PASS Results Only
demo

FAIL Results Only
demo

About

R code stress / fuzz tester: tests all combinations of function input parameters enabling 100% code coverage & simplified unit testing

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages