Data oriented stateful test-check generator for Clojure & ClojureScript.
There are already a couple existing libraries for doing stateful property based testing in Clojure:
These libraries are to varying degrees inspired by features available in QuickCheck. Often the motivating examples involve stateful data structures and programming in the "small", so to speak. In addition, the existing literature as well as libraries couple the underlying symbolic generation with direct execution against a concrete API.
What if you want to test an entire system? In this case, the complexity of the data structures involved in specific API method signatures means a significant amount of generation time is consumed on details you don't care about when the goal is simply to produce a long sequence of commands.
Thus, fugato emphasizes symbolic generation only, and by guiding the user away from the specifics of the API permits the efficient generation of long command sequences without needing to adopt sophisticated search strategies.
Once the command sequence is generated these be can run however you see fit. In this sense, fugato is not a testing framework or even a methodology. It simply generates commands and supports shrinking. The user can leverage existing testing tools without having to learn new concepts beyond the task of command generation.
Since fugato is abstract, the focus is primarily on modeling your domain. In particular since there is no pressure to program against any real interface, it's natural to model complex scenarios involving multiple actors and multiple manipulatable objects with complex relationships. Once the modeling is done you can choose how to run the commands against your real system.
A model is a map of command specifications:
{:withdraw withdraw-command-spec
:deposit deposit-command-spec}A command spec is a map that looks like the following:
{:args args-generator
:freq an-integer
:next-state next-state-fn
:run? run?-fn
:valid? valid?-fn}args-generator - a function that takes a state and returns a generator for
the arguments of the command. Only this part requires a basic understanding how
clojure.test.check.generators works.
:freq - an integer to set the likelihood of the command to be generated.
Defaults to 1.
:next-state - a function take a state and the generated command and computes
the next state.
:run? - a function that takes a state to determine whether a particular command can be
generated. Instead of randomly generating commands, you can inspect the state
and control which commands are available at each step.
:valid? - A function that takes state and a command that determines whether a command
should discard during shrinking (the process by which a minimal failing command
sequence is discovered) During shrinking commands are dropped one by one. This
function is used determine whether later commands are no longer valid if a prior
command has been removed.
The hardest part about learning how to use fugato is learning how to write test-check generators, so we've provided a motivating example and tutorial.
Currently fugato is only taking bug reports. If you find a bug or would like to see an enhancement that aligns with the following design principles please file a Github issue!
- Symbolic generation only. Bring your own runner!
Copyright (c) Vouch, Inc. All rights reserved. The use and
distribution terms for this software are covered by the Eclipse
Public License 2.0 (https://opensource.org/license/epl-2-0)
which can be found in the file epl-v20.html at the root of this
distribution. By using this software in any fashion, you are
agreeing to be bound by the terms of this license. You must
not remove this notice, or any other, from this software.