xit ("xinu" + "test") is a C-based unit testing library for Xinu, an embedded operating system developed at Purdue University. xit provides the following functionality:
- Organization of tests into suites and test cases.
- Assertions for different data types, with useful, customizable, error reporting.
- System function mocking through the fff library.
- Cleanup and teardown functions per-test.
- Time measurement. xit will keep track of how long tests run, and kill tests that take too long.
- Automatic discovery, compilation, and declaration of tests.
- Integration into the Xinu compilation infrastructure. (
make test
is all you need!)
Features slated for development:
- Test preconditions. This enables a way of determining if certain tests should run. For CS 636, the class I wrote xit for, I'll be able to use preconditions to exclude some non-router tests from router hosts, and vice versa.
- Network-based distributed testing. Using infrastructure of the xinu lab, xit will be able to spin up multiple Xinu backends and run a coordinated sequence of tests across them. This should be useful for integration testing, where behavior of the entire networking stack is observed.
See examples/
for an example suite that tests
the behavior of the gettime
system function.
- Clone this repository into your Xinu source tree. (I use a directory
test/
, but anywhere is sufficient.) Use a submodule to keep track of your xit version within your Xinu repo:
git submodule add https://github.com/elliottwilliams/cs636_test.git test
- Include xit's makefile extension into Xinu's
Makefile
. Add this to the bottom ofcompile/Makefile
(substitute the directory xit is cloned into):
-include ../test/xinu/compile.mk
- Edit your
main.c
code to start a test runner on startup, like this:
#include <xinu.h>
// Add these lines near the top of the file:
#ifdef TESTS_ENABLED
#include <test/test.h>
#endif
// ...
process main(void)
{
// Add these lines to main():
#ifdef TESTS_ENABLED
resume(create(local_test_runner, INITSTK, INITPRIO, "local_test_runner",
0));
return OK;
#endif
// ...
}
- Create a
tests/
directory in the root of your source tree, and write tests in it. See the section on writing tests below.
-
Compile Xinu with tests enabled by running
make test
from thecompile/
directory. This will discover test functions intests/
, compile them, and link to the rest of the xinu codebase. Mock functions declared as wraps will be discovered automatically by the linker. -
To run only a subset of your tests, run
make
with aTESTS
variable containing space-separated paths to match. Paths are matched starting at the roottests/
directory. For example, given atests
directory:tests ├── config │ └── fakes.def.h ├── integration │ └── net │ └── nd_rsol_events.c └── unit ├── net │ ├── colhex2ip.c │ ├── common.c │ ├── common.h │ ├── ... │ └── nd_rsol.c └── system └── defer.c
Running
make test TESTS=unit/system
builds Xinu with the one test intests/unit/system
.
Runningmake test TESTS=unit
builds Xinu with all tests intests/unit
.
Runningmake test TESTS=unit/net/nd_rsol
builds Xinu with justtests/unit/net/nd_rsol.c
The resulting xinu.xbin
image can be used to boot up a Xinu backend as
usual, and will run all tests specified.
Test suites are C source files that import the following headers:
#include <test/test.h>
for test macros and data structures#include <test/assert.h>
for the assertion library#include <test/fake.h>
for access to defined mock functions
Each suite should contain one or more test cases, declared with the TEST
macro. For example:
TEST(sky_is_blue) {
// Set up any values to test
char * color = getskycolor();
// Perform assertions
assert_str_eq(color, "blue");
}
Inside the expansion of the TEST
macro, a test_t
structure is defined
corresponding to the test case's name, and the test case's function signature
is inserted.
The assert_*
macros check their arguments, and return a failure result to the
test runner if the assertion failed. By default, failures are printed out to
the xinu console. A list of all available assertions is provided
below.
The second parameter of the TEST
macro specifies options, passed in
structure initializer syntax.
The currently supported options are:
before
TEST(test_name, .before = before_fn) { /* ... */ }
Runbefore_fn
in the same process, immediately before the test procedure.after
TEST(test_name, .after = after_fn) { /* ... */ }
Runafter_fn
in the same process, immediately after the test procedure.
The test/assert.h
header defines the following assertion macros:
assert(cond)
Passes ifcond != 0
.assert_eq(lhs, rhs)
Passes iflhs == rhs
.assert_eq_fmt(lhs, rhs, fmt)
Passes iflhs == rhs
. Uses format stringfmt
to generate a failure message.assert_str_eq(lhs, rhs)
Passes iflhs
andrhs
are determined equivalent bystrcmp
.assert_in_range(expected, actual, tolerance)
Passes ifexpected
=actual
±tolerance
.assert_mem_eq(expected, actual, size)
Passes if the firstsize
bytes ofexpected
andactual
are equivalent.- The failure message for this assertion depends on hex dump functions I've not implemented yet.
xit allows functions called within the OS to be monitored and overridden at
runtime. To see these "fake functions" in action, look at
examples/test_gettime.c
, and see their listing in config/fakes.def.h
.
xit uses fff, a C microframework for creating "fake functions", which record information about calls made to them. It combines this library with ld(8) wraps, which rewrite all references for a symbol fn to __wrap_fn at link time.
xit mocks functions defined in config/fakes.def.h
. For example, for the line:
F(__wrap_icmp6in, __real_icmp6in, struct netpacket *, struct ip6packet *, struct icmp6msg *); \
xit will:
- Create a structure
__wrap_icmp6in_fake
, which records calls made to__wrap_icmp6in
. The schema of this structure is given in fff's documentation. - Create a function
__wrap_icmp6in
that updates the__wrap_icmp6in_fake
struct when called. - Proxy calls to
__wrap_icmp6in
to__real_icmp6in
. This allows fakes to be defined without affecting OS behavior. To proxy to a different function, overwrite__wrap_icmp6in_fake.custom_fake
in a test.- Due to ld(8) wrapping,
__real_NAME
references the original symbol wrapped. So ificmp6in
is wrapped, references in code toicmp6in
go to__wrap_icmp6in
, but__real_icmp6in
goes to the original symbol.
- Due to ld(8) wrapping,
Add the function and its signature to config/fakes.def.h
. See the
documentation and example in that file.
-
Ensure your test suite includes
<test/fake.h>
. -
In a test case, call a function which calls the function being faked. The fake function will be used automatically.
-
Inspect the call history of the fake function by looking at its
_fake
struct. For example, the if__wrap_ip6in
is faked, it's struct is__wrap_ip6in_fake
. See fff's documentation for details.