fiotest provides a base that can be extended for users of a Foundries Factory to easily add testing.
Devices in a Factory communicate through a device gateway API. This API allows device's to upload test results for the current Target the device is running.
fiotest can be run as Compose Application on devices in your fleet. The application consists of 3 key pieces:
- Aktualizr-lite callback handler that controls when testing is performed
- A testing specification that defines what to test
- A "spec runner" that executes the testing specification
When the container is started, it will ensure that aktualizr-lite is configured to send callback messages to it. The container then listens for 2 callbacks:
-
install-post "OK" - This means a new Target has been installed. The testing specification should be executed.
-
install-pre - This means an install is about to be started. If testing is running, it will be stopped.
A testing specification allows a user to define how target testing should take place. Some users may want to do a single set of tests like:
# this runs the smoke test suite one time after a Target is installed
sequence:
- tests:
- name: smoke test
command:
- /usr/share/fio-tests/smoke.sh
Some users may want to do repeated "soak" testing of a device:
# run smoke test suite over and over taking a 10 minute break in between tests
sequence:
- tests:
- name: smoke test
command:
- /usr/share/fio-tests/smoke.sh
repeat:
delay_seconds: 600
A test can also be repeated a specific number of times:
# run smoke test suite 4 times with a 5 second delay between runs
sequence:
- tests:
- name: smoke test
command:
- /usr/share/fio-tests/smoke.sh
repeat:
delay_seconds: 5
total: 4
A test can even force the device to reboot with:
# Run the smoke suite, reboot, and run it again
sequence:
- tests:
- name: smoke test
command:
- /usr/share/fio-tests/smoke.sh
- reboot:
command: /sbin/reboot
- tests:
- name: smoke test
command:
- /usr/share/fio-tests/smoke.sh
A test can be run directly on the host with:
sequence:
- tests:
- name: dmesg
on_host: true
command:
- /bin/dmesg
- Decide on approach to testing. The fiotest container can do a lot including running tests on the host OS. However, creating a container based on hub.foundries.io/lmp/fiotest might be needed for customized tests.
- Create a custom test-spec.yml.
- Update docker-compose.yml to reference 1 and 2.
#!/bin/bash -e
set -o pipefail
TESTS="${TESTS-syscalls -s madvise}"
LTP_PATH="${LTP_PATH-/opt/ltp}"
LOGS="/tmp/LTP_$(date +%s)_"
[ -z $TEST_DIR ] && (echo "ERROR: TEST_DIR not defined"; exit 1)
[ -d $TEST_DIR ] || (echo "ERROR: no TEST_DIR"; exit 1)
read_ltp_results() {
grep -E "PASS|FAIL|CONF" "$1" \
| awk '{print $1" "$2}'
}
failed=0
run_ltp() {
cd "${LTP_PATH}"
./runltp -p -q -f $TESTS -l "${LOGS}.log" 2>&1 | tee ${LOGS}-output.log || true
while IFS= read -r line ; do
parts=($line)
result_dir="${TEST_DIR}/$(date +%s.%N)-${parts[0]}"
mkdir ${result_dir}
if [ "${parts[1]}" = "FAIL" ] ; then
touch ${result_dir}/failed
failed=1
fi
if [ "${parts[1]}" = "CONF" ] ; then
touch ${result_dir}/skipped
fi
done < <(read_ltp_results "${LOGS}.log")
}
run_ltp
exit $failed
# Build LTP
FROM ubuntu:20.04 as ltp
RUN apt update && \
apt install -y gcc git make pkgconf autoconf automake bison flex m4 libc6-dev wget
RUN wget -O /ltp.tar.xz https://github.com/linux-test-project/ltp/releases/download/20200515/ltp-full-20200515.tar.xz
RUN tar -xf /ltp.tar.xz
RUN cd ltp-full* && \
./configure && \
make -j8 all && \
make install
RUN cd /opt/ltp/testcases/bin && \
(strip `ls | grep -v .sh | grep -v .py` || true)
# Extend fiotest
FROM hub.foundries.io/lmp/fiotest:postmerge
COPY --from=ltp /opt/ltp /opt/ltp
COPY ./tests/ltp.sh /usr/share/fio-tests/ltp.sh
sequence:
tests:
...
- name: ltp
command:
- /usr/share/fio-tests/ltp.sh
...