Author: Tobit Flatscher (August 2021 - February 2023)

This is guide explains how one can develop inside a Docker container while being able to run real-time capable code on a Linux operating system. As such it walks you through:

• Basics of real-time systems and an overview of different real-time Linux approaches
• Set-up of a real-time system, in particular the installation of PREEMPT_RT supplying simple scripts for automatically re-compiling the kernel
• Possible optimizations of a real-time system in order to minimise the latency and benchmarking the real-time performance by means of cyclictest
• Required settings for a real-time capable container with a PREEMPT_RT patched host system

This can be useful for several different applications, in particular:

• Controlling real-time robotics hardware from Linux systems, e.g. over EtherCAT (by using EtherCAT masters such as SOEM or IgH) or setting up containers for other robotic components such as the Franka Emika Panda collaborative robotic arm
• Numerical control by turning your desktop or single-board computer into a SoftPLC e.g. with CodeSYS Control or using LinuxCNC
• Real-time programming in audio development such as with Juce

but may also apply to software developers developing any other form of real-time capable code. This guide does not apply to Windows and Macintosh operating systems as it relies on a PREEMPT_RT-patched host system.

For an introduction into development with Docker and Docker-Compose, how to set it up in Visual Studio Code and a guide for using it with graphic user interfaces and ROS workspaces have a look at my other repository docker-for-ros.

There are different ways of turning a vanilla Linux system into a real-time capable operating system. As outlined in doc/RealTimeLinux.md PREEMPT_RT is likely the most future-proof possibility as it is about to be included into the mainline of Linux.

The set-up of a real-time capable Docker with PREEMPT_RT is quite straight forward. All you need is:

• A PREEMPT_RT-patched Linux host operating system
• An arbitrary Docker container launched with the correct options so that it can set real-time priorities from inside the container as well as options for reducing the network latency

The manual set-up of PREEMPT_RT takes quite a while (see doc/PreemptRt.md). You have two options, a custom re-compilation of the kernel or an installation from an existing Debian package.

The installation procedure either by compilation from source or from an existing Debian package is lined out in doc/PreemptRt.md. The same procedure can also be performed with the provided scripts src/install_debian_preemptrt and src/compile_kernel_preemptrt. src/install_debian_preemptrt checks online if there are already pre-compiled PREEMPT_RT packages available and lets you select a suiting version graphically, while src/compile_kernel_preemptrt compiles the kernel from scratch and installs it. Before using the scripts be sure to make them executable on your system with $ sudo chmod +x install_debian_preemptrt.

Start of by launching src/install_debian_preemptrt and follow the installation instructions

$ cd src
$ ./install_debian_preemptrt

Afterwards you can reboot your system (be sure to select the correct kernel!) and should already be ready to go. You can check the kernel version with $ uname -r to verify that you are using the correct kernel and the installation was indeed successful. Quite often this installation might fail due to missing dependencies. I am not completely sure what is the issue there but in case this does not work you will have to compile the kernel yourself as outlined in the following section.

If the installation above fails (or for some good reason you have to compile the kernel yourself) you can use the src/compile_kernel_preemptrt script.

You can launch it in two different ways:

$ cd src
$ ./compile_kernel_preemptrt

will install the required dependencies and then open a dialog which lets you browse the possible versions and available options manually, reducing the copy and paste operations.

If you supply a correct real-time patch version from the list of available ones as an input argument, launching the command with superuser privileges it will download all files, patch the kernel, create a Debian package if no official one is available and install it automatically.

$ cd src
$ sudo ./compile_kernel_preemptrt 5.10.78-rt55

This might be helpful for deploying a new kernel automatically on a remote system. The possible version numbers to be passed as arguments can be found at here.

After having patched your system and restarted it, booting into the freshly installed kernel, you should already be good to go to launch a real-time capable Docker. In case you do not intend to use root as the user inside the Docker you furthermore will have to have give yourself a name of a user that belongs to a group with real-time privileges on your host computer. How this can be done can be found in doc/PreemptRt.md.

After having successfully installed PREEMPT_RT, it is sufficient to execute the Docker with the options:

    cap_add:
      - SYS_NICE
    ulimits:
      rtprio: 99
      rttime: -1 # corresponds to 'unlimited'
      memlock: 8428281856
    network_mode: host

Then any process from inside the Docker can set real-time priorities rtprio (e.g. by calling ::pthread_setschedparam from inside the C/C++ code or by using chrt from the command line). Generally it is advisable to not set rtprio to 99 (refer to pull request #3 for more information).

This Github repository comes with a simple example that can be used to try it out. Inside the Docker container a cyclictest is run to assess the real-time performance of the system. You can compare the outcome to running it on your local system. There should be virtually no difference between the two, as can be seen from the graphs below:

For launching the cyclictest open the Docker by typing

$ docker-compose -f docker/docker-compose.yml up

then browse the folder benchmark/ and run the command

$ ./mklatencyplot.bash

This should create a latency histogram by measuring the difference between a thread's intended wake-up time and its actual wake up time (resulting in an image plot.png similar to the one above). This measures any form of latency caused by hardware, firmware and operating system. For more information on this test refer to OSADL.