Enlil is a solution to ease and speed up the containerization of entire robotic infrastructures with ROS. Enlil does so by automatically generating all files required by Docker Compose in order to both containerize and simultaneously orchestrate multiple ROS workspaces. All the files generated by Enlil are ready-to-use.

It is named after the sumerian God of air and cloud.

• Enlil
  • Table of Contents
  • Features
  • Technologies
  • Installation
    • Manual Installation
    • Pip Installation
  • CLI Usage
  • Generation Pipeline
  • Template Variables
  • Syntax Documentation
    • Areas
    • Robots
    • Images
    • Packages
    • Globals
  • Project Structure
  • Articles
  • License

• Converts YAML files into complete and ready-to-use Dockerfile and Docker Compose files.
• Generates containerized ROS packages on-the-fly.
• Support for simultaneous usage of multiple ROS distributions and versions.
• Supports usage of private code bases.
• Full support of existing Docker Compose syntax.
• Completely open-source.

Enlil is completely written in Python 3. It depends solely on the following external modules:

• Docopt - to generate with ease a command line interface.
• Jinja2 - to generate all output files.
• Pip3 - package-management system for module installation.

Install the dependencies running the following command:

git clone https://gitlab.inesctec.pt/pedro.m.melo/enlil
cd enlil
pip3 install -r requirements.txt

In case of manual installtion it is recommended to create an alias for Enlil for ease of usage with:

echo 'alias enlil="python3 <PATH_TO_ENLIL>/enlil.py"' >> ~/.bashrc
. ~/.bashrc

You may now be able to run Enlil with:

enlil <ARGUMENTS>

You can install Enlil directly through pip:

pip3 install enlil

You may now be able to run Enlil with:

enlil <ARGUMENTS>

Enlil has a command line interface, that expects the following arguments:

If not specified, the CLI argument output_file creates a file docker_compose.yml on the same folder where Enlil is execited.

For more information about usage the following command can be executed:

enlil -h

Enlil must be provided an YAML configuration file (CLI argument configuration_file) as shown:

---
areas:
  - id: IILAB-fasten
    robots:
      - friday

robots:
  - id: friday
    ros: melodic
    images:
      - ros-development
    packages:
      - rosbridge
      - universal_robot
    vars:
      - UR_VERSION: 10
      - ROSBRIDGE_PORT: 9091

images:
  - id: ros-development
    image: ros
    stdin_open: true
    tty: true
    volumes:
      - ./ros_workspace:/ros_workspace

packages:
  - id: universal_robot
    path: ./packages/
    git:
      - https://github.com/ros-industrial/universal_robot:{{ROBOT_ROS_DISTRO}}-devel
    command: "roslaunch ur_gazebo ur{{UR_VERSION}}.launch gui:=false"

  - id: rosbridge
    path: ./packages/
    apt:
      - ros-{{ROBOT_ROS_DISTRO}}-rosbridge-server
      - ros-{{ROBOT_ROS_DISTRO}}-tf
    ports:
      - "{{ROSBRIDGE_PORT}}:{{ROSBRIDGE_PORT}}"
    command: "roslaunch rosbridge_server rosbridge_websocket.launch port:={{ROSBRIDGE_PORT}}"

globals:
  - id: webviz
    image: cruise/webviz
    ports:
      - "8081:8080"

Enlil divides input data into five distinct types of entities:

• Area: each area represents a physical space where multiple robots may work simultaneously.

• Robot: each robot represents a robotic agent running ROS and performing some action inside a given area.

• Image: each image represents an existing Docker image that is part of the ROS stack running inside the robot.

• Package: each package represents, in oposition to images, a ROS node that is still to be containerized. The required Dockerfile for such ROS nodes are generated alongside the final Docker Compose file.

• Global (Entity): each global consists of an existing Docker image that is relevant to the infrastructure and that may or may not interact with the ROS nodes.

The pipeline adopted by Enlil to generate the final Docker Compose file and Dockerfile is show in the next picture.

The pipeline is composed of the following stages:

• Parsing Stage (I): responsible for the parsing of the input file and initialization of typed entities.

• Decoding Stage (II): responsible for the decoding of template variables, as explained in detail in the next section.

• Converting Stage (III): responsible for transforming the decoded data into a more useful format for rendering.

• Rendering Stage (IV): responsible for generating the final Docker Compose file and all required Dockerfiles.

Enlil emphasizes reusage of code having an inner templating mechanism.

All textual information placed in between curly braces is interpreted by Enlil to be a template variable. Similar to variables in a programming language, template variables work as placeholders for useful information. They are a powerful tool to hide complexity allowing developers to describe more complex scenarios with less effort by reutilizing a same package or image declaration multiple times within multiple robots.

There are two types of template variables:

• the variables defined by developers at the configuration file level. These variables may also be used be developers.
• the variables set automatically by Enlil and that are used internally.

As shown in the next section, developers can define their own template variables developers by means of the field vars.

The decoding of template variables into real information is performed at the second stage of the presented pipeline. The decoding stage takes advantage of the hierarchy existent between the different types of entities. Starting with all entities of type area the proposed solution analyzes all data about the robots within a single area. Then, it individually analyzes the data of each image within each robot. The same is done for every package within the robot. This way, information can be passed from entities of type area to entities of type robot, and from these to entities of types image and package.

As a suggestion, developers may combine Enlil's template variables with Docker Compose's template variables ( enclosed in between ${...} ) to obtain an even higher degree of code reusage.

Enlil expects the input YAML configuration file to be divided into the following five sections: areas, robots, images, packages and globals. Each of these sections contains data about the respective typed entities and is also expected to follow a specific syntax, as explained in detail in this section. Enlil returns a proper error anytime a section does not comply with the said syntax.

Always keep in mind that the syntax adopted by Enlil is fully compliant with the syntax employed by Docker Compose. Enlil will only take into consideration the fields specified below. Therefore, any extra fields declared within sections images and packages will be passed to the final Docker Compose file. This feature allows you to to declare, for example, Docker volumes and Docker networks just as you would normally while using regular Docker tools. In spite of not being used directly by Enlil, all information declared within the said sections goes through the entire generation pipeline and may take advantages of the already explained template variables.

This section is mandatory for all input files!

Important considerations:

• Each area has a private network associated to it, to which all its robots are connected and may use to communicate.
• All areas are required to have at least one robot.

This section is mandatory for all input files!

Important considerations:

• All robots must be declared within an area or otherwise they will not be considered by Enlil.

• Robots from the same area can communicate with themselves (using their area's network) while robots from different areas can't communicate between themselves at all.

• At least one of the fields images and packages must be declared.

• Field ros follows the format <DISTRO:MASTER> where DISTRO is mandatory and corresponds to the name of the desired ROS distribution and MASTER is optional and corresponds the the networking port in which the ROS master node will be listening for communications. In case a ROS2 distribution is chosen then the value of MASTER corresponds to the desired value for environment variable ROS_DOMAIN_ID. If MASTER is ommited, then the default value of 11311 is chosen for ROS distributions and the default value of 42 is chosen for ROS2 distributions.

• Entries in vars follow the format <KEY:VALUE> where KEY is an unique identifier for the template variable within the robot and VALUE is the value for the variabel itself.

At the moment only ROS distributions melodic, noetic and foxy were tested.

Important considerations:

• Field image follows the format <IMAGE:TAG> where IMAGE is mandatory and corresponds to the name of the desired Docker image to use while TAG specifies the desired version of the specified Docker image. Contrarily to the Docker standard, where the default value for an image tag is latest, the default value used by Enlil always matches the ROS distribution of the robot where the entity is declared.

Important considerations:

• The generated Dockerfile can be used freely to build and deploy customized Docker images of ROS nodes.

• Whenever the fields git or rosinstall are declared the creation of the final Docker container is done in two stages, allowing for the secure usage of private code bases using existing SSH deploy keys. During the first stage the contents of the host folder ssh/ are copied into the container before the downloading of any declared repository. Only the downloaded repositories are then passed to the final stage, making access to the SSH deploy keys impossible through the final container or Docker image.

• At least one of the fields apt, git and rosinstall must be declared.

• The base Docker images used are ros:melodic, ros:noetic and ros:foxy, all of which come with the apt package manager already installed. Check https://hub.docker.com/_/ros for more information.

Important considerations:

• All declared globals are placed by Enlil in the final Docker Compose file as is and without any changes. Use this section to declare everything you would otherwise declare in a normal Docker Compose file.

• The default values for globals match the ones used by Docker and Docker Compose. For instance the default tag for an Docker image is again latest.

• Since they are passed unaltered to the final Docker Compose globals should not use template variables.

The source code for this project is rganized as follows:

• docs: contains all the information regarding documentation, both textual and images.

• examples: contains example YAML configuration files that serve as example for developers trying to learn how to use Enlil.

• pipeline: contains the source code for the presented file generation pipeline:

  • /decoder: contains the source code responsible for stages II and III of the presented pipeline.

  • /loader: contains the source code responsible for stages I of the presented pipeline.

    • /entities: contains the implementations for each type of entity.

  • /renderizer: contains the source code responsible for stages IV of the presented pipeline.

    • /templates: contains the templates used to generated the final Docker Compose file and necessary Dockerfiles.

• tests: contains the unit tests for the application.

• enlil.py: application entry point.

Enlil is featured at the INDIN conference (2021 edition).

More information about the submitted article will be placed soon.

Enlil is itself free, open-source and is deployed with MIT license.