Skip to content

Out of Tree Development

Benoît Thébaudeau edited this page Sep 23, 2019 · 7 revisions

Out of tree development with Git Submodules

The best way to get started with Contiki is to read and modify the examples in the contiki/examples. However, once you start making your own project "from scratch", developing your own code out-of-tree in your own git repository dedicated to the project becomes desirable.

The following guide briefly explains how this is done using Git Submodules.

Including Contiki as a Submodule

Let's say we are working on a new project foo which will be based on the hello-world example, but we will maintain it outside of the Contiki source tree.

We will start by making a git repository named foo:

cd /tmp
mkdir foo
cd foo
git init 

results in:

Initialized empty Git repository in /tmp/foo/.git/

To add mainline contiki as submodule to your repository you do:

git submodule add https://github.com/contiki-os/contiki.git

This will clone mainline Contiki into a contiki directory and checkout the current master.

Cloning into 'contiki'...
remote: Counting objects: 66414, done.
remote: Compressing objects: 100% (12986/12986), done.
remote: Total 66414 (delta 48050), reused 66177 (delta 47892)
Receiving objects: 100% (66414/66414), 50.86 MiB | 275 KiB/s, done.
Resolving deltas: 100% (48050/48050), done.

A submodule is used to include one git repository into another (the parent). The parent repository stores a reference to the submodule which can be used by git submodule update to clone the included repository.

Let's take a look at what's going on in detail to see how this works:

git status

gives:

# On branch master
#
# Initial commit
#
# Changes to be committed:
#   (use "git rm --cached <file>..." to unstage)
#
#       new file:   .gitmodules
#       new file:   contiki
#

Adding the submodule added a new directory contiki as expected, but also a new file .gitmodules. This is a text file that keeps track of where the submodules are coming from and what they are named locally.

Let's dig deeper:

git diff --cached

returns:

diff --git a/.gitmodules b/.gitmodules
new file mode 100644
index 0000000..2d97e83
--- /dev/null
+++ b/.gitmodules
@@ -0,0 +1,3 @@
+[submodule "contiki"]
+       path = contiki
+       url = https://github.com/contiki-os/contiki.git

As you can see, the format of .gitmodules is rather simple. The submodule named "contiki" comes from https://github.com/contiki-os/contiki.git, and has the local path contiki

diff --git a/contiki b/contiki
new file mode 160000
index 0000000..d746786
--- /dev/null
+++ b/contiki
@@ -0,0 +1 @@
+Subproject commit d746786709adb351c46bbadfac36265009db3169

The above makes it clear how the submodule "referencing" works. The parent directory tracks that commit "d746786709adb351c46bbadfac36265009db3169" should be used. This reference is committed into the parent repository.

Submodules are git repositories that do not have any knowledge that they have a parent module or otherwise. That means if you cd into the contiki directory it will look, feel, and work just as if you did a regular clone. git submodule add just adds a reference in the parent repository to a particular commit of the submodule repository.

The submodule references are versioned by the parent repository. To illustrate this, let's say this foo project is a little old and I haven't kept up with the latest Contiki. But I know Contiki 2.6 very well. So I want to start there and develop for a while before I upgrade to the bleeding edge development in master. Submodules makes this no problem at all.

To do this:

cd contiki
git checkout 2.6

which produces:

HEAD is now at a3e5637... Bumped version number

now lets see what happend:

cd ..
git status

gives:

# On branch master
#
# Initial commit
#
# Changes to be committed:
#   (use "git rm --cached <file>..." to unstage)
#
#       new file:   .gitmodules
#       new file:   contiki
#
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working
directory)
#
#       modified:   contiki (new commits)
#

We see that contiki is modified with "new commits". That is the unstaged change of changing to tag 2.6 instead of master. To see this do:

git diff

which shows:

diff --git a/contiki b/contiki
index d746786..a3e5637 160000
--- a/contiki
+++ b/contiki
@@ -1 +1 @@
-Subproject commit d746786709adb351c46bbadfac36265009db3169
+Subproject commit a3e56371a59e27f49464b41cdebc30c5e0ef00c1

where hashref d746786 is the master at the time of this writing and the inital reference point for the submodule. a3e56371 is the hashref for 2.6.

We like all these changes so let's commit all of them with:

git commit -a -m "adding contiki 2.6 as a submodule"

Great! Now we can get on with our top secret foo application

As mentioned before, foo is really just hello-world, so let's copy that over:

cp contiki/examples/hello-world/* .
ls

you will see:

contiki  hello-world.c  hello-world-example.csc  Makefile  README

let's clean up:

rm hello-world-example.csc README
mv hello-world.c foo.c

(your projects should always have READMEs. But this is a top-secret project, so removing the README is recommended.)

now you must point your Makefile to where Contiki is located. Edit the Makefile in your favorite way and change:

CONTIKI_PROJECT = hello-world
CONTIKI = ../..

to:

CONTIKI_PROJECT = foo
CONTIKI = contiki

perfect. Where are we? run:

git status

which shows:

# On branch master
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#       Makefile
#       foo.c

We like what we've done so add and commit:

git add Makefile foo.c
git commit -m "our foo project is complete"

Bumping the Submodule

Ok, so we've done all that submodule work but what good was it? The main advantage is that now mainline and your code are decoupled. Mainline will move ahead (we hope) and you will hack and hack and hack. The time will come when you want to upgrade. Had you hacked your own fork directly, bumping the contiki version would be a big pain. So how do you do that here?

cd contiki
git checkout master
cd ..

done.

Your code might still be broken, but this way your versioning and Contiki's versioning aren't mixed up. There is also nothing stoping you from forking Contiki into your own repo and using that as the submodule. In fact, this is a very good way to develop projects that require your own changes to the core of Contiki.

At this time, you would test your code and if happy, commit the fact that you want to use this version of contiki instead of the old version.

git add contiki
git commit -m "use the latest contiki HEAD"

Out of Tree Targets

It is also possible to compile against TARGETs that are not included in mainline. This is great if you are developing your own hardware and have prototypes or varients. It's another way to develop independently from what mainline is doing but still keep most of code.

Going back to our secret project foo. Lets say we are going to run foo on a new extra-secret TARGET hardware: bar. Lets make our own targets directory:

cd /tmp/foo
mkdir targets

The bar target is a special form of econotag. So let's copy that over.

cp -R contiki/platform/econotag/ targets/bar
mv targets/bar/Makefile.econotag targets/bar/Makefile.bar

Then edit the Makefile to add your new targets directory to the search path for targets. Add the following line"

TARGETDIRS += targets

Now, you should be able to build your project against the super-secret bar target:

make TARGET=bar

Out of tree APPS

You can also write and develop your own Contiki APPS out of tree as well.

Let's say your CTO walks into your office and demands that you develop the a new APP to be the flagship product for your company. The CTO says the APP should be --- the baz. You ask the CTO what the baz should do, but he just rambled some vauge things that don't make much sense. You shrug and decide to copy the ping6 app. It seems close enough. The CTO did mention that the baz should "leverage IPv6 full-duplex communication"...

let's copy the app:

mkdir apps
cp -R contiki/apps/ping6 apps/baz

and now edit your Makefile. This edit is more esoteric than adding out-of-tree TARGETs... I apologize.

APPDIRS += ${addprefix ../apps/, $(APPS)}

now you can include the baz APP just like you would any other app.

Clone this wiki locally