SkyForge is a high-level tool to create ready to boot debian root filesystems using a simple Dockerfile-inspired syntax.
It uses multistrap to create a basic system and it's own configuration file called Skyfile to create a rootfs tarball
SkyForge assigns a hash to each of the steps it executes. If you are developing your Skyfile and rerun the build only the actual commands that have changed will be re-run, thus saving you time and bandwidth. What's more skyforge captures the state of the rootfs at each step and makes sure that each command will be run exactly with the same rootfs as if the build had been started from scratch.
SkyForge uses OverlayFS UnionFS for fast snapshots (can be disabled with --no-overlay
command option). If you don't have OverlayFS UnionFS, you can instruct SkyForge to tarball each and every state of the rootfs being made. Result will be roughly the same, but snapshots will consume a lot of time and disk space. See
skyforge tarball
Just copy skyforge somewhere in your $PATH
(e.g. /usr/local/bin
) and execute it as root.
Since multistrap tool requires root privileges so does this tool. You are strongly adviced to use a disposable environment to run this tool, e.g. docker or a VM.
Since we are going to cook debian rootfs you should normally be running debian. Ubuntu and derivatives might/likely will work, but I haven't tested them myself.
Build rootfs from Skyfile in current directory.
skyforge build
If you don't have OverlayFS UnionFS building will not store snapshots on each step. If you don't have OverlayFS in your kernel UnionFS but still want the snapshot feature run
skyforge tarball
skyforge build
Enables tarballing of each of the steps run. This will persist until you run skyforge purge
. See build
section for more
Remove old snapshots from working directory (.forge
) and leave only those
actually present in the Skyfile
skyforge clean
Skyforge does cleaning every time it is executed, so you normally don't need to run it manually
Remove all intermediate junk created by Skyforge.
Equivalent of running rm -Rf .forge
(and removind artifacts).
When running with OverlayFS UnionFS available this command mounts the rootfs/
at the last snapshot and prints out history like status
does
Prints out current action log including Skyfile lines and their hashes. Each line of Skyfile is identified by a hash. Inserting a line before it changes the hashes of all lines following it.
Example
0 ✓ necromant @ sylwer ~/work/images-kitchen/skforge-ng $ sudo skyforge status
[S] 137e53ce7d4cdfdc450ebd8e4c26e0f5 | MULTISTRAP armel debian-devel.conf
[S] 1f9c1485f902a9d2666aac149964bd4f | INSTALL /usr/bin/qemu-arm-static
[S] 5d9b38f8739e1651cff272c8eb52967b | RUN DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true LC_ALL=C LANGUAGE=C LANG=C /var/lib/dpkg/info/dash.preinst install
[S] f65e324ccf1e930b92aa36d03d5d6bca | RUN DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true LC_ALL=C LANGUAGE=C LANG=C dpkg --configure -a
[S] 8fc57faaae70663788b66be85a350da9 | INSTALL /etc/resolv.conf /etc
[S] 7b71e6cc8c2e49f631f9359caf0b2ae5 | APT_KEY https://www.module.ru/mb7707/repo/repository.gpg
[S] c69c646d9f312a12921e351af0c46286 | PASSWD 12345678
[S] b99f635eac300deb0ac0cbd4f23b6b1b | HOSTNAME shade
[S] 20b00807b38463389ece47f5d44f268e | LOCALE en_US.UTF8 UTF8
[S] c89fd713906fad5f2a88dacce5071897 | IFACE_STATIC eth0 192.168.20.9 255.255.255.0 192.168.20.1 8.8.8.8
[S] 1cfbab080e8c49db1df2347eebc37edf | RUN sed "'s/PermitRootLogin without-password/PermitRootLogin yes/'" -i etc/ssh/sshd_config
[R] b1fabddf2afd249fc0a6cb6197ce3e6b | REMOVE /etc/ssh/ssh_host_*
[R] d1c3a6448f04d6656cfdf187a096ed06 | REMOVE /etc/resolv.conf
[R] e0b5c4cea6ae1dad709b751e13b10efd | STORE rootfs.tgz
Legend:
Skyforge will skip this step, it has already been run/we have a snapshot
Skyforge will execute this step when you run build
Reset filesystem to state before executing step with <hash>
. This step amd following will ne reruned on next build.
Example
skyforge reset b1fabddf2afd249fc0a6cb6197ce3e6b
Equal to run skyforge purge
and skyforge build
sequentially.
Runs multistrap with the supplied debarch (e.g. armel
) using the config file supplied. Should be the first command to run. This step also makes sure an adequate qemu-static binary is copied to target chroot. (e.g. qemu-arm-static
for arm
).
If part with RETRY_WITH
exist then on multistrap fail provided command will be executed and multistrap try to run again.
NOTE: Since debian architecture names are not always mapped one-to-one to qemu architecture names this script takes care to guess the right one. This logic currently covers arm
, mips
and x86/x86_64
variants. If you have something else - patches are welcome.
MULTISTRAP armel debian-armel.conf
Runs the steps required to configure the unpacked packages in chroot. This command does the following:
- Prohibits the start of any services via
policy-rc.d
2. Runs the dash.preinst script (As described in debian wiki) - Runs
dpkg --configure -a
in chroot - Allows service startup
The default behavior is interactive (e.g. allow dpkg-configure
to ask different
questions). If this is not desired, supply the silent argument.
If you want to other variables you can do it here as well.
Configure packages, ask no questions
DPKG_CONFIGURE noninteractive
or
DPKG_CONFIGURE DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true
Configure packages, ask questions about system setup
DPKG_CONFIGURE
or
DPKG_CONFIGURE interactive
Prohibits any services to be started in the target root filesystem via policy-rc.d
Undos what DENY_SERVICE_START has done
Include a Skyfile script.
INCLUDE Skyfile.inc
Copies a file from host operating system to target
INSTALL /etc/resolv.conf /etc
Runs a command in chrooted environment. If you have a cross-chroot and that doesn't work check if you have qemu-user-static
installed and that MULTISTRAP
command actually copied the right one to your chroot.
RUN uname -a
Delete a file from target root filesystem
REMOVE /etc/resolv.conf
Install a debian package to target rootfs the package can be either a local file or an URL
ADD_DEB https://example.com/myfile.deb
ADD_DEB myotherfile.deb
Call apt-get to install additional packages. Use of this command is not recommended. Edit multistrap.conf
instead.
ADD_PACKAGES usb-utils
Tarball the root filesystem into the archive filename.
The archive generated will be deleted if you run skyforge purge
STORE rootfs.tar.gz
Setup root password on the target system.
PASSWD 12345678
Setup hostname on the target root filesystem
HOSTNAME iltharia
Add locale to /etc/locale.gen
and generate locales on the target. Additionally set LANG and LANGUAGE to primary_locale
.
LOCALE en_US.UTF8 UTF8
Configure ethernet interface parameters (static IP)
IFACE_STATIC eth0 192.168.20.9 255.255.255.0 192.168.20.1 8.8.8.8
Configure ethernet interface with DHCP
IFACE_DHCP eth0
Add an apt gpg key into the target rootfs
APT_KEY https://www.module.ru/mb7707/repo/repository.gpg
Just run a shell command in this directory. You can omit SH and it will still work the same. Magic!
SH ls rootfs/
and
ls rootfs/
are essentially the same
Replace all symlinks in the root filesystem with file copies. This bloats the filesystem but is sometimes the only choice if you want to use the rootfs as a sysroot for development in a windows-environment that (sic!) doesn't play with symlinks well.
SYMLINK2COPY
Replace all symlinks containing absolute paths in the root filesystem with symlinks containing relative paths. This is required if you plan to use the resulting rootfs as a development sysroot (Because unpacking this filesystem anywhere save for / will corrupt relative symlinks)
SYMLINK2RELATIVE
Concatenate contents of rootfs/etc/ld.so.conf.d into one ld.so.conf
This is usually required if you plan to use the rootfs as a sysroot with linaro abe-based crooss toolchains that don't read properly the contents of /etc/ld.so.conf.d/
If you plan to actually boot the system - you don't need this one.
Unpack the filename.tgz into the rootfs/<dir>
directory.
ADD stuff.tgz /
ADD cross-compiler.tgz /opt
Mark filename as a build artifact. The only effect this has - this file will be removed when you run skyforge purge
Store the rootfs into an ext4 image. The resulting image is NOT an SD card image.
You'll need to create a partition table and dd
the resulting image to a partition
Same as the above, but with ext3 filesystem
Same as the above, but with ext2 filesystem
#The basic multistrap stuff
MULTISTRAP armel debian-devel.conf
INSTALL /usr/bin/qemu-arm-static /usr/bin
RUN DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true LC_ALL=C LANGUAGE=C LANG=C /var/lib/dpkg/info/dash.preinst install
RUN DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true LC_ALL=C LANGUAGE=C LANG=C dpkg --configure -a
#Install a proper resolv.conf
INSTALL /etc/resolv.conf /etc
#Install RCM Repository GPG Key
APT_KEY https://www.module.ru/mb7707/repo/repository.gpg
#Basic system setup
PASSWD 12345678
HOSTNAME shade
LOCALE en_US.UTF8 UTF8
IFACE_STATIC eth0 192.168.20.9 255.255.255.0 192.168.20.1 8.8.8.8
#IFACE_DHCP eth0
#Enable root access over ssh
RUN sed "'s/PermitRootLogin without-password/PermitRootLogin yes/'" -i /etc/ssh/sshd_config
#Force SSH to generate host keys at boot
REMOVE /etc/ssh/ssh_host_*
REMOVE /etc/resolv.conf
STORE rootfs.tgz
See more examples in example/
directory
See LICENSE.TXT