Skip to content

Latest commit

 

History

History
203 lines (147 loc) · 8.08 KB

bundle.md

File metadata and controls

203 lines (147 loc) · 8.08 KB
Raw
tagline
single-executable app deployment

What is

Bundle is a small framework for bundling together LuaJIT, Lua modules, Lua/C modules, DynASM/Lua modules, C libraries, and other static assets (and even directory listings) into a single fat executable. In its default configuration, it assumes luapower's [toolchain][building] and [directory layout][get-involved] (meaning that you have to place your own code in the luapower directory) and it works on Windows, Linux and OSX, x86 and x64.

Usage

 Compile and link together LuaJIT, Lua modules, Lua/C modules, C libraries,
 and other static assets into a single fat executable.

 Tested with MinGW, GCC and Clang on Windows, Linux and OSX respectively.
 Written by Cosmin Apreutesei. Public Domain.

 USAGE: mgit bundle options...

  -o  --output FILE                  Output executable (required)

  -m  --modules "FILE1 ..."|--all|-- Lua (or other) modules to bundle [1]
  -a  --alibs "LIB1 ..."|--all|--    Static libs to bundle            [2]
  -d  --dlibs "LIB1 ..."|--          Dynamic libs to link against     [3]
  -f  --frameworks "FRM1 ..."        Frameworks to link against (OSX) [4]
  -b  --bin-modules "FILE1 ..."      Files to force-bundle as binary blobs
  -b  --bin-modules "FILE1 ..."      Files to force-bundle as binary blobs
  -D  --dir-modules "DIR1 ..."       Directory listings to bundle as blobs

  -M  --main MODULE                  Module to run on start-up

  -m32                               Compile for 32bit (OSX)
  -z  --compress                     Compress the executable (needs UPX)
  -w  --no-console                   Hide console (Windows)
  -w  --no-console                   Make app bundle (OSX)
  -i  --icon FILE.ico                Set icon (Windows)
  -i  --icon FILE.png                Set icon (OSX; requires -w)
  -vi --versioninfo "Name=Val;..."   Set VERSIONINFO fields (Windows)
  -av --appversion VERSION|auto      Set bundle.appversion to VERSION
  -ar --apprepo REPO                 Git repo for -av auto

  -ll --list-lua-modules             List Lua modules
  -la --list-alibs                   List static libs (.a files)

  -C  --clean                        Ignore the object cache

  -v  --verbose                      Be verbose
  -h  --help                         Show this screen

 Passing -- clears the list of args for that option, including implicit args.

 [1] .lua, .c and .dasl are compiled, other files are added as blobs.

 [2] implicit static libs:           luajit
 [3] implicit dynamic libs:
 [4] implicit frameworks:            ApplicationServices

Examples

# full bundle: all Lua modules plus all static libraries
mgit bundle -a --all -m --all -M main -o fat.exe

# minimal bundle: two Lua modules, one static lib, one blob
mgit bundle -a sha2 -m 'sha2 media/bmp/bg.bmp' -M main -o lean.exe

# luajit frontend with built-in luasocket support, no main module
mgit bundle -a 'socket_core mime_core' -m 'socket mime ltn12 socket/*.lua' -o luajit.exe

# run the unit tests
mgit bundle-test

NOTE: Pass -m --all before any explicit -m arguments!
NOTE: Pass -a --all before any explicit -a arguments!

TIP: Pass -vi "FileDescription=..." to set the process description that is shown in the Windows task manager.

How it works

The core of it is a slightly modifed LuaJIT frontend which adds two additional loaders at the end of the package.loaders table, enabling require() to load modules embedded in the executable when they are not found externally. ffi.load() is also modified to return ffi.C if the requested library is not found, allowing embedded C symbols to be used instead. Assets can be loaded with bundle.load(filename) (see below), subject to the same policy: load the embedded asset if the corresponding file is not present in the filesystem.

This allows mixed deployments where some modules and assets are bundled inside the exe and some are left outside, with no changes to the code and no rebundling needed. External modules always take precedence over embedded ones, allowing partial upgrades to the original executable without the need for a rebuild. Finally, one of the modules (embedded or not) can be specified to run instead of the usual REPL, effectively enabling single-executable app deployment for pure Lua apps with no glue C code needed.

Components

.mgit/bundle.sh

The bundler script: compiles and links modules to create a fat executable.

The reason the script is hidden inside the .mgit dir is to allow you to use the same command mgit bundle on all platforms. In particular, mgit will drive the script using Git bash on Windows, if git is in your PATH. You can run the script directly without mgit of course but always run it from the root directory like this: .mgit/bundle.sh or move it there.

csrc/bundle/luajit.c

The standard LuaJIT frontend, slightly modified to run stuff from bundle.c.

csrc/bundle/bundle.c

The bundle loader (C part):

  • installs require() loaders on startup for loading embedded Lua and C modules
  • fills _G.arg with the command-line args
  • sets _G.arg[-1] to the name of the main script (-M option)
  • calls require'bundle_loader', which means bundle_loader itself can be upgraded without a rebuild.

bundle_loader.lua

The bundle loader (Lua part):

  • sets package.path and package.cpath to only load modules relative to the exe's dir and not look into the current directory.
  • overrides ffi.load to return ffi.C when a library is not found.
  • loads the main module, if any, per arg[-1]
  • falls back to LuaJIT REPL if there's no main module.

One subtle point about ffi.load is that if a library was found in a system path but that library is also bundled, the bundled one will be used instead. So the search order is:

  1. external shared library in the directory of the executable.
  2. bundled library inside the executable.
  3. external shared library in the system's search path.

bundle.lua

Optional module with an API for loading embedded binary files:

bundle.canopen(file) -> t|f check if a file exists and can be opened bundle.load(filename) -> string load a file bundle.mmap(filename) -> mmap memory-map a file mmap.data pointer to file data mmap.size file size mmap:close() close the mmap object bundle.fs_open(filename) -> f open a file with [fs] bundle.fs_dir(dirname) -> d open a dir with [fs] bundle.appversion -> string app version from the -av cmdline option

NOTE: These functions look in the filesystem first and only if that fails they use the embedded blobs.

Search paths

External files are looked for relative to the executable directory, regardless of the current directory, as follows:

  • shared library dependencies (either link-time or ffi.load-time) are searched for in $exedir
  • Lua modules are searched for in $exedir
  • Lua/C modules are searched for in $exedir/clib
  • static assets are searched for in $exedir

A note on compression

Compressed executables cannot be mmapped, so they have to stay in RAM fully and always. If the bundled assets are large and compressible, better results can be acheived by compressing them individually or not compressing them at all, instead of compressing the entire exe. Compression also adds up to the exe's loading time.

LGPL compliance

Some libraries in luapower are LGPL (check the package table on the homepage to see which). LGPL does not normally allow static linking on closed-source projects, but because a bundled executable will always load the dynamic version of a bundled library if one is found in the directory of the exe, this behavior complies with the requirement of LGPL to provide a way for the end-user to use the app with a different version of the LGPL library without requiring you to ship the dynamic version of the library along with the executable.