The idea behind piku
is that it provides the simplest possible way to deploy web apps or services. Simplicity comes at the expense of features, of course, and this document tries to capture the trade-offs.
Using uWSGI in emperor mode gives us the following features for free:
- Painless Python WSGI and
virtualenv
integration - Process monitoring, restarting, basic resource limiting, etc.
- Basic security scaffolding, beginning with the ability to define
uid
/gid
on a per-app basis (if necessary)
An app is simply a git
repository with some additional files on the top level, the most important of which is the Procfile
.
piku
recognizes five kinds of process declarations in the Procfile
:
wsgi
workers, in the formatdotted.module:entry_point
(Python-only)web
workers, which can be anything that honors thePORT
environment variablestatic
workers, which simply mount the first argument as the root static pathcron
workers, which require a simplifiedcron
expression preceding the command to be run (e.g.cron: */5 * * * * python batch.py
to run a batch every 5 minutes)worker
processes, which are standalone workers and can have arbitrary names
So a Python application could have a Procfile
like such:
wsgi: module.submodule:app
worker: python long_running_script.py
fetcher: python fetcher.py
# Simple cron expression: minute [0-59], hour [0-23], day [0-31], month [1-12], weekday [1-7] (starting Monday, no ranges allowed on any field)
cron: 0 0 * * * python midnight_cleanup.py
...whereas a generic app would be:
web: embedded_server --port $PORT
worker: background_worker
Any worker will be automatically respawned upon failure (uWSGI will automatically shun/throttle crashy workers).
Since piku
is targeted at 12 Factor apps, it allows you to set environment variables in a number of ways, the simplest of which is by adding an ENV
file to your repository:
SETTING1=foo
# piku supports comments and variable expansion
SETTING2=${SETTING1}/bar
# if this isn't defined, piku will assign a random TCP port
PORT=9080
See ENV.md for a full list of Piku variables which can also be set.
Environment variables can be changed after deployment using config:set
.
piku
follows a very simple set of rules to determine what kind of runtime is required:
- If there's a
requirements.txt
file at the top level, then the app is assumed to require Python. - TODO: Go
- TODO: Node
- TODO: Java
- For all the rest, a
Procfile
is required to determine application entry points.
Application isolation can be tackled at several levels, the most relevant of which being:
- OS/process isolation
- Runtime/library isolation
For 1.0, all applications run under the same uid
, under separate branches of the same filesystem, and without any resource limiting.
Ways to improve upon that (short of full containerisation) typically entail the use of a chroot
jail environment (which is available under most POSIX systems in one form or another) or Linux kernel namespaces - both of which are supported by uWSGI (which can also handle resource limiting to a degree).
As to runtime isolation, piku
only provides virtualenv
support until 1.0. Python apps can run under Python 2 or 3 depending on the setting of PYTHON_VERSION
, but will always use pre-installed interpreters (Go, Node and Java support will share these limitations in each major version).
piku
uses two git
repositories for each app: a bare repository for client push, and a clone for deployment (which is efficient in terms of storage since git
tries to use hardlinks on local clones whenever possible).
This separation makes it easier to cope with long/large deployments and restore apps to a pristine condition, since the app will only go live after the deployment clone is reset (via git checkout -f
).
This diagram (available as a dot
file in the img
folder) outlines how its components interact: