aptly is a swiss army knife for Debian repository management: it allows you to mirror remote repositories, manage local package repositories, take snapshots, pull new versions of packages along with dependencies, publish as Debian repository. More info are on aptly.info and on github.

nginx is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP proxy server, originally written by Igor Sysoev. More info is on nginx.org.

1. Create docker aptly-data volume if it doesn't exist, otherwise use directory:

   docker volume create --name aptly-data

   Also you can use --driver option. By default it equals to local. More info is here.

   All of aptly's data (including GPG keyrings) is bind mounted outside of the container to preserve it if the container is removed or rebuilt.

2. If you want to customize image or build the container locally, check out this repository and build, otherwise skip this step and use prepared image from DockerHub:

   git clone https://github.com/coralhl/aptly-docker.git
   docker build docker-aptly --tag coralhl/aptly:latest

   If you decide build own I suggest you use docker-compose commands. It will build own image before use.

3. Then generate keypair. It won't regenerate that, if you already have keypair (if a keyring exist at aptly-data):

   docker run --rm --log-driver=none \
     --volume aptly-data:/opt/aptly \
     coralhl/aptly:latest \
     /opt/keys_gen.sh "First Last" "[email protected]" "Password"

   "First Last" "[email protected]" "Password" will be associated with the GPG apt signing key.

   It creates /opt/aptly/public dir and repo_signing.gpg, repo_signing.key files. If /opt/aptly/public doesn't exist it will raise 404 error of Nginx.

   Keep in the mind that the GPG passphrase using ONLY BY USERS (at the external container) for:

   1. Generating GPG keys (In the temporary container)
   2. Singing Aptly packages (using CLI tool or REST API)

   Keep your GPG passphrase separately from the GPG keypair (keyrings).

4. Run aptly and nginx

   docker run \
     --detach=true \
     --log-driver=syslog \
     --restart=always \
     --name="aptly" \
     --publish 80:80 \
     --volume aptly-data:/opt/aptly \
     coralhl/aptly:latest

   If it returned (usualy on macOS):

   docker: Error response from daemon: failed to initialize logging driver: Unix syslog delivery error.

   Probably you don't have some driver. Execute docker rm aptly and try again without --log-driver=syslog.

   Flag	Explanation
   --detach=true	Run the container in the background
   --log-driver=syslog	Send nginx logs to syslog on the Docker host (requires Docker 1.6 or higher), more info is here.
   --restart=always	Automatically start the container when the Docker daemon starts
   --name="aptly"	Name of the container
   --volume aptly-data:/opt/aptly	Path (if you want set path use absolute path) or volume's name that aptly will use to store his data : mapped path in the container
   --publish 80:80	Docker host port : mapped port in the container
   --network="some-nw"	Attach a docker container to some network
   --rm	Remove a docker container when it will be stopped

   Also if you need to upload .deb file more than 100MB you can specify the nginx parameter via docker environment -e NGINX_CLIENT_MAX_BODY_SIZE='100M'.

5. Next steps

   • You can manage docker container with docker or docker-compose. For example:

     docker start aptly
     docker restart aptly
     docker stop aptly
     docker rm aptly

   • Use docker volume to manage the created volume.

   • Configure your own debian-repository. See here.

     You can also use Aptly REST API at YOUR-HOST/api. You need to generate htpasswd file before.

     docker run --rm --log-driver=none \
       --volume aptly-data:/opt/aptly \
       coralhl/aptly:latest \
       /opt/gen_htpasswd.sh "admin" "passwd"

     After executing:

     curl -u admin:passwd https://YOUR-HOST/api/version

     Security: Please note using http is not safety (your password sends as plain text). Add mandatory SSL encryption for /api via proxy server fe nginx. See here.

   • Configure a mirror of some repo. See here.

   • Configure clients. See here.

If you want to build and run locally I suggest use docker-compose. But before create volume and generate keypair.

By default, Docker will map port 80 on the Docker host to port 80 within the container where nginx is configured to listen. You can change the external listening port to map to any port you like. Change the docker-compose file if you use him.

git clone https://github.com/coralhl/aptly-docker
cd docker-aptly/

# Build and run
# `--build` - if rebuild is requiring
# `-d` - run container in the background
# More info at: `docker-compose up --help`
docker-compose up -d --build

# Stop & remove (it doesn't remove created volumes)
docker-compose down

# Start / restart / stop container
docker-compose start
docker-compose restart
docker-compose stop

# If you want to test code with run_test.sh, use:
docker-compose --file docker-compose.test.yml build
docker-compose --file docker-compose.test.yml run sut

May be conflict if you already pull aptly from docker hub:

ERROR: for aptly Cannot create container for service aptly: Conflict. The container name "/aptly" is already in use by container "85de5904f6fc73c04f4f8e7d08a09a1a63c2ba28afb5ce45aa9578ebdefeadc7". You have to remove (or rename) that container to be able to reuse that name.

In this situation you need remove currently aptly container (or rename it):

docker rm 85de5904f6fc73c04f4f8e7d08a09a1a63c2ba28afb5ce45aa9578ebdefeadc7

1. Fetch the public GPG key from your aptly repository and add it to your trusted repositories

   curl -O https://YOUR_HOST_FOR_APTLY/repo_signing.key
   apt-key add repo_signing.key

2. Backup then replace /etc/apt/sources.list

   cp /etc/apt/sources.list /etc/apt/sources.list.bak
   echo "deb https://YOUR_HOST_FOR_APTLY/ ubuntu main" > /etc/apt/sources.list
   apt-get update

   ubuntu & main may be another. It's require from your repos on aptly.

1. Copy files to volume. First of all start docker container, because access to volume produced through him. Copy files to container (volume) directory, use docker cp:

   docker cp aptly:/opt/aptly/<SRC_PATH> <DEST_PATH>
   docker cp <SRC_PATH> aptly:/opt/aptly/<DEST_PATH>

2. Create and update debian-repo:

   # Attach container
   docker exec -it aptly /bin/bash

   # Create repository folder
   aptly repo create -comment="ROS packages for Raspbian Stretch" -component="main" -distribution="stretch" rpi-ros-kinetic
   
   # Add deb-packages to index from `/opt/aptly/ros-kinetic-*`
   aptly repo add rpi-ros-kinetic /opt/aptly/ros-kinetic-*
   
   # Publish updates
   aptly publish repo rpi-ros-kinetic rpi-ros-kinetic

   Add new packages:

   # Add deb-packages to index from `/opt/aptly/ros-kinetic/`
   aptly repo add rpi-ros-kinetic /opt/aptly/ros-kinetic/
   
   # Publish updates
   aptly publish update stretch rpi-ros-kinetic

   Read the official documentation for learn more about aptly.

1. Edit the mirrors.yml file and configure the repositories you want to mirror. For example, it has a mirror configured for https://packages.sury.org/php/:

   repositories:
   - name: php
       url: https://packages.sury.org/php/
       dist:
       - bookworm
       - bullseye
       components:
       - main
       architectures:
       - amd64
       - arm64

2. Enter to the container. How to attach? See here.

3. After that you need to setup the public key is associated with this repo to /opt/aptly/gpg/trustedkeys.gpg, for this use /opt/keys_imp.sh, otherwise you will catch the error:

   ERROR: unable to fetch mirror: verification of detached signature failed: exit status 2
   

In the process, you will be asked for a passphrase. See p.3. Then you will have to specify it in the docker environment variable GPG_PASSPHRASE.

4. Run /opt/update_mirror.sh. This script takes mirror settings from the mirrors.yml file. It will create a mirror of the repository or update it, take a snapshot and publish it.

If the script fails due to network disconnects etc, just re-run it.

The initial download of the repository may take quite some time depending on your bandwidth limits, it may be in your best interest to open a screen, tmux or byobu session before proceeding.

For host a mirror of Ubuntu's main repository, you'll need upwards of 80GB+ (x86_64 only) of free space as of Feb 2016, plan for growth. Raspbian takes 69GB+.

When the script completes, you should have a functional mirror that you can point a client to.

As example:

5. Run /opt/db_cleanup.sh, if you need to save free space and delete all unpublished snapshots.

6. In order to automate the process of updating mirrors and cleaning the aptly database you can use the ofelia container.

   [job-exec "aptly sync"]
   schedule = 0 0 0,12 * * *
   container = aptly-mirror
   command = /opt/update_mirror.sh
   no-overlap = true
   
   [job-exec "aptly db cleanup"]
   schedule = 0 0 1,13 * * *
   container = aptly-mirror
   command = /opt/db_cleanup.sh
   no-overlap = true

© 2024 coral xciii.ru
© 2018-2020 Artem Smirnov
© 2016 Bryan J. Hong
Licensed under the Apache License, Version 2.0