[Feature Request] Support for backing up docker bind mounts

[jacobblock]

One of my primary backup use cases is stopping a container, backing up the bind mounted host folders, and starting the container again. It would be awesome if autorestic had this kind of functionality for locations.

[agateblue]

Maybe it could work in the form of a command to launch before/after the backup?

locations:
  docker-postgres:
    from: /srv/docker-postgres/data
    to: storage
    run_before: docker stop postgres
    run_after: docker start postgres

this way it isn't specific to docker, and works for other use cases?

[cupcakearmy]

This WE I'll be thinking about some potential ways to backup docker volumes. I assume you mean volumes that are not mapped to the local file system :)

Good suggestion!

[jacobblock]

@EliotBerriot your proposed hook looks great for CLI support; docker, kubectl, service, systemctl, etc. would have basic support with this.

Just as an FYI, referencing similar functionality in other provisioning apps can be helpful, e.g. https://github.com/ansible/ansible-modules-core/blob/devel/system/service.py to avoid common pitfalls with sleep/timeouts/status codes/etc.

@cupcakearmy I primarily use local file system mounts (or NFS, etc.) and my concern is stopping apps to keep data integrity. Of course first-class support for docker volumes (and/or other management APIs) would be fantastic. A basic shell hook would go a long way to supporting more cases.

[cupcakearmy]

It makes the tool way more flexible. I'll implement it as soon as I have time :)

[cupcakearmy]

https://github.com/cupcakearmy/autorestic#before--after-hooks

I still need to think about what the best way is to integrate docker volumes. Any suggestions are welcome :)

[cupcakearmy]

# Backup
docker run --rm --volume my_volume_to_backup:/data -v $(pwd):/backup alpine tar cf /backup/backup.tar -C /data .

# Restore
docker run --rm --volume my_volume_to_backup:/data -v $(pwd):/backup alpine tar xf /backup/backup.tar -C /data

This could be the way to go

[cupcakearmy]

So, I tried a few things

1

Mount the volume -> copy data to host -> run restic

This worked but it had the problem that docker will modify permissions and groups of the files -> no,no
(Tried with cp -aT to preserve permissions)

2

Then i tried to do the same thing bypassing the mounted volume permissions limitation with tar

Mount volume -> tar -> copy tar to host -> untar in host -> run restic

This unfortunately did work but again the permissions were lost. My guess is that if tar does not find the user when decompressing it will replace it. Not sure..

3

Mount volume -> tar -> copy tar to host -> run restic

This approach works perfectly but it does not allow incremental updates as restic just sees the tarbal. It's not the best thing, but at least it guarantees that permissions are preserved 100%

So I implemented version 3. Will be released soon :)

[cupcakearmy]

It's done 🚀 docker volumes can now be backed up and restored with out needing them to be mounted to the host.

@jacobblock @EliotBerriot

https://github.com/cupcakearmy/autorestic/blob/master/README.md#-docker-volumes

[FarisZR]

can i backup mounted volumes using autorestic in docker ?
for example, i have autorestic running in docker.
and the container is postgres running in /home/user/postgres.
i mount /home/user/postgres to autorestic, and then it stops the container temporarily while the backup is running.

is this possible ? or does it need autorestic running outside of docker to execute commands to stop/start the container ?

thanks for the awesome project @cupcakearmy !

[agateblue]

@FarisZR you can do that exactly with the before and after hooks:

locations:
  mydb-backup:
    from:  /home/user/postgres
    hooks:
      before:
        - sudo docker stop my-db
      after:
        - sudo docker start my-db

EDIT: a better option to avoid downtime might be not to stop the container at all and use pg_dumpall, e.g :

locations:
  mydb-backup:
    from:  /var/backups/postgres
    hooks:
      before:
        - mkdir -p /var/backups/postgres
        - sudo docker run my-db pg_dumpall > /var/backups/postgres/backup.sql
      after:
        - rm /var/backups/postgres/backup.sql

[FarisZR]

The commands wouldn't work if autorestic is also running inside docker.

…

-------- Original Message --------

On Jan. 17, 2022, 12:50 p.m., cupcakearmy/autorestic - reply+aipxadswhzassie3t7s4wsf76eoorevbnhhb7iivfu at reply.github.com wrote: ***@***.***(https://github.com/fariszr) you can do that exactly with the before and after hooks: locations : mydb-backup : from : /home/user/postgres hooks : before : - sudo docker stop my-db after : - sudo docker start my-db — Reply to this email directly, [view it on GitHub](#4 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AIPXADRYXBMRRTTROB5JRPDUWPQ6RANCNFSM4JTJCB3A). Triage notifications on the go with GitHub Mobile for [iOS](https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675) or [Android](https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub). You are receiving this because you were mentioned.Message ID: ***@***.***>