Rotate Backups

[jan-brinkmann]

Hey @jareware,

things addressing rotate backups got a little bit messy. That's why I open this issue to summarize what we currently have and to discuss what we want to have.

• Since v2.5.0, we are enabled to upload backups by means of scp to a remote host and to make use of POST_SCP_COMMAND in order to execute rotate-backups directly on that remote host. Here, we have to overhand the complete configuration, e.g., POST_SCP_COMMAND: rotate-backups --daily 7 --weekly 4 --monthly 12 --yearly always /backup-directory.
• We have Trigger rotate backups on remote host #42 where we, again, execute rotate-backups directly on the remote host where we upload our backup to by means of scp. Unlike in the first option, we have a default configuration that can be customized by environmental varibales ROTATE_DAILY and so on.
• We have Integrate rotate backups #36 where we are enabled to execute rotate-backups directly inside the container on the directoy /archive. A default configuration ist given by a .rotate-backups.ini file . A custom configuration file can be mounted.

Here are my thoughts about all these stuff:

• If we like to make use of rotate-backups without scp, we need an external solution (e.g., cron job outside of the container) or we Integrate rotate backups #36.
• All three option handle customizing configuration in some own way. Imho, it would be nice to overhand configuration in some common way. As POST_SCP_COMMAND takes some arbitrary command, we cannot improve anything here. But Trigger rotate backups on remote host #42 provides most likely the most easy way to customize the configuration. These way can also used in Integrate rotate backups #36. That would result in a much better user experience.
• Integrate rotate backups #36 increases the image size from round about 600MB to 900MB (on ARM64). Installing rotate-backups at runtime, if it has been activated, is not a nice idea because the package itself and some dependencies have to be downloaded everytime the container has been removed and restarted (e.g., by docker-compose down followed by docker-compose up -d).
• If we do not merge Integrate rotate backups #36, we definitely do not need Trigger rotate backups on remote host #42 because the functionality is already there.

As far as I see, the first question that has to be answered is if #36 can be merged. In order to reduce the image size, maybe we can draw on #27. In the end, maybe we end up by the same image size we currently have.

What do you think?

Regards,
Jan

[jareware]

Hi!

I do kinda like the "batteries included" attitude of just including a rotation utility in the image itself. There's a few caveats, though:

1. As you said, it bloats the image. Personally I wouldn't lose a lot of sleep over it, as most of the time you pull an image like this once, and then it sticks either on the host cache or some other cache (e.g. your cloud provider's) along the way. But I also know some people are quite put off by huge images.
2. You still can include any such tool by using this image as a base in your own Dockerfile. But we don't need to make that choice of tool (and bloat) for users.
3. Even if we include such an utility into the image, it still won't work for scp targets (right?), only local targets.

Given all of the above, I quite like the (PRE|POST)_(|SCP_)COMMAND approach. It's slightly less "batteries included", but by far the most flexible.

[jan-brinkmann]

Hy @jareware,

1. Sounds good. This is also my thinking.
2. Yes, I can integrate it in my fork of the project. I have to make two things clear: First is that I do not prefer to release my fork of the project. Second is that I did not find out if you are open to integrate it into your project (the root project so to say). This is a minor communication issue. Please give me a unequivocally "Yes, please work in this.!" or a "No, I do not want this here." :)
3. If we integrate rotate-backups directly into the image and if we upload a backup by means of scp, I do believe that we could mount the target directory on the remote host by means of sshfs and execute rotate-backups there. Not 100% sure, but I think it is worth a try.

Can you please provide a little bit more information on your idea of PRE_COMMAND and POST_COMMAND?
I understood that these commands are executed before and after the backup has been copied to /archive. To execute rotate-backups (like POST_COMMAND: rotate-backups --daily 7 /archive), it has to be integrated in the image. Is that something you that covers your idea?

Regards,
Jan

[jareware]

I did not find out if you are open to integrate it into your project

Sorry, maybe I didn't quite catch that.

I would say that "no", I don't think we should integrate rotate-backups to the main project. The added size and complexity doesn't seem worth it right now. We definitely want to support backup rotation, but for now, I would just add options that allow calling an external script/container do to that. People can then use something simple (like rm) or fancy (like rotate-backups) as they please.

Can you please provide a little bit more information on your idea

Yes, I would hope it worked exactly as (PRE|POST)_SCP_COMMAND, except the command would be executed locally (i.e. within the backup container) instead.

If you do something simple (like rm), you don't need to do anything extra, just come up with a glob pattern or age restriction that works the way you like.

If you want something fancy (like rotate-backups), and you've mounted the Docker socket, you can docker run any image/command that does it for you. For a really crappy example:

docker run --rm -v /your/backup/location:/archive python:3-alpine bash -c 'pip install rotate-backups && rotate-backups ...'

But of course this could be packaged into a dedicated image:

docker run --rm -v /your/backup/location:/archive jan-brinkmann/rotate-backups

Assuming your SCP_HOST also has Docker available, you can use the exact same command over there.

What do you think?

[jareware]

Oh, and looks like there's also some solutions out there already, e.g. https://github.com/Glideh/docker-cron-rotate-backups

[jan-brinkmann]

Hi,

thank you for point out that we are able to start arbitrary Docker containers.

Indeed, this is a possibility to run rotate-backups on /archive without integrating it into the backup image.
I already have created a merge request for PRE_COMMAND and POST_COMMAND and a simple example where all backups older than seven days are deleted.

The docker-cron-rotate-backups you mentioned is imho no suitable solution for POST_COMMAND because is creates a cron job. What we need is a container that executes rotate-backups once and immediatelly. I have played a little bit around here: https://github.com/jan-brinkmann/docker-rotate-backups. I think I will add an example soon.

Regards,
Jan

[jareware]

#46 was exactly what I had in mind! 🙇

Also https://github.com/jan-brinkmann/docker-rotate-backups looks really neat. Let's add an example to the README in this repo on how to use that?

[jan-brinkmann]

Thank you for merging!

I will add an example the next days.

[jareware]

Allright, let me know and let's cut a release after!

[jan-brinkmann]

One more point related to this came to my mind:

(PRE|POST)_COMMAND are not really telling names revealing what will be done.
Can we rename them to e.g. (PRE|POST)_ARCHIVE_COMMAND? I.e., "commands that are executed before and after the backup has been moved to /archive".

I will give an update soon. :)

[jan-brinkmann]

Recently, I found out that rotate-backups can be applied on remote directories. I have implemented that feature in the external container here: https://github.com/jan-brinkmann/docker-rotate-backups#remote-directories

We can use this feature when we upload backups by means of SCP. If we would do this, rotate-backups does not have to be installed on the remote host where you transfer the backup to. Currently, rotate-backups must be installed. Thus, we can get rid of this restriction.

In order to apply rotate-backups with the external container docker-rotate-backups, we need a command that is executed in the docker-volume-backup container. Currently, we have POST_SCP_COMMAND which is executed on the remote host, and we have POST_COMMAND which is executed only if the backup is transferred to /archive.

I propose to move POST_COMMAND below the block that ends here:

docker-volume-backup/src/backup.sh

Line 148 in 612e633

fi

Does this make sense to you?

[jareware]

Oh yeah, makes total sense; being able to run pre/post commands shouldn't depend on whether you're backing up locally or remote, they can be useful for both. 👍