avidamoeba

It isn't.

I've used Ubuntu on a laptop during my undergrad 2008-13. I used LyX to write anything I'd submit, including some psych work. I've used LibreOffice (OpenOffice) for some stuff too. I had to use MS Office or some other Windows-only software on occasion. I used a Windows VM for that. I've kept this formula till present day. Linux (Ubuntu LTS/Debian) on the hardware, Windows VM on Linux for special occasions.

Now this makes perfect sense.

It's rockets for space tourism.

I dump the db too.

With that said if backing up the raw files of a db while the service is stopped can produce a bad backup, I think we have bigger problems. That's because restoring the raw files and starting the service is functionally equivalent to just starting the service with its existing raw files. If that could cause a problem then the service can't be trusted to be stopped and restarted either. Am I wrong?

Used/off-lease ThinkPad T-series.

Oh yeah, that would be a disaster. If not handled correctly.

Contents of application memory lost, yes. Contents of unflushed OS buffers, no. Your db will be fsyncing (or moral equivalent thereof) if it's worth the name.

Good point. I guess kill -9 is somewhat less catastrophic than a power-yank. If a service is written well enough to handle the latter it should be able to handle the former. Should, subject to very interesting bugs that can hide in the difference.

This is an aside; backing up from a volume snapshot is half a reasonable idea. (The other half is ensuring that you can restore from the backup, regularly, automatically, and the third half is ensuring that your automated validation can be relied on.)

I'm currently thinking of setting up automatic restore of these backups on the off-site backup machine. That is the backups are transferred to the off-site machine, restored to the dirs of the services, then the services are started. This should cover the second half I think. Of course those services can't be used to store new data because they'll be regularly overwritten with every backup. In the event of a hard snafu where the main machine disappears, I could stop the auto restore on the off-site machine and start using the services from it, effectively making it the main machine. If this turns out to be reasonable and working, I might trash all of the file-based backup-and-transfer mechanisms and switch to ZFS send/recv. That should allow to shrink the data delta between main and off-site to minutes instead of hours or days. Does this make any sense?

Good suggestion. I've done blue-green professionally with services that are built to have high availability and in cloud environments. If I were to actually setup some form of that, I'd probably use ZFS send/rcv to keep a backup server always 15 minutes behind and ready to go. I wouldn't deal with file-based backups that take an hour to just walk the dataset to just figure out what's new. 😅 Probably not happening for now.

When you say stopping the service for an instant you must mean pausing its execution or at least its IO. Actually stopping the service can't be guaranteed to take an instant. It can't be guaranteed to start in an instant. Worst of all, it can't even be guaranteed that it'll be able to start again. When I say stopping I mean sysemctl stop or docker stop or pkill etc. In other words delivering an orderly, graceful kill signal and waiting for the process/es to stop execution.

That's the trivial scenario that we know won't fail - stopping the service during snapshot. The scenario that I was asking people's opinions on is not stopping the service during snapshot and what restoring from such backup would mean.

Let me contrast the two by completing your example:

• docker start container
• Time passes
• Time to backup
• docker stop container
• Make your snapshot
• docker start container
• Time passes
• Shit happens and restore from backup is needed
• docker stop container
• Restore from snapshot
• docker start container

Now here's the interesting scenario:

• docker start container
• Time passes
• Time to backup
• Make your snapshot
• Time passes
• Shit happens and restore from backup is needed
• docker stop container
• Restore from snapshot
• docker start container

Notice that in the second scenario we are not stopping the container. The snapshot is taken while it's live. This means databases and other files are open, likely actively being written to. Some files are likely only partially written. There are also likely various temporary lock files present. All of that is stored in the snapshot. When we restore from this snapshot and start the service it will see all of that. Contrast this with the trivial scenario when the service is stopped. Upon stopping it, all data is synced to disk, inflight database operations are completed or canceled, partial writes are completed or discarded, lock files are cleaned up. When we restore from such a snapshot and start the service, it will "think" it just starts from a clean stop, nothing extra to do. In the live snapshot scenario the service will have to do cleanup. For example it will have to decide what to do with existing lock files. Are they there because there's another instance of the service that is running and writing to the database or did someone kill its process before it had the chance to go through its shutdown procedure. In the former case it might have to log an error and quit. In the other it would have to remove the lock files. And so on and so forth.

As for th effect of docker on any of this, whether you have docker stop container or systemctl stop service or pkill service the effects on the process and its data is all the same. In fact the docker and systemctl commands will result in a kill signal being sent to the process of the service anyway.