Since I switched from VMware to Debian, I have struggled to find a proper solution for my KVM backups.
I tried hard!
A top-rated solution doesn’t like recent Linux kernels. Another one doesn’t support Debian 11. Another one put me into permission hell, and another didn’t even work as the first install requires a server with a GUI.
Why is it so difficult?
So, I left my comfort zone of dealing with a web dashboard and looked for command line options.
Good news: I found one, and besides the first steps at the beginning, it works very well!
What a name, can you pronounce it?
You can find the documentation here.
I’m on Debian, so be careful with my steps below as they might differ slightly from other distros.
Allow incremental backups
The tool requires a few changes on all VMs.
Yes, that includes the templates, so you don’t have to repeat them in the future.
Issue this command; in my example, “mssql” is the name of the VM:
virsh edit mssql
Delete the very first line and replace it with this one:
<domain type='kvm' id='1' xmlns:qemu=‘http://libvirt.org/schemas/domain/qemu/1.0'>
Now scroll to the bottom and right before </domain> paste these lines:
<qemu:capabilities> <qemu:add capability='incremental-backup'/> </qemu:capabilities>
It should look like this:
The change requires a restart of the VM, but as we need exclusive access to the disk image for our next step, I will shut it down.
Converting the image files
In one of my first attempts, I received an error message about “dirty bitmaps”.
The documentation let me down, as it only said, “consider migrating your qcow files to version 3 format”. What’s that about, now?
It took some google-fu to understand that there are different versions of the qcow image file format.
The latest one, and even that has existed for many years, is version 3. To make it worse, the files are still called qcow2.
There’s no qcow3.
Instead, they use an internal version number, and 0.10 is v2 while 1.1 is v3.
Got it? Well, it doesn’t matter, as here are the steps.
qemu-img convert -f qcow2 -O qcow2 -o compat=1.1 /path/to/your-image /path/to/your-new-image
Using my mssql example, it looks like this:
qemu-img convert -f qcow2 -O qcow2 -o compat=1.1 /mnt/vm/mssql /mnt/vm/mssqlx
Depending on the image size, this might take a few minutes, but eventually, you end up with two files on the datastore.
I suggest renaming mssql to mssqlold and the mssqlx to mssql.
Now start the VM and see if everything works.
Fine? Great, delete mssqlold.
Rinse, and repeat with all your VMs.
The first KVM backup
Now with all the prep work out of the way, everything else is easy.
For the first time, use this command to initiate the backup:
virtnbdbackup -d VMNAME -l full -o /backup/location/VMNAME
Again, here’s my example:
virtnbdbackup -d mssql -l full -o /mnt/vm-backup/mssql
The path /mnt/vm-backup leads to one of my Qnap devices.
It will take a few minutes and look like this:
Once the first full backup is finished, we continue to use incremental backups for the future.
It’s a similar command:
virtnbdbackup -d mssql -l inc -o /mnt/vm-backup/mssql
To automate it, I suggest visiting the crontab-generator.
Here are my results:
Crontab -e 0 2 * * * virtnbdbackup -d mssql -l inc -o /mnt/vm-backup/mssql >/dev/null 2>&1 5 2 * * * virtnbdbackup -d hco -l inc -o /mnt/vm-backup/hco >/dev/null 2>&1 10 2 * * * virtnbdbackup -d docker01 -l inc -o /mnt/vm-backup/docker01 >/dev/null 2>&1 15 2 * * * virtnbdbackup -d plex -l inc -o /mnt/vm-backup/plex >/dev/null 2>&1 20 2 * * * virtnbdbackup -d parrot -l inc -o /mnt/vm-backup/parrot >/dev/null 2>&1
Basically, I start at 02:00 in the night and give each backup 5 minutes; that’s plenty of time for an incremental update. The documentation explains that multiple jobs simultaneously are supported, but I’m in no rush.
Restoring a VM
A backup isn’t a backup until it’s tested.
I deleted one of my VMs via the Cockpit dashboard for my test. You can confirm the deletion with this command:
I’m honest with you; I didn’t use my mssql box for the test; I used an unimportant one. But I made sure I created a few incrementals after applying changes before killing it.
This command will provide information about the required disk space etc.:
virtnbdrestore -i /tmp/backupset/ -o dump
The command for a full restore is this:
virtnbdrestore -i /tmp/backupset/ -o /tmp/restore
In my case:
virtnbdrestore -i /mnt/vm-backup/parrot/ -o dump virtnbdrestore -i /mnt/vm-backup/parrot/ -o /mnt/vm03/parrot
I received an error message:
Restore path [/mnt/vm03/parrot] seems not to be an libvirt managed pool, skipping refresh.
What’s that? I’m not sure!
But according to the almighty internet, it helps to send a “virsh pool-refresh vm03,” but maybe I could have ignored it.
I will check it next time!
Inside the folder, I find both the XML and the image. I’m jumping into Cockpit, importing a VM, pointing to the image, and starting it.
It’s working fine, including the last changes I applied!
Okay, the restore process could be a little more convenient than placing the XML file in its location, but it’ll do for me.
That’s it. The preparations took a while, but it didn’t require an additional VM for a management system, and it took me less than two hours from discover virtnbdbackup until I could backup and restore a machine.
I’m wondering why all those vendors struggle.
Anyway, not my problem. I’m sorted.
More homelab posts:
Google Domains DDclient Dynamic DNS OPNsense
Earlier I wrote about setting up an OPNsense firewall. As I’m using Google Domains, I…
OPNsense IPv6 Telekom Magenta
Ich weiß nicht, wie viele physische und virtuelle Firewalls ich in meinem Homelab in den…
SolarWinds Hybrid Cloud Observability First Steps, part two
Are you ready to continue our first steps in the SolarWinds Hybrid Cloud Observability platform? …
Mullvad Wireguard qBittorrent Docker
Change is a process, or so they say.That applies to a homelab.Two weeks ago, I…
SolarWinds Hybrid Cloud Observability First Steps, part one
I deployed SolarWinds Hybrid Cloud Observability (HCO), and now I have started to adjust it.I…
SolarWinds Hybrid Cloud Observability – Installation
This is a “death by screenshot” style tutorial about a SolarWinds Hybrid Cloud Observability installation.I’m…
2 thoughts on “KVM Backups”
author of the utility, here! virtnbdrestore can also automatically define the virtual machine and adjust its configuration to the new restore pathes if you use the -cD and –name options.
Thanks for the update! That will surely make it even more convenient.