Block copy still active: disk 'vda' not ready for pivot yet #23
Description
I am having some issues with backup-vm. The below is happening to me fairly often. Seems to be more frequent for some VMs than others, which could be depending on the load/io in the guest I guess.
Most VMs have worked on first try. Some have required 2-3 tries. But I have one that has not yet worked at all, after 5-6 tries. It has however finished vda once or twice, but then it got stuck on vdb instead, so it's a bit random on that one as well. With the plan to run backups automatically daily or weekly this is a bit of an issue.
backup progress: 100%
libvirt: error code 83: block copy still active: disk 'vda' not ready for pivot yet
Traceback (most recent call last):
File "/usr/local/bin/backup-vm", line 11, in
load_entry_point('backup-vm==0.1.dev30+gf2d6dfd', 'console_scripts', 'backup-vm')()
File "/usr/local/lib/python3.7/dist-packages/backup_vm-0.1.dev30+gf2d6dfd-py3.7.egg/backup_vm/backup.py", line 54, in main
borg_failed = multi.assimilate(args.archives)
File "/usr/local/lib/python3.7/dist-packages/backup_vm-0.1.dev30+gf2d6dfd-py3.7.egg/backup_vm/snapshot.py", line 175, in exit
self.blockcommit(disks_to_backup)
File "/usr/local/lib/python3.7/dist-packages/backup_vm-0.1.dev30+gf2d6dfd-py3.7.egg/backup_vm/snapshot.py", line 105, in blockcommit
if self.dom.blockJobAbort(disk.target, libvirt.VIR_DOMAIN_BLOCK_JOB_ABORT_PIVOT) < 0:
File "/usr/lib/python3/dist-packages/libvirt.py", line 784, in blockJobAbort
if ret == -1: raise libvirtError ('virDomainBlockJobAbort() failed', dom=self)
libvirt.libvirtError: block copy still active: disk 'vda' not ready for pivot yet
I don't understand if the issue is in backup-vm or libvirt actually. I found this old, but recently fixed bug in Ubuntu. It's only for older releases though. The package libvirt doesn't exist in neither Ubuntu or Debian anymore.
https://launchpad.net/ubuntu/+source/libvirt/1.3.1-1ubuntu10.29
When the above happens I can see this:
virsh blockjob pgc-srtm-01 vda --info
Active Block Commit: [100 %]
virsh domblklist pgc-srtm-01
Target Source
vda /var/lib/libvirt/images/pgc-srtm-01-vda-tempsnap.qcow2
vdb /dev/pgc-kvm-04/pgc-srtm-01_srtm
I can fix the state by running these two commands:
virsh blockjob pgc-srtm-01 vda --abort
virsh blockcommit pgc-srtm-01 vda --active --verbose --pivot
Sometimes it does however leave the qcow2 in /var/lib/libvirt/images/, and also still links it in the xml. It seems to work fine to just remove the qcow2 file, edit the xml and virsh define it again.
My VMs uses LVM logical volumes as storage back-end. System is Debian Stable (Buster), fully upgraded.
I have also experienced the error message seen in issue #15. Not sure if it's related, but I feel that it could be.
Since I can reproduce it, anything I can do to provide more information?
Otherwise, thank you for a great software. I am really hoping to get it working in my environment as well. It does almost everything I wish for!