5
0
mirror of git://git.proxmox.com/git/qemu-server.git synced 2025-01-25 06:03:52 +03:00

106 Commits

Author SHA1 Message Date
Dominik Csapak
fbec3f894a use get_repository from PVE::PBSClient
this fixes the issue that we did not generate the correct repository
url for pbs storages that contained an ipv6 address or a port

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-03 17:25:32 +01:00
Stefan Reiter
962d4d647d vzdump: use dirty bitmap for not running VMs too
Now that VMs can be started during a backup, it makes sense to create a
dirty bitmap in these cases too, since the VM might be resumed and thus
continue running normally even after the backup is done.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-05 11:22:47 +01:00
Stefan Reiter
4ac842cbab vzdump: connect to qmeventd for duration of backup
Connect and send the vmid of the VM being backed up. This prevents
qmeventd from SIGTERMing the underlying QEMU instance, even if the guest
shuts itself down, until we close the socket connection (in cleanup,
which happens on success and abort, or if we crash the file handle will
be closed as well).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-05 11:22:47 +01:00
Dominik Csapak
edae17185b partially fix #3056: try to cancel backup without uuid
if the 'backup' qmp call itself times out or fails, we still want to
try to cancel the backup, else it can happen that there is still
a backup running even when vzdump thinks it was canceled

qapi docs says that backup cancel always returns success, even
if no backup is running

since we hold a global and a per vm lock for the backup, this should be
ok, since we should not reach this code without that lock

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-29 18:19:06 +01:00
Thomas Lamprecht
d5b0cfb197 followup: s/verification/validation/ to avoid confusion
We use verification for something more in-depth on the PBS server, so
avoid that term to avoid misunderstandings.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-29 17:39:28 +02:00
Stefan Reiter
1eb7e59017 vzdump: log 'finishing' state
...and avoid printing 100% status twice

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-09-29 17:28:15 +02:00
Thomas Lamprecht
5749c38350 vzdump: do not log per disk dirty bitmap state for templates
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-21 14:11:48 +02:00
Fabian Grünbichler
4ef13a7f9a fix #2862: properly backup (all) VM templates
until we maybe have a 'pbs-backup' that links Qemu and PBS like
'pbs-restore', we need to do a regular backup for the template case to
support all storage types and image formats.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-21 13:45:55 +02:00
Stefan Reiter
1ca43f1cde vzdump: show "reused" message even if nothing was transferred
This still works even if all drives were clean. It then shows the very
magical line:

  INFO: backup was done incrementally, reused 34.00 GiB (100%)

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-20 15:54:54 +02:00
Stefan Reiter
2790636afd vzdump: allow bandwidth limit for PBS backup too
QEMU handles it just as well as with VMA, so this was probably just
forgotten to implement for PBS.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-20 15:54:54 +02:00
Thomas Lamprecht
f856671495 vzdump: restore log behavior for VMs running with older QEMU
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-20 10:45:34 +02:00
Thomas Lamprecht
a40295b6aa vdzump: padd percentagde done counter, format duration for humans
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-20 10:25:08 +02:00
Thomas Lamprecht
d35412a32a vzdump: output sparseness again
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-20 10:25:08 +02:00
Thomas Lamprecht
bafae3ec84 vzdump: log reused information also if we run only <1s
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-20 10:24:38 +02:00
Thomas Lamprecht
13ddc7eb91 vzdump: followup: log total dirtiness again, minor output adaptions
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-20 10:24:38 +02:00
Stefan Reiter
78179bda0f vzdump: don't use dirty bitmap when VM was off
There can't be a dirty bitmap when the VM was off, and if it was off we
will also shut it down after the backup, so no point in creating one.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-19 18:20:34 +02:00
Stefan Reiter
0f6c69814b vzdump: log 100% percent in case $target is 0
When $target is 0, that means we don't have to upload any data, in which
case we're immediately done.

Otherwise incremental backups with no changes display a really weird
  status: 0% (0.0 B of 0.0 B), duration 0, read: 0 B/s, write: 0 B/s
when they're actually done already.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-19 18:20:34 +02:00
Stefan Reiter
8825248c43 vzdump: display actually uploaded chunks as 'write' speed
Previously 'read' and 'write' would always show the same value, which is
of little use. Change it so 'write' excludes reused bytes, thus
displaying the actual upload speed.

$last_reused needs to be initialized to contain reused data from 'clean'
dirty bitmaps to ensure the first output line is correct.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-19 18:20:34 +02:00
Stefan Reiter
3a44897f74 vzdump: improve logging output with dirty bitmaps
Uses the new 'query-pbs-bitmap-info' QMP call to retrieve additional
information about each drive's dirty bitmap. Returned info is also used
to calculate $target by simply adding all the dirty values (dirty is
equal to size in case the entire drive will be backed up).

"Backup is sparse" message is suppressed for PBS, since it makes little
sense (if zero chunks appear in the clean area of a bitmap, they won't
be counted, and a user is probably more interested in the 'reused' data
anyway).

Also removes the need for the hacky $first_round query-backup handling.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-19 18:20:34 +02:00
Wolfgang Bumiller
ece740308f support for encrypted pbs backups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-10 14:23:17 +02:00
Stefan Reiter
fb9f512c19 backup: detect PBS features and use only supported
If 'query-proxmox-support' is not known to QEMU, assume that no other
features are supported either.

If 'pbs' is not supported at all, error out with a nice message.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 13:44:22 +02:00
Thomas Lamprecht
91c9f3b062 vzdump: log: small fixups
reuse can also come from the current backup - so drop the "from last
backup" as this can be very confusing if one reads it after making
the first backup ever, with no last backup existing.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-07 17:32:27 +02:00
Thomas Lamprecht
2098f2ff87 vzdump: fix variable redeclaration warning
happened due to moving the code from another scope which had no $res,
and not noticing as it was still working after all.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-07 08:41:30 +02:00
Thomas Lamprecht
b4be9c0207 backup: PBS: save backup task size
normally this is done centrally in the managers code, but we do not
have the info for PBS there.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 21:59:50 +02:00
Thomas Lamprecht
6cdb568c33 backup: query status: use latest up-to-date info when finished
we used the info from the previous loop for some summary logs, fix
that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 21:59:50 +02:00
Thomas Lamprecht
4d159c24d9 backup: restructure log: use human-readable sizes and include PBS info
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 21:59:50 +02:00
Thomas Lamprecht
bbdf1ea5c8 backup: rename confusing incremental param to use-dirty-bitmap
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 20:33:13 +02:00
Stefan Reiter
69e6289470 enable dirty-bitmap incremental backups for PBS
The $total != $transferred check is changed to a log, as QEMU reports
only actually transferred bytes, and it is indeed correct for
incremental backups to have differing values from $total.

The 'incremental' parameter is always set, QEMU will figure out if it should
re-use an existing bitmap or create a new one on its own.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-29 13:15:47 +02:00
Dietmar Maurer
f57666e949 avoid backup command timeout with pbs 2020-06-26 09:30:31 +02:00
Thomas Lamprecht
5a92276e2d trivial whitespace followup fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-24 10:59:05 +02:00
Aaron Lauterer
185df962a5 vzdump: move include logic for volumes to method
Move the logic which volumes are included in the backup job to its own
method and adapt the VZDump code accordingly. This makes it possible to
develop other features around backup jobs.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-06-24 10:56:57 +02:00
Dominik Csapak
77b24c6267 fix pbs backups of templates/vms without disks
fingerprint is now given with an environment variable not as parameter

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-03 14:36:31 +02:00
Fabian Grünbichler
690748632b vzdump: use new 'pbs' option
instead of storage config to determine whether we are in 'PBS mode'

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-05-06 16:20:46 +02:00
Fabian Grünbichler
6bb12239cc vzdump: fix template backup to stdout
redirecting to the saved STDOUT in case of a template backup or a VM
without any disks failed because of the erroneous '=':

Backup of VM 123123 failed - command '/usr/bin/vma create -v -c [...]' failed:
Bad filehandle: =5 at /usr/share/perl/5.28/IPC/Open3.pm line 58.

https://forum.proxmox.com/threads/vzdump-to-stdout.69364

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-05-06 14:23:04 +02:00
Thomas Lamprecht
1bd6fe132f vzdump: assemble: fix skipping all pending and snapshot config entries
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-04 17:29:09 +02:00
Thomas Lamprecht
cb521f2d88 backup: never try to freeze in stop mode backup
VM was can be true for stop mode backup, we cannot check the "is VM
currently running" as that doesn't tells us anything (could be the
backup process), so check the mode also..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-03 11:13:38 +02:00
Fabian Ebner
912792e245 Switch to using foreach_volume instead of foreach_drive
It was necessary to move foreach_volid back to QemuServer.pm

In VZDump/QemuServer.pm and QemuMigrate.pm the dependency on
QemuConfig.pm was already there, just the explicit "use" was missing.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-08 22:11:54 +02:00
Fabian Grünbichler
0c498cca36 vm_start: condense signature
as preparation for refactoring it further. remote migration will add
another 1-2 parameters, and it is already unwieldly enough as it is.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-01 17:42:15 +02:00
Thomas Lamprecht
e485854100 vzdump: fix up PBS plugin
fixes commit 0b2f574b4ccd68ba7c1ebf1927beeff33cca179d

enforce_vm_running_for_backup is now witout return value, for the PBS
I forgot to remove an now outdated call to handle_vm_powerstate, drop
that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-26 08:51:29 +01:00
Thomas Lamprecht
e8705fc59e vzdump archive_vma: move getting QMPClient nearer to using sites
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:48:41 +01:00
Thomas Lamprecht
d11e91d2de vzdump: indentation and code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:46:49 +01:00
Thomas Lamprecht
0b2f574b4c vzdump: factor out VM powerstate handling and qmp backup cancel
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:41:43 +01:00
Thomas Lamprecht
1ece829a39 vzdump: factor out QGA FS freeze/thaw
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:37:20 +01:00
Thomas Lamprecht
02da0c65bb vzdump: factor out fork_compressor_pipe
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:34:31 +01:00
Thomas Lamprecht
81dcd4799f vzdump: factor out _get_task_devlist
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:32:00 +01:00
Thomas Lamprecht
09eb196b9e vzdump: refactor and cleanup code in query_backup_status_loop
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:26:34 +01:00
Thomas Lamprecht
f301bc0d34 vzdump: use QemuConfig directly to set/unset locks
no need to fork/exec a process for that..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 15:58:43 +01:00
Thomas Lamprecht
cd544fb2b9 vzdump: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 15:58:13 +01:00
Thomas Lamprecht
a113a58a64 vzdump prepare: refactor and code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 11:48:19 +01:00
Dietmar Maurer
f6168f1a6a archive_pbs: call backup_cancel only when job is really started 2020-03-11 11:47:24 +01:00