5
0
mirror of git://git.proxmox.com/git/qemu-server.git synced 2025-02-03 13:47:15 +03:00

2473 Commits

Author SHA1 Message Date
Fabian Ebner
48831384b8 create test environment for migration
and the associated parts for 'qm start'.

Each test will first populate the MigrationTest/run directory
with the relevant configuration files and files keeping track of the
state of everything necessary. Second, the mock-script for migration
is executed, which in turn will execute the 'qm start' mock-script
(if it's an online test that gets far enough). The scripts will simulate
a migration and update the relevant files in the MigrationTest/run directory.
Finally, the main test script will evaluate the state.

The main checks are the volume IDs on the source and target and the VM
configuration itself. Additional checks are the vm_status and expected_calls,
keeping track if certain calls have been made.

The rationale behind creating two mock-scripts is two-fold:
1. It removes the need to hard code responses for the tunnel
   and to recycle logic for determining and allocating migration volumes.
   Some of that logic already happens in the API part, so it was necessary
   to mock the whole CLI-Handler.
2. It allows testing the code relevant for migration in 'qm start' as well,
   and it should even be possible to test different versions of the
   mock-scripts against each other. With a bit of extra work and things
   like 'git worktree', it might even be possible to automate this.

The helper get_patched config is useful to change pre-defined configuration
files on the fly, avoiding the new to explicitly define whole configurations to
test for something in many cases.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-12-15 15:21:37 +01:00
Fabian Ebner
eb3acec88a migration: sort volumes migrated with storage_migrate
Having a deterministic order here is useful for testing.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-12-15 15:21:37 +01:00
Fabian Ebner
7d730f953c migration: factor out starting remote tunnel
so it can be mocked when testing.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-12-15 15:21:37 +01:00
Fabian Ebner
27fa645e66 use new move_config_to_node method
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-12-15 15:21:37 +01:00
Fabian Ebner
b5688f69a0 clone_disk: fix offline clone of efidisk
by partially reverting 4df98f2f14348d0ed57529c4c04a1b5ffb840055 and fixing the
line-length issue differently. The commit didn't update two later usages of
$size, breaking copying the efidisk. The other usage as a parameter to
qemu_img_convert() is luckily only cosmetic, for progress output.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-12-15 14:50:06 +01:00
Thomas Lamprecht
e00319af4d api: adapt VM destroy description
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-15 14:49:41 +01:00
Dominic Jäger
3eecc92525 qm destroy: Extend --purge description
Add replication jobs & HA. This makes the enumeration complete.

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2020-12-15 14:46:14 +01:00
Thomas Lamprecht
6711e689c6 bump version to 6.3-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-03 18:08:04 +01:00
Thomas Lamprecht
82d30d5acf d/control: bump versioned dependency for libpve-common-perl
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-03 17:26:24 +01:00
Dominik Csapak
fbec3f894a use get_repository from PVE::PBSClient
this fixes the issue that we did not generate the correct repository
url for pbs storages that contained an ipv6 address or a port

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-03 17:25:32 +01:00
Thomas Lamprecht
012b520e5b bump version to 6.3-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-25 14:33:38 +01:00
Dominik Csapak
e8e0fd93bf qmeventd: flush after verbose printing
if one would try to use -v in a systemd service, systemd would disable
line buffering for stdout and no output would happen (until the buffer
is full)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-25 14:33:38 +01:00
Thomas Lamprecht
3bae384f75 clone disk: avoid errors after disk was moved by QEMU
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-25 14:18:23 +01:00
Thomas Lamprecht
1b987638a8 api: cleanup code format of clone_disk call
showing off it's monstrosity of a method signature, needs to be
cleaned up in a followup commit

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-25 14:18:23 +01:00
Thomas Lamprecht
a2af1bbe89 add and use get_qga_key
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-25 14:18:23 +01:00
Fabian Grünbichler
e5b18771b8 status: skip query-proxmox-support if VM is offline
otherwise pvestatd will print lots of warnings..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-25 11:26:37 +01:00
Stefan Reiter
6891fd70ed print query-proxmox-support result in 'full' status
Extends print_recursive_hash for the CLI to handle JSON booleans so the
result will actually show up in 'qm status --verbose'.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-24 17:20:56 +01:00
Fabian Grünbichler
1b535ca9f9 d/control: bump versioned dependency on pve-storage
for 'activate_volumes in storage_migrate', which we now rely on in
migration code

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 16:28:08 +01:00
Fabian Ebner
e219712561 deactivate volumes after storage_migrate
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-24 16:19:35 +01:00
Fabian Ebner
78bd57d9c3 adapt to new storage_migrate activation behavior
Offline migrated volumes are now activated within storage_migrate.
Online migrated volumes can be assumed to be already active.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-24 16:19:29 +01:00
Alexandre Derumier
6cbd3eb82c systemd scope: add CPUWeight for cgroupv2 2020-11-24 12:00:38 +01:00
Alexandre Derumier
5b65b00d04 replace cgroups_write by cgroup change_cpu_shares && change_cpu_quota 2020-11-24 12:00:38 +01:00
Wolfgang Bumiller
114d2e765a add PVE::QemuServer::Cgroup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-24 12:00:33 +01:00
Thomas Lamprecht
ff0721517d bump version to 6.2-20
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-12 17:09:44 +01:00
Thomas Lamprecht
c0d096321a stop qmeventd after pve-guests and pve-ha-lrm services
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Co-developed-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-12 17:06:33 +01:00
Fabian Ebner
19ff368213 don't migrate replicated VM whose replication job is marked for removal
while it didn't actually fail, we probably want to avoid the behavior:

With remove_job=full:
    * run_replication called during migration causes the replicated volumes to
      be removed
    * migration continues by fully copying all volumes

With remove_job=local:
    * run_replication called during migration causes the job (and local
      replication snapshots) to be removed
    * migration continues by fully copying all volumes and renaming them to
      avoid collision with the still existing remote volumes

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-09 10:08:22 +01:00
Fabian Ebner
c2c96d7378 fix checks for transfering replication state/switching job target
In some cases $self->{replicated_volumes} will be auto-vivified
to {} by checks like
next if $self->{replicated_volumes}->{$volid}
and then {} would evaluate to true in a boolean context.

Now the replication information is retrieved once in prepare,
and used to decide whether to make the calls or not.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-09 10:08:22 +01:00
Fabian Ebner
68980d6626 Repeat check for replication target in locked section
No need to warn twice, so the warning from the outside check
was removed.

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-09 10:08:22 +01:00
Thomas Lamprecht
1749a376dc bump version to 6.2-19
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-05 13:37:03 +01:00
Stefan Reiter
8e0c97bbbf fix vm_resume and allow vm_start with QMP status 'shutdown'
When the VM is in status 'shutdown', i.e. after the guest issues a
powerdown while a backup is running, QEMU requires a 'system_reset' to
be issued before 'cont' can boot the guest again.

Additionally, when the VM has been powered down during a backup, the
logically correct call would be a 'vm_start', so automatically vm_resume
from vm_start in case this situation occurs. This also means the GUI can
cope with this almost unchanged.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-05 11:22:47 +01:00
Stefan Reiter
27b25d037e config_to_command: use -no-shutdown option
Ignore shutdowns triggered from within the guest in favor of detecting
them via qmeventd and stopping the QEMU process that way.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-05 11:22:47 +01:00
Stefan Reiter
962d4d647d vzdump: use dirty bitmap for not running VMs too
Now that VMs can be started during a backup, it makes sense to create a
dirty bitmap in these cases too, since the VM might be resumed and thus
continue running normally even after the backup is done.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-05 11:22:47 +01:00
Stefan Reiter
4ac842cbab vzdump: connect to qmeventd for duration of backup
Connect and send the vmid of the VM being backed up. This prevents
qmeventd from SIGTERMing the underlying QEMU instance, even if the guest
shuts itself down, until we close the socket connection (in cleanup,
which happens on success and abort, or if we crash the file handle will
be closed as well).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-05 11:22:47 +01:00
Stefan Reiter
4c500f1696 qmeventd: add last-ditch effort SIGKILL cleanup
'alarm' is used to schedule an additionaly cleanup round 5 seconds after
sending SIGTERM via terminate_client. This then sends SIGKILL via a
pidfd (if supported by the kernel) or directly via kill, making sure
that the QEMU process is *really* dead and won't be left behind in an
undetermined state.

This shouldn't be an issue under normal circumstances, but can help
avoid dead processes lying around if QEMU hangs after SIGTERM.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-05 11:22:47 +01:00
Stefan Reiter
3ff8500175 qmeventd: add handling for -no-shutdown QEMU instances
We take care of killing QEMU processes when a guest shuts down manually.
QEMU will not exit itself, if started with -no-shutdown, but it will
still emit a "SHUTDOWN" event, which we await and then send SIGTERM.

This additionally allows us to handle backups in such situations. A
vzdump instance will connect to our socket and identify itself as such
in the handshake, sending along a VMID which will be marked as backing
up until the file handle is closed.

When a SHUTDOWN event is received while the VM is backing up, we do not
kill the VM. And when the vzdump handle is closed, we check if the
guest has started up since, and only if it's determined to still be
turned off, we then finally kill QEMU.

We cannot wait for QEMU directly to finish the backup (i.e. with
query-backup), as that would kill the VM too fast for vzdump to send the
last 'query-backup' to mark the backup as successful and done.

For handling 'query-status' messages sent to QEMU, a state-machine-esque
protocol is implemented into the Client struct (ClientState). This is
necessary, since QMP works asynchronously, and results arrive on the
same channel as events and even the handshake.

For referencing QEMU Clients from vzdump messages, they are kept in a
hash table. This requires linking against glib.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-05 11:22:47 +01:00
Fabian Grünbichler
acfc6ef8e0 fix #3113: unbreak drive hotplug
by adding the missing argument (otherwise all the other ones are shifted
one slot to the left, which is of course bogus).

this has been broken since 2018 (d559309), but was only made
visible/caused a failure with the recent changes adding

use strict;
use warnings;

to PVE::QemuServer::PCI

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-05 10:29:21 +01:00
Thomas Lamprecht
8c9021cd69 bump version to 6.2-18
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-29 18:23:23 +01:00
Dominik Csapak
edae17185b partially fix #3056: try to cancel backup without uuid
if the 'backup' qmp call itself times out or fails, we still want to
try to cancel the backup, else it can happen that there is still
a backup running even when vzdump thinks it was canceled

qapi docs says that backup cancel always returns success, even
if no backup is running

since we hold a global and a per vm lock for the backup, this should be
ok, since we should not reach this code without that lock

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-29 18:19:06 +01:00
Stefan Reiter
acc10e5159 migrate: enable dirty-bitmap migration
We query QEMU if it's safe before enabling it, as on versions without
the necessary patches it not only would be useless, but can actually
lead to hangs.

PBS state is always migrated, as it's a small amount of data anyway, so
we don't need to set a specific flag for it.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-29 18:18:02 +01:00
Thomas Lamprecht
31655cbb7f bump version to 6.2-17
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-22 16:09:03 +02:00
Stefan Reiter
4c27b18c08 bootorder: don't print empty 'order=' property
Specifying 'boot: order=' was intended to be used for an empty bootorder
(i.e. no boot devices), but as it turns out our format parser doesn't
like empty '-list' properties if they are nested in a subformat.

Fixing this in JSONSchema sounds like a risky move, so instead just
write 'boot: ' (without 'order=') to indicate an empty bootorder. The
rest of the code handles it just fine, as this was valid before too.

Incidentally also fixes a bug where you couldn't create a new VM without
any disks if no explicit 'boot' property was specified (i.e. a simple
'qm create 100' without any parameters would fail).

Reported-by: Dominic Jäger <d.jaeger@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-22 08:47:45 +02:00
Thomas Lamprecht
75127d1996 bump version to 6.2-16
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-19 15:51:54 +02:00
Thomas Lamprecht
2bf945fcb9 tests: make module truthy
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-19 15:33:16 +02:00
Thomas Lamprecht
ce11958aab tests: do not use for-loop for globs
they are rather inefficient for this

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-19 15:32:31 +02:00
Thomas Lamprecht
6ef6d68f6c cloudinit: use normal grep
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-19 15:32:03 +02:00
Thomas Lamprecht
50bbe37787 comment out USB hotplug code
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-19 15:31:38 +02:00
Thomas Lamprecht
808a65b522 fix some FH close
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-19 15:30:53 +02:00
Thomas Lamprecht
41af2dfc25 PCI: use warnings/strict and fix setting $vga from config2command
fixes commit 74c17b7a23c8a953d1dcec9bd53449d71d88cd5d which moved
this code here, but forgot to pass $vga ref, as the module was not
using warning nor strict mode this was not caught..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 18:03:32 +02:00
Thomas Lamprecht
f7d1505b0c tree wide cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 18:03:32 +02:00
Thomas Lamprecht
009cbf61bc api: factor out common vm_is_paused code in private helper
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 17:51:24 +02:00