5
0
mirror of git://git.proxmox.com/git/qemu-server.git synced 2025-01-25 06:03:52 +03:00

1746 Commits

Author SHA1 Message Date
Thomas Lamprecht
e214cda8f5 followup: s/targetstorage map/storage map/ for parameter exceptions
to reduce duplicated info in the error slightly

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-07 17:20:37 +02:00
Thomas Lamprecht
abff03211f followup: refactor loop to use perls values
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-07 17:15:28 +02:00
Fabian Grünbichler
84da82174e vm_start: collect and return migration info
in addition to printing it. preparation for remote cluster migration,
where we want to return this in a structured fashion over the migration
tunnel instead of parsing stdout via SSH.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-07 17:07:36 +02:00
Fabian Grünbichler
ba5396b580 vm_start: split NBD disk allocation further
into one sub that retrieves the local disks, and the actual NBD
allocation. that way, remote incoming migration can just call the NBD
allocation with a custom list of volume names/storages/..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-07 17:07:32 +02:00
Fabian Grünbichler
aea447bb9d migrate: check storage permissions and content type
both where previously missing. the existing 'check_storage_access'
helper is not applicable here since it operates on a full set of VM
config options, not just storage IDs.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-07 17:07:23 +02:00
Fabian Grünbichler
bf8fc5a307 migrate: allow arbitrary source->target storage maps
the syntax is backwards compatible, providing a single storage ID or '1'
works like before. the new helper ensures consistent behaviour at all
call sites.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-02 17:47:14 +02:00
Stefan Reiter
c05f1b33ea migration: fix downtime limit auto-increase
485449e37 ("qmp: use migrate-set-parameters in favor of deprecated options")
changed the initial "migrate_set_downtime" QMP call to the more recent
"migrate-set-parameters", but forgot to do so for the auto-increase code
further below.

Since the units of the two calls don't match, this would have caused the
auto-increase to increase the limit to absurd levels as soon as it kicked
in (ms treated as s).

Update the second call to the new version as well, and while at it remove
the unnecessary "defined()" check for $migrate_downtime, which is always
initialized from the defaults anyway.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-04-02 16:48:51 +02:00
Fabian Grünbichler
e5fd1c6591 restore: parse volid with $noerr
otherwise VMA files passed in as paths instead of as volids don't
work anymore.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-02 15:14:42 +02:00
Fabian Grünbichler
13cfe3b7a7 vm_start: split out NBD disk allocation
as preparation of targetstorage mapping and remote migration. this also
removes re-using of the $local_volumes hash in the original code.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-01 18:43:00 +02:00
Fabian Grünbichler
3898a56343 vm_start: split out lock + initial checks
to start breaking up vm_start before extending parts for new migration
features like storage and network mapping.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-01 18:41:58 +02:00
Fabian Grünbichler
0c498cca36 vm_start: condense signature
as preparation for refactoring it further. remote migration will add
another 1-2 parameters, and it is already unwieldly enough as it is.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-01 17:42:15 +02:00
Fabian Grünbichler
6a039d06e9 migrate: improve cleanup_remotedisks
to also handle cases where disk allocation failed in the remote
vm_start, and we only have a bitmap but no target drive information.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-01 17:41:07 +02:00
Fabian Grünbichler
0c4fef3f26 rescan: fix call to foreach_volume
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-01 13:02:59 +02:00
Fabian Ebner
ca04977c63 update_disk_config: use config volume iterators instead of loops
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-03-30 09:59:48 +02:00
Fabian Ebner
beed0e3b0c Implement volume-related helpers and use new foreach_volume
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-03-30 09:59:48 +02:00
Dominik Csapak
818ce80ec1 fix efidisks on storages with minimum sizes bigger than OVMF_VARS.fd
on storages where the minimum size of images is bigger than the real
OVMF_VARS.fd file, they get padded to their minimum size

when using such an image, qemu maps it fully to the vm, but the efi
does not find the vars region and creates a file on the first efi
partition it finds

this breaks some settings in the ovmf, such as resolution

to fix this, we have to specify the size for the pflash, so that
qemu only maps the first n bytes in the vm (this only works for
raw files, not for qcow2)

we also have to use the correct size when converting between storages
in 'clone_disk' (used for move disk and cloning vms) and when
live migrating to different storages

when we now expect that the source image is always correctly used/created
(e.g. raw with size=x in pflash argument) then we always create the
target correctly

when encountering users which have a non-valid image (e.g. a efidisk
moved from zfs to qcow2 before this patch), we have to tell them to
recreate the efidisk and the settings on it

we have to version_guard it to 4.1+pve2 (since we haven't bumped yet
since the change to pve2)

also add 2 tests, one for the old version and one for the new

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>
[ Thomas: rebased to master ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-30 09:41:55 +02:00
Fabian Ebner
5c50a84f23 migration with targetstorage: check if target storage supports images
This makes sure that live migration also respects content types.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-03-27 14:32:42 +01:00
Thomas Lamprecht
2cd808d331 migrate sync disks: split long line
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-27 10:17:54 +01:00
Thomas Lamprecht
b10afa311d migrate sync_disks: use own variable for often referenced storage config
also fix two places where we used $self->{vmid} even if $vmid was in
scope (and the same).

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-27 10:13:10 +01:00
Fabian Grünbichler
9b3f5a5c99 migrate: cleanup disk/bitmaps if 'qm start' failed
since bitmaps are set early on, and 'qm start' potentially has allocated
the disks but still failed. we can only clean up what we know about
anyway, so the disk part is still only best effort.

also use replicated_volumes instead of bitmap existence to check for
replicated volumes, since 'qm start' on an old node that does not
understand replicated volumes might have allocated a new volume that we
DO want to clean up, and not skip.

also cleanup disks after stopping target VM, otherwise we might end up
in a situation where the target VM is still running and using the disks,
thus blocking the disk cleanup.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-03-27 07:54:44 +01:00
Fabian Grünbichler
7f5fb49a7c migrate: fix auto-vivification in cleanup_bitmaps
this does not currently trigger since nothing uses $self->{target_drive}
afterwards.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-03-27 07:54:44 +01:00
Fabian Grünbichler
88126be3f7 migrate: fix replication false-positives
by only checking for replicatable volumes when a replication job is
defined, and passing only actually replicated volumes to the target node
via STDIN, and back via STDOUT.

otherwise this can pick up theoretically replicatable, but not actually
replicated volumes and treat them wrong.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-03-27 07:54:44 +01:00
Fabian Grünbichler
1dbe979c7c CPUConfig: fix module load when pmxcfs is unavailable
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-03-26 09:03:07 +01:00
Thomas Lamprecht
e485854100 vzdump: fix up PBS plugin
fixes commit 0b2f574b4ccd68ba7c1ebf1927beeff33cca179d

enforce_vm_running_for_backup is now witout return value, for the PBS
I forgot to remove an now outdated call to handle_vm_powerstate, drop
that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-26 08:51:29 +01:00
Stefan Reiter
5d008ad383 Verify VM-specific CPU configs seperately
$cpu_fmt is being reused for custom CPUs as well as VM-specific CPU
settings. The "pve-vm-cpu-conf" format is introduced to verify a config
specifically for use as VM-specific settings.

"pve-cpu-conf" is registered for use in custom CPU API calls (where no
additional checks are required).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-25 18:42:24 +01:00
Stefan Reiter
b3e894883a Adapt CPUConfig to handle custom models
Turn CPUConfig into a SectionConfig with parsing/writing support for
custom CPU models. IO is handled using cfs.

Namespacing will be provided using "custom-" prefix for custom model
names (in VM config only, cpu-models.conf will contain unprefixed
names).

Includes two overrides to avoid writing redundant information to the
config file, additionally get_custom_model is used to retrieve a custom
model configuration by name.

Resolve custom names in print_cpu_device when a custom cpu is passed.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-25 18:42:21 +01:00
Fabian Ebner
47250f03ef Fix calls to get_replicateable_volumes
There is a need to set $noerr, because otherwise migration for a
VM with a non-replicatable volume fails with:
missing replicate feature on volume 'myfs:107/vm-107-disk-2.raw'

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-03-25 14:53:17 +01:00
Thomas Lamprecht
6d7450cbec qemu migrate: sort and split module usage
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-25 10:05:58 +01:00
Thomas Lamprecht
28e6e180bc add basic version check for live-migration with replicated disks
as we need at least pve-qemu in 4.2 for this to work, the target side
is implicitly checked with "to old version" check for migrate or the
mirror will fail anyway.

Just use the simple "qemu binary version check", as we could stil
live migrate an older snapshot with older machine versions if both
sides have a recent enough qemu.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-25 10:02:36 +01:00
Fabian Grünbichler
9b6efe436d migrate: add live-migration of replicated disks
with incremental drive-mirror and dirty-bitmap tracking.

1.) get replicated disks that are currently referenced by running VM
2.) add a block-dirty-bitmap to each of them
3.) replicate ALL replicated disks
4.) pass bitmaps from 2) to drive-mirror for disks from 1)
5.) skip replicated disks when cleaning up volumes on either source or
target

added error handling is just removing the bitmaps if an error occurs at
any point after 2, except when the handover to the target node has
already happened, since the bitmaps are cleaned up together with the
source VM in that case.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-24 12:22:32 +01:00
Fabian Grünbichler
b9f44d2773 migrate: add replication info to disk overview
to make migration logs a bit easier to grasp with a quick glance.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Tested-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-24 11:54:32 +01:00
Fabian Grünbichler
bc6c823164 drive-mirror: add support for incremental sync
by re-using a dirty bitmap that represents changes since the divergence
of source and target volume. requires a qemu that supports incremental
drive-mirroring, and will die otherwise.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Tested-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-24 11:54:20 +01:00
Fabian Ebner
43c4c7b693 Add unused description to drivedesc_hash
Moved code so that initialization of drivedesc_hash stays a single block.
Avoid auto-vivication in parse_drive.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-03-23 09:58:30 +01:00
Stefan Reiter
47f35977cb version_guard: early out when major/minor version is high enough
E.g.: If a feature requires 4.1+pveN and we're using machine version 4.2
we don't need to increase the pve version to N (4.2+pve0 is enough).

We check this by doing a min_version call against a non-existant higher
pve-version for the major/minor tuple we want to test for, which can
only work if the major/minor alone is high enough.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-23 09:58:17 +01:00
Thomas Lamprecht
1e0074c437 migrate phase3: add to comment why a blockjob cancel is OK here
Clarify why a cancel is actually not really canceling here, because
we're already finished with storage migration and the block jobs are
all in ready state and we (source) are going to stop soon to hand
over to target.

> Note that if you issue 'block-job-cancel' after 'drive-mirror' has
> indicated (via the event BLOCK_JOB_READY) that the source and
> destination are synchronized, then the event triggered by this
> command changes to BLOCK_JOB_COMPLETED, to indicate that the
> mirroring has ended and the destination now has a point-in-time
> copy tied to the time of the cancellation
-- qapi/block-core.json (QEMU 4.2)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-20 11:08:23 +01:00
Mira Limbeck
ff09c795ed revert spice_ticket prefix change in 7827de4
The change to the prefixed version broke migration from new to old
qemu-server version. This reverts the change and adds a TODO comment for
7.0 to change it to the prefixed version then.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2020-03-20 10:37:33 +01:00
Stefan Reiter
746232eeb1 Die on misaligned memory for hotplugging
...instead of booting with an invalid config once and then silently
changing the memory size for consequent VM starts.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Tested-by: Alwin Antreich <a.antreich@proxmox.com>
2020-03-19 18:55:27 +01:00
Stefan Reiter
456bab5445 Disable memory hotplugging for custom NUMA topologies
This cannot work, since we adjust the 'memory' property of the VM config
on hotplugging, but then the user-defined NUMA topology won't match for
the next start attempt.

Check needs to happen here, since it otherwise fails early with "total
memory for NUMA nodes must be equal to vm static memory".

With this change the error message reflects what is actually happening
and doesn't allow VMs with exactly 1GB of RAM either.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Tested-by: Alwin Antreich <a.antreich@proxmox.com>
2020-03-19 18:54:53 +01:00
Thomas Lamprecht
e8705fc59e vzdump archive_vma: move getting QMPClient nearer to using sites
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:48:41 +01:00
Thomas Lamprecht
d11e91d2de vzdump: indentation and code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:46:49 +01:00
Thomas Lamprecht
0b2f574b4c vzdump: factor out VM powerstate handling and qmp backup cancel
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:41:43 +01:00
Thomas Lamprecht
1ece829a39 vzdump: factor out QGA FS freeze/thaw
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:37:20 +01:00
Thomas Lamprecht
02da0c65bb vzdump: factor out fork_compressor_pipe
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:34:31 +01:00
Thomas Lamprecht
81dcd4799f vzdump: factor out _get_task_devlist
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:32:00 +01:00
Thomas Lamprecht
09eb196b9e vzdump: refactor and cleanup code in query_backup_status_loop
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:26:34 +01:00
Thomas Lamprecht
f301bc0d34 vzdump: use QemuConfig directly to set/unset locks
no need to fork/exec a process for that..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 15:58:43 +01:00
Thomas Lamprecht
cd544fb2b9 vzdump: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 15:58:13 +01:00
Thomas Lamprecht
a113a58a64 vzdump prepare: refactor and code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 11:48:19 +01:00
Fabian Grünbichler
756cd7ea56 vm_start: drop redundant if
since both if and else branch contain identical code

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-03-18 10:08:54 +01:00
Fabian Grünbichler
db1f8b39e1 drive_mirror: rename variables and values
and add some more details to comments.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-03-18 08:21:29 +01:00