5
0
mirror of git://git.proxmox.com/git/qemu-server.git synced 2025-03-08 04:58:26 +03:00

2223 Commits

Author SHA1 Message Date
Stefan Reiter
5d008ad383 Verify VM-specific CPU configs seperately
$cpu_fmt is being reused for custom CPUs as well as VM-specific CPU
settings. The "pve-vm-cpu-conf" format is introduced to verify a config
specifically for use as VM-specific settings.

"pve-cpu-conf" is registered for use in custom CPU API calls (where no
additional checks are required).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-25 18:42:24 +01:00
Stefan Reiter
b3e894883a Adapt CPUConfig to handle custom models
Turn CPUConfig into a SectionConfig with parsing/writing support for
custom CPU models. IO is handled using cfs.

Namespacing will be provided using "custom-" prefix for custom model
names (in VM config only, cpu-models.conf will contain unprefixed
names).

Includes two overrides to avoid writing redundant information to the
config file, additionally get_custom_model is used to retrieve a custom
model configuration by name.

Resolve custom names in print_cpu_device when a custom cpu is passed.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-25 18:42:21 +01:00
Thomas Lamprecht
404c90070d bump version to 6.1-10
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-25 15:16:29 +01:00
Fabian Ebner
47250f03ef Fix calls to get_replicateable_volumes
There is a need to set $noerr, because otherwise migration for a
VM with a non-replicatable volume fails with:
missing replicate feature on volume 'myfs:107/vm-107-disk-2.raw'

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-03-25 14:53:17 +01:00
Thomas Lamprecht
a99b470e75 bump version to 6.1-9
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-25 10:07:23 +01:00
Thomas Lamprecht
6d7450cbec qemu migrate: sort and split module usage
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-25 10:05:58 +01:00
Thomas Lamprecht
28e6e180bc add basic version check for live-migration with replicated disks
as we need at least pve-qemu in 4.2 for this to work, the target side
is implicitly checked with "to old version" check for migrate or the
mirror will fail anyway.

Just use the simple "qemu binary version check", as we could stil
live migrate an older snapshot with older machine versions if both
sides have a recent enough qemu.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-25 10:02:36 +01:00
Fabian Grünbichler
9b6efe436d migrate: add live-migration of replicated disks
with incremental drive-mirror and dirty-bitmap tracking.

1.) get replicated disks that are currently referenced by running VM
2.) add a block-dirty-bitmap to each of them
3.) replicate ALL replicated disks
4.) pass bitmaps from 2) to drive-mirror for disks from 1)
5.) skip replicated disks when cleaning up volumes on either source or
target

added error handling is just removing the bitmaps if an error occurs at
any point after 2, except when the handover to the target node has
already happened, since the bitmaps are cleaned up together with the
source VM in that case.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-24 12:22:32 +01:00
Fabian Grünbichler
b9f44d2773 migrate: add replication info to disk overview
to make migration logs a bit easier to grasp with a quick glance.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Tested-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-24 11:54:32 +01:00
Fabian Grünbichler
bc6c823164 drive-mirror: add support for incremental sync
by re-using a dirty bitmap that represents changes since the divergence
of source and target volume. requires a qemu that supports incremental
drive-mirroring, and will die otherwise.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Tested-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-24 11:54:20 +01:00
Fabian Ebner
43c4c7b693 Add unused description to drivedesc_hash
Moved code so that initialization of drivedesc_hash stays a single block.
Avoid auto-vivication in parse_drive.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-03-23 09:58:30 +01:00
Stefan Reiter
47f35977cb version_guard: early out when major/minor version is high enough
E.g.: If a feature requires 4.1+pveN and we're using machine version 4.2
we don't need to increase the pve version to N (4.2+pve0 is enough).

We check this by doing a min_version call against a non-existant higher
pve-version for the major/minor tuple we want to test for, which can
only work if the major/minor alone is high enough.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-23 09:58:17 +01:00
Thomas Lamprecht
03c8cbd1dc bump version to 6.1-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-20 11:12:26 +01:00
Thomas Lamprecht
1e0074c437 migrate phase3: add to comment why a blockjob cancel is OK here
Clarify why a cancel is actually not really canceling here, because
we're already finished with storage migration and the block jobs are
all in ready state and we (source) are going to stop soon to hand
over to target.

> Note that if you issue 'block-job-cancel' after 'drive-mirror' has
> indicated (via the event BLOCK_JOB_READY) that the source and
> destination are synchronized, then the event triggered by this
> command changes to BLOCK_JOB_COMPLETED, to indicate that the
> mirroring has ended and the destination now has a point-in-time
> copy tied to the time of the cancellation
-- qapi/block-core.json (QEMU 4.2)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-20 11:08:23 +01:00
Mira Limbeck
ff09c795ed revert spice_ticket prefix change in 7827de4
The change to the prefixed version broke migration from new to old
qemu-server version. This reverts the change and adds a TODO comment for
7.0 to change it to the prefixed version then.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2020-03-20 10:37:33 +01:00
Stefan Reiter
746232eeb1 Die on misaligned memory for hotplugging
...instead of booting with an invalid config once and then silently
changing the memory size for consequent VM starts.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Tested-by: Alwin Antreich <a.antreich@proxmox.com>
2020-03-19 18:55:27 +01:00
Stefan Reiter
456bab5445 Disable memory hotplugging for custom NUMA topologies
This cannot work, since we adjust the 'memory' property of the VM config
on hotplugging, but then the user-defined NUMA topology won't match for
the next start attempt.

Check needs to happen here, since it otherwise fails early with "total
memory for NUMA nodes must be equal to vm static memory".

With this change the error message reflects what is actually happening
and doesn't allow VMs with exactly 1GB of RAM either.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Tested-by: Alwin Antreich <a.antreich@proxmox.com>
2020-03-19 18:54:53 +01:00
Thomas Lamprecht
e8705fc59e vzdump archive_vma: move getting QMPClient nearer to using sites
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:48:41 +01:00
Thomas Lamprecht
d11e91d2de vzdump: indentation and code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:46:49 +01:00
Thomas Lamprecht
0b2f574b4c vzdump: factor out VM powerstate handling and qmp backup cancel
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:41:43 +01:00
Thomas Lamprecht
1ece829a39 vzdump: factor out QGA FS freeze/thaw
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:37:20 +01:00
Thomas Lamprecht
02da0c65bb vzdump: factor out fork_compressor_pipe
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:34:31 +01:00
Thomas Lamprecht
81dcd4799f vzdump: factor out _get_task_devlist
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:32:00 +01:00
Thomas Lamprecht
09eb196b9e vzdump: refactor and cleanup code in query_backup_status_loop
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:26:34 +01:00
Thomas Lamprecht
f301bc0d34 vzdump: use QemuConfig directly to set/unset locks
no need to fork/exec a process for that..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 15:58:43 +01:00
Thomas Lamprecht
cd544fb2b9 vzdump: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 15:58:13 +01:00
Thomas Lamprecht
a113a58a64 vzdump prepare: refactor and code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 11:48:19 +01:00
Fabian Grünbichler
756cd7ea56 vm_start: drop redundant if
since both if and else branch contain identical code

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-03-18 10:08:54 +01:00
Fabian Grünbichler
db1f8b39e1 drive_mirror: rename variables and values
and add some more details to comments.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-03-18 08:21:29 +01:00
Mira Limbeck
7827de41a2 add unix socket support for NBD storage migration
The reuse of the tunnel, which we're opening to communicate with the target
node and to forward the unix socket for the state migration, for the NBD unix
socket requires adding support for an array of sockets to forward, not just a
single one. We also have to change the $sock_addr variable to an array
for the cleanup of the socket file as SSH does not remove the file.

To communicate to the target node the support of unix sockets for NBD
storage migration, we're specifying an nbd_protocol_version which is set
to 1. This version is then passed to the target node via STDIN. Because
we don't want to be dependent on the order of arguments being passed
via STDIN, we also prefix the spice ticket with 'spice_ticket: '. The
target side handles both the spice ticket and the nbd protocol version
with a fallback for old source nodes passing the spice ticket without a
prefix.
All arguments are line based and require a newline in between.

When the NBD server on the target node is started with a unix socket, we
get a different line containing all the information required to start
the drive-mirror. This contains the unix socket path used on the target node
which we require for forwarding and cleanup.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2020-03-18 08:03:44 +01:00
Mira Limbeck
c4ac8f713c parse nbd_protocol_version if available
For secure live migration with local disks via NBD over a unix socket,
we have to somehow communicate from the source node to the target node
if it supports it. This is because there can only be one NBD server with
exactly one socket bound.

The source node passes that information via STDIN. Support for
'spice_ticket: (...)' is added in addition to 'nbd_protocol_version:
<version>'. As old source nodes send the spice ticket without a prefix,
we still have to have a fallback for this case. New information should
always be passed via a prefix that is matched, otherwise it will be
recognized as spice ticket.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2020-03-18 08:03:44 +01:00
Mira Limbeck
eb8cddb55e add NBD server unix socket support in vm_start
As the NBD server spawned by qemu can only listen on a single socket,
we're dependent on a version being passed to vm_start that indicates
which protocol can be used, TCP or Unix, by the source node.

The change in socket type (TCP to Unix) comes with a different URI. For
unix sockets it has the form: 'nbd:unix:<path/to/socket>:exportname=<device>'.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2020-03-18 08:03:44 +01:00
Mira Limbeck
e02fb12620 add qemu_drive_mirror_monitor completion modes
With Qemu 4.2 we encountered a problem with unix sockets and SSH socket
forwarding for drive-mirror. It seems the socket gets reopened again and
again after it closes for some reason. This can be worked around by
specifying 'block-job-cancel' instead of 'block-job-complete' when we're
not interested in swapping the disks again from NBD to their original
protocol. This is always the case when we use drive-mirror for live
migrating a VM.

qemu_drive_mirror is used for migration and for clone_disk. All in all
we have 3 cases to handle. Either the 'skip' case which skips the
completion of the job. The 'wait' case which was the default before and
still is when $completion is undefined. And the new 'wait_noswap' case
which is used for the live migration.
If 'wait_noswap' is specified, we issue a 'block-job-cancel' once the block
job is in 'ready' state. This completes the block job without swapping the
disks.

clone_disk always uses 'block-job-cancel' via the qemu_blockjobs_cancel
sub.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2020-03-18 08:03:44 +01:00
Fabian Ebner
758a08eb39 Change format for unused drives
and make it match with what parse_drive does. Even though the 'real' format
was pve-volume-id, callers already expected that parse_drive returns a hash
with a valid 'file' key (e.g. PVE/API2/Qemu.pm:1147ff).

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Reviewed-By: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-03-16 13:30:50 +01:00
Thomas Lamprecht
7c4351f7d4 fix #2580: api/delete: drop VM from HA resources if purge is set
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-12 13:39:00 +01:00
Thomas Lamprecht
7ed8e940f1 d/control: bump versioned dependency for libpve-guest-common-perl (>= 3.0-5)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-12 11:11:58 +01:00
Dietmar Maurer
f6168f1a6a archive_pbs: call backup_cancel only when job is really started 2020-03-11 11:47:24 +01:00
Dietmar Maurer
d2cc2cbe69 archive_vma: call backup_cancel only when job is really started 2020-03-11 11:47:24 +01:00
Dietmar Maurer
e400f5e523 PVE/QemuServer.pm - proxmox backup server restore: use new pbs-restore binary
This is much faster than using a pipe with qemu-img.
2020-03-11 11:47:24 +01:00
Dietmar Maurer
9f3d73bc35 cleanup backup code: share code between restore_vma_archive and restore_proxmox_backup_archive 2020-03-11 11:47:24 +01:00
Dietmar Maurer
d1e92cf622 implement restore from proxmox backup storage 2020-03-11 11:47:24 +01:00
Dietmar Maurer
c5983223b0 add support for proxmox backup 2020-03-11 11:47:24 +01:00
Kamil Trzcinski
148850f693 Append newline to all QGA commands
Since the MacOS Mojave Apple ships AppleQEMUGuestAgent by default.
However, it does not fully adhere to QGA specs as they do expect each
command to be newline delimited.

This makes each command to be newline delimited which is harmless for
all other systems (Windows, Linux), but enable guest agent by default
without any changes on OSX.

Signed-off-by: Kamil Trzcinski <ayufan@ayufan.eu>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
2020-03-11 11:11:45 +01:00
Alexandre Derumier
28e129cccb sdn: use sdn tap_create|plug
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-10 19:08:31 +01:00
Thomas Lamprecht
86a2e85a26 cloudinit: make genisoimage only output errors
avoids a genisoimage output like:
> Total translation table size: 0
> Total rockridge attributes bytes: 417
> Total directory bytes: 0
> Path table size(bytes): 10
> Max brk space used 0
> 178 extents written (0 MB)

on every VM start.

Rather than that useless output, tell genisoimage to be quiet, which
still prints errors but nothing else. Additionally print a short
single line about that we're to create the cloud-init iso.

Reformat while at it.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-10 14:50:39 +01:00
Thomas Lamprecht
37e9b2d031 d/control: bump versioned dependency for libpve-guest-common-perl (>= 3.0-4)
for the moved definedness-safe comparison methods

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-10 14:28:29 +01:00
Thomas Lamprecht
81d6e4e1b8 followup: do not import unused methods
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-10 14:26:29 +01:00
Oguz Bektas
0f1af9e777 QemuServer: use helper functions from GuestHelpers
removes safe_string_ne and safe_num_ne code which is now shared in
GuestHelpers. also change all the calls to use the shared definitions.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-03-10 09:39:15 +01:00
Thomas Lamprecht
00d8327000 bump version to 6.1-7
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-09 19:37:05 +01:00
Fabian Ebner
0ad295f9fb Consistently use format determined in 'PVE::Storage::foreach_volid'
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
LGTM-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-03-09 19:36:58 +01:00