5
0
mirror of git://git.proxmox.com/git/qemu-server.git synced 2024-12-24 21:34:43 +03:00
Commit Graph

3013 Commits

Author SHA1 Message Date
Thomas Lamprecht
c3d151080a cdrom path: code reduction
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-13 13:52:39 +01:00
Stefan Sterz
490b730854 cd rom handling: refactor cd rom path helper function
to stop returning results of assignments

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2023-01-13 13:20:35 +01:00
Stefan Sterz
259470ee41 cd rom handling: return a clearer error when there is no cd rom drive
when a vm is configured to use a physical cd rom drive but there is no
such drive a cryptic "uninitialized value" error is thrown. this is
due to `$path` being undefined in `sub print_drive_commandline_full`.
warn that no cd rom drive is available instead.

note that the error was cosmetic as the vm would start just fine.

forum thread: https://forum.proxmox.com/threads/119592/

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2023-01-13 13:20:35 +01:00
Fiona Ebner
55d0741197 migration: log error from query-migrate upon migration failure
if it is present. Should give more information for some failures and
even if it's not present, that fact helps to narrow things down.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-12 11:12:14 +01:00
Stefan Hanreich
30fdf99cff fix #4358: destroy_vm: Ignore 'suspended' lock when destroying VM
Since we can now differentiate between 'suspended' and 'suspending',
it is possible to ignore the 'suspended' lock when destroying a VM.
It shouldn't matter whether the VM is locked because of hibernation
when you want to remove it. Therefore we can safely ignore the lock.
2023-01-11 10:59:32 +01:00
Fiona Ebner
b3a3e92962 fix #4435: devices list: avoid error for undefined value
When $d->{'pci_bridge'}->{devices} is undef, @-dereferencing it will
die with:
> Can't use an undefined value as an ARRAY reference

This can happen (at least) when the VM is in 'prelaunch' state. The
QAPI definition for '@PciBridgeInfo' also declares the 'devices'
member as optional.

Before commit 721624b ("collect device list for nested pci-bridges"),
there was no issue, because $d->{'pci_bridge'}->{devices} was used in
foreach, so auto-vivified if undef.

Fixes: f721624b ("collect device list for nested pci-bridges")
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-11 10:14:44 +01:00
Alexandre Derumier
62b26624c6 memory hot-plug: fix check for maximal memory value
Fixes: 4d3f29e ("memory hotplug patch v10")
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-02 13:56:43 +01:00
Stefan Hanreich
a46d039d3c rollback: Only create start task with --start if VM is not running
When rolling back to the snapshot of a VM that includes RAM, the VM
gets started by the rollback task anyway, so no additional start task
is needed. Previously, when rolling back with the start parameter and
the VM snapshot included RAM, a start task was created. That task
failed because the VM had already been started by the rollback task.

Additionally documented this behaviour in the description of the start
parameter

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
2022-12-30 15:39:19 +01:00
Fiona Ebner
7bd9abd243 tree-wide: switch to official spelling of QEMU in descriptions/messages
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-12-20 10:26:41 +01:00
Thomas Lamprecht
9cdb3150f0 bump version to 7.3-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-12-16 12:55:00 +01:00
Thomas Lamprecht
3d07669cf1 ovmf cmd assembly: rework now that it's in a separate method
We can now do a few things that would be not really possible, or at
least mess with readability when this was still mostly inline
config2command, shaves of quite a few lines of code.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-12-12 11:43:19 +01:00
Thomas Lamprecht
2ceb59d4b1 ovmf cmd assembly: reorder arguments
in preparation of reworking the new separate method for OVMF cmd
assembly, do this in a separate very targeted commit to make it more
clear that the next reworking-commit doesn't messes with our tests at
all.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-12-12 11:41:50 +01:00
Fiona Ebner
b7d80c7905 cfg2cmd: factor out ovmf drives printing
No functional change is intended.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-12-06 16:56:30 +01:00
Fabian Grünbichler
ad9e347c46 fix #4372: fix vm_resume migration callback
the fix for the recently introduced requirement of loading the VM config while
migrating was incomplete, since the vmlist node value could already be out of
date by the time load_config is called.

extend the fallback behaviour even further, by doing the following sequence:
- try regular load_config (likely case, rename already fully processed)
- if it fails, get node from vmlist, and load_config using that
- it that fails, invalidate the PVE::Cluster cache, retry regular load_config

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-11-30 16:21:39 +01:00
Thomas Lamprecht
a116a7e08e bump version to 7.3-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-21 13:44:02 +01:00
Fabian Grünbichler
a20dc58a1b explain 'nocheck' in more places
was only explained in git history and vm_stop, add comments in other
relevant places to avoid future breakage.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-11-21 13:42:52 +01:00
Fabian Grünbichler
270bfff2e1 vm_resume: fix nocheck/migrate handling
it's not deterministic whether the rename/move of the VM config
triggered on the source side of a migration is already visible on the
target side when vm_resume is executed. check the vmlist for the node
where the config is currently located if $nocheck is set - it is now
needed to add the forwarding DB entries to the bridge.

this fixes an issue on busier or slower clusters, where pmxcfs hasn't
yet processed the rename, and resuming would fail with an error about
the config not existing.

Reported-by: Dominik Csapak <d.csapak@proxmox.com>

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-11-21 13:42:52 +01:00
Thomas Lamprecht
c9a0924df8 bump version to 7.2-12
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-21 08:36:17 +01:00
Fiona Ebner
8481721f22 api: create/update vm: fix clamping cpuunits function calls
When applying the series introducing those calls, the helper was moved
to pve-common's CGroup.pm (see 07c10d5 ("cgroup: move get_cpuunits
helper from qemu-server as clamp_cpu_shares") in pve-common) instead
of pve-guest-common's GuestHelpers.pm. But these calls were not
updated.

Reported in the community forum:
https://forum.proxmox.com/threads/118267

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-11-21 08:27:03 +01:00
Thomas Lamprecht
28768fc317 d/control: depend on newer libpve-guest-common-perl 4.2-3
for new unique tag helper

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-21 08:21:42 +01:00
Thomas Lamprecht
db22080c33 config: ensure that tags are unique
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-21 08:19:33 +01:00
Thomas Lamprecht
eaa3a2b35a bump version to 7.2-11
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-20 16:30:32 +01:00
Thomas Lamprecht
fe62da4f97 fdb: only manage FDB entries for Linux bridges for now
we need to handle OVS setups differently, so for now just ignore it
there (behavior as it was in 7.2)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-20 16:03:45 +01:00
Thomas Lamprecht
f81c9843c3 fdb: pull out bridge variable
no semantic change intended

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-20 16:03:17 +01:00
Thomas Lamprecht
4d44c3e5c6 bump version to 7.2-10
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-17 17:48:06 +01:00
Thomas Lamprecht
305e9cec5d memory hot plug: round down to nearest even phys-bits for heuristics
Mira found out that 41 phys-bits the limit is pretty much the same as
with 40, as such odd sizes are a bit unexpected anyway lets mask the
LSB and use that as base, that way we're good again.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-17 17:45:59 +01:00
Thomas Lamprecht
461d057026 bump version to 7.2-9
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-17 17:12:34 +01:00
Thomas Lamprecht
33b0d3b7be memory hotplug: rework max memory handling, make phys-bits dependent
QEMU 7.1 introduced some actual checks for the max memory value in
1caab5cf86bd ("i386/pc: bounds check phys-bits against max used GPA")
and while correct it breaks our by-luck working hard coded max mem of
4 TB for cases with smaller phys bit address sizes, like some older
CPUs or most CPU types have per default if not 'host' or 'max'.

QEMU uses 40 bits per default if the CPU isn't set to host or
phys-bits is not set explicitly.

For 40 bit it seems that depending on machine type one has a max
possible mem of: i440 -> 752, q35 -> 722 GiB, but instead of reducing
it to 704 GiB (512+1128+64) in a hard coded way we acutally check for
the bit size that will probably be used and use that to determine the
max memory size useable.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-17 16:54:49 +01:00
Thomas Lamprecht
d1901fe2ce cpu config: indentation fixup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-17 16:46:40 +01:00
Thomas Lamprecht
7dbe0cbe26 d/changelog: fixup release
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-17 15:23:26 +01:00
Fabian Grünbichler
192bbfda82 qm: add remote-migrate command
which wraps the remote_migrate_vm API endpoint, but does the
precondition checks that can be done up front itself.

this now just leaves the FP retrieval and target node name lookup to the
sync part of the API endpoint, which should be do-able in <30s ..

an example invocation:

$ qm remote-migrate 1234 4321 'host=123.123.123.123,apitoken=PVEAPIToken=user@pve!incoming=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee,fingerprint=aa:bb:cc:dd:ee:ff:aa:bb:cc:dd:ee:ff:aa:bb:cc:dd:ee:ff:aa:bb:cc:dd:ee:ff:aa:bb:cc:dd:ee:ff:aa:bb' --target-bridge vmbr0 --target-storage zfs-a:rbd-b,nfs-c:dir-d,zfs-e --online

will migrate the local VM 1234 to the host 123.123.1232.123 using the
given API token, mapping the VMID to 4321 on the target cluster, all its
virtual NICs to the target vm bridge 'vmbr0', any volumes on storage
zfs-a to storage rbd-b, any on storage nfs-c to storage dir-d, and any
other volumes to storage zfs-e. the source VM will be stopped but remain
on the source node/cluster after the migration has finished.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-11-17 15:21:39 +01:00
Fabian Grünbichler
06fedff675 api: add remote migrate endpoint
entry point for the remote migration on the source side, mainly
preparing the API client that gets passed to the actual migration code
and doing some parameter parsing.

querying of the remote sides resources (like available storages, free
VMIDs, lookup of endpoint details for specific nodes, ...) should be
done by the client - see next commit with CLI example.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-11-17 15:21:39 +01:00
Fabian Grünbichler
eef93bc590 migrate: add remote migration handling
remote migration uses a websocket connection to a task worker running on
the target node instead of commands via SSH to control the migration.
this websocket tunnel is started earlier than the SSH tunnel, and allows
adding UNIX-socket forwarding over additional websocket connections
on-demand.

the main differences to regular intra-cluster migration are:
- source VM config and disks are only removed upon request via --delete
- shared storages are treated like local storages, since we can't
assume they are shared across clusters (with potentical to extend this
by marking storages as shared)
- NBD migrated disks are explicitly pre-allocated on the target node via
tunnel command before starting the target VM instance
- in addition to storages, network bridges and the VMID itself is
transformed via a user defined mapping
- all commands and migration data streams are sent via a WS tunnel proxy
- pending changes and snapshots are discarded on the target side (for
  the time being)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-11-17 15:21:39 +01:00
Fabian Grünbichler
05b2a4ae9c migrate: refactor remote VM/tunnel start
no semantic changes intended, except for:
- no longer passing the main migration UNIX socket to SSH twice for
forwarding
- dropping the 'unix:' prefix in start_remote_tunnel's timeout error message

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-11-17 15:21:39 +01:00
Fabian Grünbichler
347dc13650 mtunnel: add API endpoints
the following two endpoints are used for migration on the remote side

POST /nodes/NODE/qemu/VMID/mtunnel

which creates and locks an empty VM config, and spawns the main qmtunnel
worker which binds to a VM-specific UNIX socket.

this worker handles JSON-encoded migration commands coming in via this
UNIX socket:
- config (set target VM config)
-- checks permissions for updating config
-- strips pending changes and snapshots
-- sets (optional) firewall config
- disk (allocate disk for NBD migration)
-- checks permission for target storage
-- returns drive string for allocated volume
- disk-import, query-disk-import, bwlimit
-- handled by PVE::StorageTunnel
- start (returning migration info)
- fstrim (via agent)
- ticket (creates a ticket for a WS connection to a specific socket)
- resume
- stop
- nbdstop
- unlock
- quit (+ cleanup)

this worker serves as a replacement for both 'qm mtunnel' and various
manual calls via SSH. the API call will return a ticket valid for
connecting to the worker's UNIX socket via a websocket connection.

GET+WebSocket upgrade /nodes/NODE/qemu/VMID/mtunnelwebsocket

gets called for connecting to a UNIX socket via websocket forwarding,
i.e. once for the main command mtunnel, and once each for the memory
migration and each NBD drive-mirror/storage migration.

access is guarded by a short-lived ticket binding the authenticated user
to the socket path. such tickets can be requested over the main mtunnel,
which keeps track of socket paths currently used by that
mtunnel/migration instance.

each command handler should check privileges for the requested action if
necessary.

both mtunnel and mtunnelwebsocket endpoints are not proxied, the
client/caller is responsible for ensuring the passed 'node' parameter
and the endpoint handling the call are matching.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-11-17 15:21:39 +01:00
Fabian Grünbichler
7a24c98af6 pending: fix typo in variable name
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-11-17 15:21:39 +01:00
Fabian Grünbichler
e97bbbb64d pending changes: allow skipping cloud-init
in case of remote migration, we use the `update_vm_api` helper for
checking permissions on the incoming config. this would also cause an
incoming cloud-init image to be overwritten, since the VM is not running
yet at this point.

provide a parameter which can be set by an incoming *remote* migration
to avoid having inconsistent cloud init images on the source and target
side.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-11-17 15:21:39 +01:00
Wolfgang Bumiller
9660e606ee fixup delayed cloudinit hotplug
cloudinit generation needs to see the cloudinit drive so we
need to pass a config with it already updated

Fixes: 4b785da1a9 ("delay cloudinit generation in hotplug")
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-17 15:16:21 +01:00
Wolfgang Bumiller
a540985120 rework cloudinit_record_changed logic
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-17 14:35:27 +01:00
Thomas Lamprecht
c229961ad3 parse config: do not validate informative values in cloud init section
Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-17 13:19:19 +01:00
Thomas Lamprecht
7e554ab102 d/control: bump versioned dependency for guest-common
to ensure the tag helper is available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-17 13:19:16 +01:00
Dominik Csapak
97c760399e api: update: check for tags permissions with 'assert_tag_permissions'
from GuestHelpers. This function checks all necessary permissions and
raises an exception if the user does not have the correct ones.

This is necessary for the new 'privileged' tags and 'user-tag-access'
permissions to work.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-17 10:26:26 +01:00
Leo Nunner
f5a88e9870 fix #4321: properly check cloud-init drive permissions
The process for editing Cloud-init drives checked for inconsistent
permissions: for adding, the VM.Config.Disk permission was needed, while
the VM.Config.CDROM permission was needed to remove a drive. The regex
in drive_is_cloudinit needed to be adapted since the drive names have
different formats before/after they are actually generated.

Due to the regex letting names fall through before, Cloud-init drives
were being checked as disks, even though they are actually treated as
CDROM drives. Due to this, it makes more sense to check for
VM.Config.CDROM instead, while also requiring VM.Config.Cloudinit, since
generating a Cloud-init drive already generates default values that are
passed to the VM.

Signed-off-by: Leo Nunner <l.nunner@proxmox.com>
2022-11-17 08:10:28 +01:00
Thomas Lamprecht
0a1c4503e5 bump version to 7.2-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-16 18:31:16 +01:00
Wolfgang Bumiller
1b5706cd16 drop get_pending_changes and simplify cloudinit_pending api call
- The forced-remove flag wasn't really used AFAICT and makes
  no sense IMO.
- Whether or not we care about non-MAC changes does not
  belong here, but should instead taken into account in the
  actual hotplug path recording the cloud-init state (iow.
  into $cloudinit_record_changed().)
  (This is not done here atm.)
- It seems much simpler to just have:
  * 'old' = the old value if it's not a new value
  * 'new' = the new value unless it's being deleted
  * If only one of them is set it's an addition or removal.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-16 18:17:07 +01:00
Wolfgang Bumiller
d29483147d don't call 'cleanup_config' the cloudinit section
It performs schema valdiation (and normalization).

We only ever write values into it which came from an
already validated config, and we also add an additional
"added" key which is not covered by the schema, so this
would fail.

Simply skip it.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-16 18:17:07 +01:00
Wolfgang Bumiller
f16cf6c37d record cloud-init changes in the cloudinit section
introducing an 'added' value in the cloudinit section for
values which have not been present when the cloudinit image
has been generated

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-16 18:17:07 +01:00
Wolfgang Bumiller
4b785da1a9 delay cloudinit generation in hotplug
Hotpluggieg generated a cloudinit image based on old values
in order to attach the device and later update it again, but
the update was only done if cloudinit hotplug was enabled.
This is weird, let's not.

Also introduce 'apply_cloudinit_config' which also write the
config, which, as it turns out, is the only thing we
actually need anyway, currently.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-16 18:17:07 +01:00
Wolfgang Bumiller
0337d531a0 Partially-revert "cloudinit: add cloudinit section for current generated config"
This partially reverts commit 95a5135dad.
Particularly the unprotected write to the config when
generating the cloudinit file. We leave the rest as is for
now and update the callers to deal with the config later.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-16 18:17:07 +01:00
Wolfgang Bumiller
3de134ef4a Revert "cloudinit: avoid unsafe write of VM config"
This reverts commit b137c30c3a.

In preparation of fixing the special:cloudinig section.
2022-11-16 18:16:56 +01:00