5
0
mirror of git://git.proxmox.com/git/pve-guest-common.git synced 2025-03-10 12:58:19 +03:00

359 Commits

Author SHA1 Message Date
Thomas Lamprecht
ea022f01c3 abstract migrate: code-style cleanup get_bwlimit
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2025-02-10 16:48:36 +01:00
Fiona Ebner
8d62b1e563 fix #6130: remote migration: untaint bandwidth limit from remote
Remote migration via API will be invoked under Perl's '-T' switch to
detect tainted input used in commands. For remote migration, the
bandwidth limit from the remote side would be such tainted input. This
would lead to failure for offline disk migration when the target
node's bandwidth limit is stricter when invoking the 'pvesm export'
command:

> command 'set -o pipefail && pvesm export rbd:vm-400-disk-0 \
> raw+size - -with-snapshots 0 | /usr/bin/cstream -t 307232768' \
> failed: Insecure dependency in exec while running with -T switch

Untaint the value to fix the issue. Note that the schema for the
bandwidth limits in datacenter.cfg and storage.cfg allows fractional
values.

Avoid re-using the same variable for both, the reply from the remote
(which is a hash) and the actual remote bandwidth limit. This also
makes it possible to use the "assign regex match or die" pattern while
accessing the original value in the error message.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-02-10 16:40:38 +01:00
Thomas Lamprecht
4c2dd7c226 bump version to 5.1.6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-11-18 08:38:44 +01:00
Thomas Lamprecht
9d15995df3 vzdump common: fix missing comma
Fixes: 5118fac ("vzdump: change-detection: fix description and drop experimental note")
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-11-18 08:37:47 +01:00
Thomas Lamprecht
ea43284837 bump version to 5.1.5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-11-17 18:33:49 +01:00
Thomas Lamprecht
5118facafd vzdump: change-detection: fix description and drop experimental note
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-11-17 18:32:54 +01:00
Markus Frank
e476bfe4df AbstractConfig: add abstract method to check for resources preventing a snapshot
Signed-off-by: Markus Frank <m.frank@proxmox.com>
Suggested-by: Fiona Ebner <f.ebner@proxmox.com>
2024-11-17 18:08:17 +01:00
Thomas Lamprecht
23ef2fb414 vzdump: avoid declaring whole PBS change-detection mode as experimental
As the legacy option is the long-time and stable default.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-07-31 14:35:46 +02:00
Thomas Lamprecht
58923bbb7a bump version to 5.1.4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-07-22 19:18:12 +02:00
Lukas Wagner
1489970288 vzdump: common: allow 'job-id' as a parameter without being in schema
'job-id' is passed when a backup as started as a job and will be
passed to the notification system as matchable metadata. It
can be considered 'internal'.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Max Carrara <m.carrara@proxmox.com>
2024-07-22 19:11:29 +02:00
Dominik Csapak
c29c6fc736 mapping: pci: make sure all desired properties are checked
by placing all expected properties from the hardware into an 'expected_props'
and those fromt he config into 'configured_props'

the names makes clearer what's what, and we can easily extend it, even
if the data does not come from the mapping (like we'll do with 'mdev')

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-07-05 10:22:11 +02:00
Dominik Csapak
8052f6f96f mapping: pci: assert_valid: reword error messages
makes them a bit clearer

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-07-05 10:22:11 +02:00
Dominik Csapak
903d06de13 mapping: pci: assert_valid: rename cfg to mapping
to make it clearer what it actually is. Also we want to add the
'real' config as parameter too, and so it's less confusing.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-07-05 10:22:11 +02:00
Fabian Grünbichler
a9604f72eb bump version to 5.1.3
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-10 13:59:34 +02:00
Christian Ebner
35d0e15e84 vzdump: add PBS change detection mode configuration
Define the possible modes to be configured in a PBS file change
detection mode.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-10 13:02:52 +02:00
Thomas Lamprecht
6ca6ff2365 bump version to 5.1.2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-05-15 19:23:28 +02:00
Fabian Grünbichler
172ab9f5cc storage tunnel: correctly set disk-import parameters
according to the schema, else some combinations of migration / guest /
storage settings will fail validation:

2024-05-15 11:48:51 ERROR: migration_snapshot: type check ('boolean') failed - got ''

since this is client / source side, remote migrations to a remote node
with validation enabled can fail without this change.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-05-15 19:21:10 +02:00
Fabian Grünbichler
5dbf4609bd storage tunnel: fix schema definitions
to avoid breakage with schema validation turned on.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-05-15 19:21:10 +02:00
Thomas Lamprecht
ff2208849c bump version to 5.1.1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-18 10:20:23 +02:00
Fiona Ebner
45c3553562 replication: snapshot cleanup: only attempt to remove snapshots that exist
Since commit a6f5b35 ("replication: prepare: include volumes without
snapshots in the result"), attempts would be made to remove previous
replication snapshots from volumes on which they didn't exist. This
was noticed by Thomas since the output of a replication test in
pve-manager changed.

The issue is not completely new, i.e. there was no check that the
(previous) replication snapshot acutally exists before attempting
removal during the cleanup phase. Fix the issue by adding such a
check.

The $replicate_snapshots hash is only used for this, so the change
there is fine.

Fixes: a6f5b35 ("replication: prepare: include volumes without snapshots in the result")
Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-18 10:19:59 +02:00
Thomas Lamprecht
bfc998e0f0 bump version to 5.1.0
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-17 17:39:52 +02:00
Thomas Lamprecht
1d06c1bf95 guest helpers: avoid checking user/token if one can abort all tasks
If the user can already stop all tasks there is no point in spending
some work on every task to check if the user could also stop if
without those powerful permissions.

To avoid to much indentation rework the filter to an early-next style.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-17 17:30:53 +02:00
Friedrich Weber
253a2ea93d guest helpers: add helper to abort active guest tasks of a certain type
Given a `(type, user, vmid)` tuple, the helper aborts all tasks of the
given `type` for guest `vmid` that `user` is allowed to abort:

- If `user` has `Sys.Modify` on the node, they can abort any task
- If `user` is an API token, it can abort any task it started itself
- If `user` is a user, they can abort any task started by themselves
  or one of their API tokens.

The helper is used to overrule any active qmshutdown/vzshutdown tasks
when attempting to stop a VM/CT (if requested).

Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
2024-04-17 17:24:11 +02:00
Fiona Ebner
58c48f3ef0 vzdump: schema: make storage for fleecing semi-optional
so it doesn't need to be set when explicitly disabling fleecing. Needs
a custom verifier to enforce it being set when enabled.

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-11 19:13:29 +02:00
Fiona Ebner
ceb527c870 vzdump: schema: add fleecing property string
It's a property string, because that avoids having an implicit
"enabled" as part of a 'fleecing-storage' property. And there likely
will be more options in the future, e.g. threshold/limit for the
fleecing image size.

Storage is non-optional, so the storage choice needs to be a conscious
decision. Can allow for a default later, when a good choice can be
made further down the stack. The original idea with "same storage as
VM disk" is not great, because e.g. for LVM, it would require the same
size as the disk up front.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
 [ TL: style fix for whitespace placement in multi-line strings ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-11 19:08:39 +02:00
Dominik Csapak
bdf06beb0a mapping: pci: fix missing description/default for mdev
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-04-11 18:20:17 +02:00
Fiona Ebner
5e1f2f38d2 abstract config: fix snapshot needed by replication check
Do not pass the cleanup flag to get_replicatable_volumes() which leads
to replicatable volumes that have the replicate setting turned off to
be part of the result.

Instead pass the noerr flag, because things like missing the
storage-level replicate feature should not lead to an error here.

Reported in the community forum:
https://forum.proxmox.com/threads/120910/post-605574

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-11 18:13:44 +02:00
Fiona Ebner
f657b6abeb replication: find common base: improve error when no common base snapshot exists
Suggest an alternative solution by removing the problematic volumes
from the replication target rather than the whole job.

This is helpful if there are multiple replicated volumes to avoid the
need to fully re-sync all volumes in many cases.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-11 18:13:44 +02:00
Fiona Ebner
a6f5b35851 replication: prepare: include volumes without snapshots in the result
Note that PVE::Storage::volume_snapshot_info() will fail when a volume
does not exist, so no non-existing volume will end up in the result
(prepare() is only called with volumes that should exist).

This makes it possible to detect a volume without snapshots in the
result of prepare(), and as a consequence, replication will now also
fail early in a situation where source and remote volume both exist,
but (at least) one of them doesn't have any snapshots.

Such a situation can happen, for example, by deleting and re-creating
a volume with the same name on the source side without running
replication after deletion.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-11 18:13:44 +02:00
Thomas Lamprecht
f28a4d1398 bump version to 5.0.6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-21 13:31:14 +01:00
Thomas Lamprecht
06c318c45e vzdump: fixup style for multi-line literal strings
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-21 13:29:43 +01:00
Lukas Wagner
e5278e9827 vzdump: config: add 'notification-mode' param for backup jobs
'legacy-sendmail': Use mailto/mailnotification parameters and send
emails directly.
'notification-system': Always notify via notification system
'auto': Notify via mail if mailto is set, otherwise use notification
system.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-11-21 13:23:12 +01:00
Lukas Wagner
53bee4f466 vzdump: deprecate mailto/mailnotification/notification-{target, policy}
The first two will be migrated to the notification system, the second
were part for the first attempt for the new notification system.
The first attempt only ever hit pvetest, so we simply tell the user
to not use the two params.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-11-17 14:59:15 +01:00
Thomas Lamprecht
16bf01217c bump version to 5.0.5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-09-18 16:47:40 +02:00
Alexander Zeidler
9234d8a1fe fix #3069: vzdump: schema: add 'pbs-entries-max' property
configuring pbs-entries-max can avoid failing backups due to a high
amount of files in folders where a folder exclusion is not possible

Signed-off-by: Alexander Zeidler <a.zeidler@proxmox.com>
2023-09-18 16:46:40 +02:00
Fiona Ebner
018565379e replication: avoid passing removed storages to target
After removing a storage, replication states can still contain
references to it, even if no volume references it anymore.

If a storage does not exist in the storage configuration, the
replication target runs into an error when preparing the job locally.
This error prevents both running and removing the replication job. Fix
it by not passing the invalid storage ID in the first place.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-08-30 11:52:56 +02:00
Wolfgang Bumiller
8a81208e4d bump version to 5.0.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-07-24 15:22:15 +02:00
Lukas Wagner
21ec073055 vzdump: add config options for new notification backend
- Add new option 'notification-target'
  Allows to select to which endpoint/group notifications shall be sent
- Add new option 'notification-policy'
  Replacement for the now deprecated 'mailnotification' option. Mostly
  just a rename for consistency, but also adds the 'never' option.
- Mark 'mailnotification' as deprecated in favor of 'notification-policy'
- Clarify that 'mailto' is ignored if 'notification-target' is set

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-07-24 15:15:07 +02:00
Thomas Lamprecht
1f9d736721 vzdump: use worker aware log_warn from rest environment for warn level
This ensures that the alert counter is incremented when a message
with such a level is logged, and that the task is prominently marked
in the web UI task log.

The log_warn produces the exact same message format for the warn
level, so we can just swap printing to STDERR for the warning level
without any change to the resulting text in the log. Keep printing to
the (on storage saved) backup log-fd as is.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-07-05 16:21:21 +02:00
Thomas Lamprecht
af536a40f3 vzdump: always lower-case level in message logging base method
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-07-05 16:21:21 +02:00
Wolfgang Bumiller
831a2fffb2 minor style and doc fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-13 10:49:17 +02:00
Thomas Lamprecht
94b4883904 bump version to 5.0.3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-07 19:35:25 +02:00
Fiona Ebner
56f09314cb abstract config: add method to calculate derived properties from a config
HA manager currently needs to know about internal details about the
configs and how the properties are calculated. With this method, those
details are abstracted away, allowing to change the configuration
structure. In particular, QemuConfig's 'memory' can be turned into
a property string without HA manager needing to know about it (once HA
manager switched to using this mehtod).

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 19:35:04 +02:00
Thomas Lamprecht
f9c40120f1 bump version to 5.0.2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-07 19:21:15 +02:00
Fiona Ebner
29b4ce9390 vzdump: config: improve description of ionice setting
The CFQ scheduler was removed with Linux 5.0 and ionice is now used
by the newer BFQ scheduler. Mention what the special value 8 does.
Also mention that for snapshot and suspend mode backups of VMs, the
setting only affects the compressor, because the kvm process is not a
child process of vzdump then and does not inherit the ionice priority.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 19:18:42 +02:00
Thomas Lamprecht
59b336314f mappings: cleanup imports
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-07 19:16:02 +02:00
Dominik Csapak
11fa95682e add PCI/USB Mapping configs
adds a config file for each type of resource (usb/pci) by using a 'map'
array propertystring for each node mapping

in each mapping we save the path(s) and some other information to detect
hardware changes (if possible) like the vendor/device id

both configs have custom header parser/formatter to omit the type (since
we only want one type per config here)

also each config has some helpers like find_on_current_node

the resulting config (e.g. for pci) would look like this:
2023-06-07 19:14:38 +02:00
Alexandre Derumier
aabf879e0c helpers : add check_vnet_access
if a tag is defined, test if user have a specific access to the vlan (or propagate from full bridge acl or zone)
if trunks is defined, we check permissions for each vlan of the trunks
if no tag, test if user have access to full bridge.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>

FG:
- conditionalize check for bridge
- make trunk to tags helper private for now

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-06-07 16:47:31 +02:00
Wolfgang Bumiller
d8777f5653 bump version to 5.0.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-07 13:28:49 +02:00
Wolfgang Bumiller
50e7b5d656 break libpve-http-server-perl << 5.0.2 and pve-manager << 8.0.0~7
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-07 13:28:32 +02:00