5
0
mirror of git://git.proxmox.com/git/pve-storage.git synced 2024-12-22 13:34:16 +03:00
Commit Graph

1761 Commits

Author SHA1 Message Date
Fabian Ebner
21a75847a8 api: disk: work around udev bug to ensure its database is updated
There is a udev bug [0] which can ultimately lead to the udev database
for certain devices not being actively updated. Determining whether a
disk is used or not in get_disks() (in part) relies upon lsblk, which
queries the udev database. Ensure the information is updated by
manually calling 'udevadm trigger' for the changed devices.

It's most important for the 'directory' API path, as mounting depends
on the '/dev/disk/by-uuid'-symlink to be generated.

[0]: https://github.com/systemd/systemd/issues/18525

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-09-30 18:04:25 +02:00
Fabian Ebner
e99bc248d4 api: disks: create: re-check disk after fork/lock
Because then it might not be unused anymore. If there really is a
race, this prevents e.g. sgdisk creating a partition on a device
already in use by LVM or LVM destroying a partitioned device.

For ZFS, also get the latest udev info once inside the worker.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-09-30 18:04:22 +02:00
Fabian Ebner
dcd8f3a3dd btrfs: call free_image correctly
Currently, 'PVE::Storage::DirPlugin' is implicitly passed along as
$class, which means that if the base class's free_image calls another
method (e.g.  filesystem_path) then the DirPlugin's method will be
used, rather than the one from BTRFSPlugin. Change it so that $class
itself is passed along.

See also commit 279d9de510 for context,
where the approach in this patch was suggested.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-09-22 08:52:53 +02:00
Thomas Lamprecht
e1667a2253 cifs: negotiates the highest SMB2+ version supported by default
instead of hardcoding it to a potential outdated value.

For `smbclient` we only set max-protocol version and that could only
be smb2 or smb3 (no finer granularity) any how, so this was not
really correct.

Nowadays the kernel dropped SMB1 and tries to go for SMB2.1 or higher
by default, depending on what client and server supports. SMB2.1 is
Windows 7/2008R2 - both EOL since quite a bit, so ok as default lower
boundary.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-15 10:59:00 +02:00
Thomas Lamprecht
9fff8c7aca cifs: allow "3" and "default" for version
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-14 14:28:15 +02:00
Moayad Almalat
396ea58b65 fix #3609: cifs: add support to SMB 3.11
Added support for the SMB version SMB3_11 When the `min protocol =
SMB3_11` in the smb.conf, the CIFS mount will return with the
following error:
```
CIFS VFS: cifs_mount failed w/return code = -95
```
added an optional option to use the `vers=3.11`

Signed-off-by: Moayad Almalat <m.almalat@proxmox.com>
Tested-by: Fabian Ebner <f.ebner@proxmox.com>
[ Thomas: move text from cover letter to commit message &
  add S-o-b ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-14 11:50:14 +02:00
Fabian Ebner
576e143ac1 fix #3610: properly build ZFS detail tree
Previously, top-level vdevs like log or special were wrongly added as
children of the previous outer vdev instead of the root.

Fix it by also showing the vdev with the same name as the pool and
start counting from level 1 (the pool itself serves as the root and
should be the only one with level 0). This results in the same kind
of structure as in PBS and (except for the root) zpool status itself.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-09-10 14:19:39 +02:00
Fabian Ebner
d5fc368503 fix prune-backups validation (again)
Commit a000e26ce7 caused a test failure
in pve-manager, because now 'keep-all=0' is not thrown out upon
validation anymore. Fix the issue the commit addressed differently,
by simply creating a copy of the (shallow) hash first, and using
the logic from before the commit.

Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-09-10 14:16:26 +02:00
Fabian Ebner
a000e26ce7 prune {validate, mark}: preserve input parameter
While the current way to detect settings like { 'keep-last' => 0 } is
concise, it's also wrong, because but the delete operation is visible
to the caller. This resulted in e.g.
    # $hash is { 'keep-all' => 1 }
    my $s = print_property_string($hash, 'prune-backups');
    # $hash is now {}, $s is 'keep-all=1'
because validation is called in print_property_string. The same issue
is present when calling prune_mark_backup_group.

Because validation complains when keep-all and something else is set,
this shouldn't have caused any real issues, besides vzdump with
keep-all=1 wrongly taking the removal path, but without any settings,
so not removing anything:
    INFO: prune older backups with retention:
    INFO: pruned 0 backup(s)

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-09-09 18:07:13 +02:00
Fabian Ebner
08ca395503 btrfs: style: add missing semicolon
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-09-08 14:26:12 +02:00
Fabian Ebner
6c315e4587 btrfs: avoid undef warnings with format
which is only set by parse_volname when the volume is a VM or
container image, but not for other content types.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-09-08 14:26:12 +02:00
Thomas Lamprecht
4aecade4dc bump version to 7.0-11
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-06 08:40:39 +02:00
Thomas Lamprecht
3de1c50a89 api: followup style/comment improvements
try to comment why not what, what is already described good enough by
the code here.

Also, we want to go up to 100cc text-width if it improves
readability, which for post-if's it most often does.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-06 08:32:20 +02:00
Lorenz Stechauner
f6aeefff54 status: move unlink from http-server to enpoint
this is the first step in which not the http server removes the
temporary file, but the worker itself.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-09-06 08:28:26 +02:00
Thomas Lamprecht
e2f8e86c83 btrfs: fix calling alloc_image from DirPlugin
similar to commit 279d9de510

This calling style is pretty dangerous in general for such plugin
systems...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-06 08:25:58 +02:00
Aaron Lauterer
22b68016f7 Ceph: add keyring parameter for external clusters
By adding the keyring for RBD storage or the secret for CephFS ones, it
is possible to add an external Ceph cluster with only one API call.

Previously the keyring / secret file needed to be placed in
/etc/pve/priv/ceph/$storeID.{keyring,secret} manually.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2021-08-26 18:15:30 +02:00
Aaron Lauterer
a4a1fe6419 CephConfig: add optional $secret parameter
This allows us to manually pass the used RBD keyring or CephFS secret.
Useful mostly when adding external Ceph clusters where we have no other
means to fetch them.

I renamed the previous $secret to $cephfs_secret to be able to use
$secret as parameter.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2021-08-26 18:15:30 +02:00
Fabian Ebner
ab3516a6d7 zfs: fix unmount request
by not dying when the dataset is already unmounted. Can be triggered
for a container by doing two rollbacks in a row.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-08-12 11:48:42 +02:00
Thomas Lamprecht
04bb277db4 buildsys: change upload dist to bullseye
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-30 15:37:13 +02:00
Thomas Lamprecht
c0a4b3d127 bump version to 7.0-10
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-30 15:23:19 +02:00
Dominik Csapak
ae098a191c api: disks: allow zstd compression for zfs pools
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-30 15:21:01 +02:00
Hannes Laimer
279d9de510 fix #3555: BTRFS: call DirPlugin's free_image correctly
The method is only derived in the DirPlugin module from the base
Plugin, so we do not have it available there through a static module
method call using ::, but only when using a class dereference.

Other fix options would have been:

  PVE::Storage::Plugin::free_image(@_);

or:
  $class->SUPER::free_image($storeid, ...);

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
[ Thomas: add some background to the commit message ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-30 14:52:51 +02:00
Thomas Lamprecht
b11d054be0 api: status: rework err cleanup
avoid open3 on local node.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-29 17:12:40 +02:00
Thomas Lamprecht
a84804c432 api: status: import run_command
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-29 17:12:40 +02:00
Lorenz Stechauner
31bd43648d api: status: fix unlink on file upload
after an error while copying the file to its destination the local
path of the destination was unlinked in every case, even when on the
destination was copied to via scp.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-07-29 17:12:40 +02:00
Thomas Lamprecht
0a65c237ea api: status: sort/cleanup module use-statements
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-29 17:12:40 +02:00
Lorenz Stechauner
ca8c865871 status: add max length and disclaimer to file upload's filename
Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-07-21 16:47:01 +02:00
Lorenz Stechauner
1e96ffc954 status: add enum for file upload content type
the addition of this enum does not change API behaviour, because
it is checked for 'iso' or 'vztmpl' aftwerwards anyway.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-07-21 16:47:01 +02:00
Thomas Lamprecht
24adf51e7e bump version to 7.0-9
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-06 16:28:54 +02:00
Fabian Ebner
576e560d17 extract backup config: less precise matching for broken pipe detection
Extracting the config for zstd compressed vma files was broken:
    Failed to extract config from VMA archive: zstd: error 70 : Write
    error : cannot write decoded block : Broken pipe (500)
since the error message changed and wouldn't match anymore.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-07-06 16:24:22 +02:00
Thomas Lamprecht
49a2f518bc bump version to 7.0-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-06 12:41:42 +02:00
Stoiko Ivanov
5a16629577 lvm: wipe signatures on lvcreate
With PVE 7.0 we use upstream's lvm2 packages, which seem to detect
'more' signatures (and refuse creating lvs when they are present)

This prevents creating new disks on LVM (thick) storages as reported
on pve-user [0].

Adding -Wy to wipe signatures, and --yes (to actually wipe them
instead of prompting) fixes the aborted lvcreate.

Adding only to LVMPlugin and not to the lvcreate calls in
LvmThinPlugin, since I assume (and my quick tests confirm) that thin
pools are not affected by this issue..

Tested on a virtual test-setup with a LVM storage on a (virtual) iscsi
target and a local lvmthin storage.

[0] https://lists.proxmox.com/pipermail/pve-user/2021-July/172660.html

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-07-06 12:39:50 +02:00
Thomas Lamprecht
b4e88b7fd3 cifs: improve warning for password but no username set
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-06 07:50:29 +02:00
Thomas Lamprecht
02f43ab4a8 cifs: fix sensitive parameter name for on-update/add
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-06 07:50:06 +02:00
Thomas Lamprecht
54e315e91b bump version to 7.0-7
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-05 19:01:15 +02:00
Wolfgang Bumiller
38f0f4698e btrfs: fix path_is_mounted invocation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-06-28 08:40:03 +02:00
Thomas Lamprecht
3577266e39 bump version to 7.0-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-24 16:13:12 +02:00
Thomas Lamprecht
1c1589e60d btrfs: support newer prune-backups for backup retention
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-24 16:11:48 +02:00
Thomas Lamprecht
f31143701d bump version to 7.0-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-24 11:45:45 +02:00
Thomas Lamprecht
a1234a04df btrfs: add mkdir as option for now
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-24 11:45:45 +02:00
Thomas Lamprecht
f449cddc79 btrfs: do not reuse DirPlugins activate_storage directrly
as then the btrfs assertion would happen after we already created
subdirectories on some path, leaving those left-over..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-24 11:18:40 +02:00
Wolfgang Bumiller
f6abd82a6d btrfs: check for btrfs in on_add_hook and activate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-06-24 11:06:02 +02:00
Thomas Lamprecht
5d65bf5f12 bump version to 7.0-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 22:51:02 +02:00
Thomas Lamprecht
fc015f3d9c api: file download: limit filename length and encode it for the worker id
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 22:40:40 +02:00
Thomas Lamprecht
d0a3db1bf5 api: file download: code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 22:40:05 +02:00
Lorenz Stechauner
837b194224 status: add download_url method
uses common function PVE::Tools::download_file_from_url to download
iso files.

Only users with permissions `Sys.Audit` and `Sys.Modify` on `/` are
permitted to perform this action. This restriction is due to the
fact, that the download function is able to download files from
internal networks (which are not visible/accessible from outside).
Users with these permissions anyway have the means to alter node
(network) config, so this does not create any further security risk.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-06-23 22:28:44 +02:00
Lorenz Stechauner
edda43ed4f status: factoring out normalize_content_filename
Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-06-23 22:28:44 +02:00
Fabian Ebner
0153334270 api: content: correctly handle warnings status for delayed task
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 22:21:55 +02:00
Thomas Lamprecht
347e677b78 btrfs: drop qcow2 and vmdk for now
the web-interface always prefers qcow2 once that is in the list,
itself a bug on it's own as the preferred one from the backend should
be preferred too, but still, vmdk support should not be extended we
can only cope with that in a limited way anyway, and both can always
get enabled later easily, if there's actual user-request for it.
Disabling is never that easy, at least if one cares about backward
compat.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 20:22:52 +02:00
Wolfgang Bumiller
d3c5cf2487 btrfs: make NOCOW optional
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-06-23 20:20:31 +02:00