5
0
mirror of git://git.proxmox.com/git/pve-storage.git synced 2024-12-22 13:34:16 +03:00
Commit Graph

1867 Commits

Author SHA1 Message Date
Thomas Lamprecht
2deca27032 disk api: avoid using unrelated Datastore.Audit priv for disk management
Local disk and storage creation and listing is something rather
different than the Proxmox VE storage client ABI that provides an
abstract access to a variety of storage types, specifically targeted
to virtual guests images, templates and backups.

The Datastore.* privilege group is specifically made for auditing the
abstract configuration, here the name must be interpreted in context
and not just assumed that due to "datastore" sounding like it could
have to do something with disks or creation of local storage it just
must be a good fit.

Luckily, Sys.Audit was already used too, which is the correct one
here, this is for node specific (HW) details, not some config for
accessing datastore in a restricted way.

This is a step in splitting the disk manage code out of the
pve-storage package, and maybe even repository.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-17 14:57:20 +02:00
Thomas Lamprecht
b89854ad46 disk management api: comment how storages are added and cleanup style
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-17 14:57:20 +02:00
Thomas Lamprecht
c61e609eb6 fix #4785: avoid overly noisy mkdir deprecation warning for now
Move the warning over to creating and updating storage configs, which
is much less noisy as the constantly called activate storage (e.g.,
pvestatd).

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-17 13:05:28 +02:00
Friedrich Weber
09f1f847ae content-dirs check: silently skip paths that cannot be resolved
Since commit 8e623a2930, the inequality
check for content-dirs prints a warning if a content directory path
could not be resolved, i.e., if `abs_path` returns undef. Among other
things, `abs_path` returns undef if the path has an inner (= any but
last) component that does not exist. This can happen for a storage
with content type `iso,vztmpl` and `create-subdirs` set to 0, in case
`template/` does not exist. In this case, the warnings printed by
pvestatd are quite noisy.

As missing content directories are not a problem per se, remove the
warning and just ignore the directory during the inequality check.

Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
2023-06-15 14:19:33 +02:00
Fiona Ebner
46b9c1fe31 api: config: add/update storage: check for type mismatch first
This avoids confusing errors about other properties when the storage
type doesn't match. By highlighting that the type doesn't match, users
should know right away what the issue is.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-15 14:19:33 +02:00
Thomas Lamprecht
da6be324d9 bump version to 8.0.0
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-09 14:01:31 +02:00
Thomas Lamprecht
030e5ad550 content dirs: skip creation if either mkdir or create-subdirs is false
This is slightly confusing due to both options, the legacy convoluted
one and the new targeted one, exist, but before the rework we skip if
either of those sub-expressions was true, so doing it needs both to
be true.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-09 13:57:17 +02:00
Thomas Lamprecht
8e623a2930 activate storage: ensure content directories are created before checking them
checking the content dirs for clashes via abs_path must be done after
the logic for creating them ran, as abs_path is working on actual
filesystem level, so it will return undf if the directory does not
exist, in which case we then set a hash entry for "undef", and the
next for loop round then resolved again to "undef", resulting in a
false-positive of the check.

Avoid the dangerous "return if" stanzas and reverse them to an actual
if block, which is much safer to adapt. Then move the check for
duplicate content-dir usage after that.

best viewed with white space change ignored: git show -w

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-09 13:32:11 +02:00
Thomas Lamprecht
dcc761c628 bump version to 8.0.0~2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-07 19:10:47 +02:00
Stefan Hrdlicka
13ee4fc859 fix #2920: cifs: add options parameter
This makes it possible to add all mount options offered by mount.cifs.
NFS & CIFS now share the options parameter since they use it for the
same purpose.

Signed-off-by: Stefan Hrdlicka <s.hrdlicka@proxmox.com>
 [FE: rebase + style fixes]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Friedrich Weber <f.weber@proxmox.com>
 [T: fix merge conflict ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-07 10:45:12 +02:00
Markus Frank
765e0da23f changed regex to allow @ in directory storage paths
@ is often used to signify btrfs subvolumes.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
2023-06-07 08:58:04 +02:00
Thomas Lamprecht
6d9aac955d rbd: volume resize: wrap ceil in int, just to be sure
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-06 19:40:04 +02:00
Fiona Ebner
93e34f7259 rbd: volume resize: avoid passing floating point value to rbd
which causes an error "the argument for option '--size' is invalid".
Just round up to the nearest integer to have at least the requested
size. This is similar to what is done for ZFS with d3e3e5d ("When
resizing a ZFS volume, align size to 1M") and makes commands like 'qm
resize 102 scsi1 +0.01G' work.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-06 19:25:27 +02:00
Fiona Ebner
4ff0f451bd rbd: don't specify allow-shrink flag
It was introduced by commit 4b7dd9d ("allow --allow-shrink on RBD
resize"), but doesn't give a rationale. A mail gives more[0],
indicating that the user also uses the function to shrink images.
However, the volume_resize function is only reachable via the resize
API endpoints for VMs and containers, which have an explicit check to
disallow shrinkage. If somebody really wants to shrink the image, just
let them use the storage's tools directly. Calling into Proxmox VE's
perl functions directly is not supported.

[0]: https://lists.proxmox.com/pipermail/pve-devel/2016-November/024077.html

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-06 19:25:27 +02:00
Stefan Hanreich
0bbb358c91 volume export: explicitly state unit of bandwidth limit
Other places like replication explicitly state whether byte or bit
are used as a unit rather than only the abbreviation.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
2023-06-06 17:54:58 +02:00
Friedrich Weber
5f4b5bd11c content-dirs: check that all content dirs are pairwise inequal
This prevents strange interactions in case the same content directory
is used for multiple content types.

Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
2023-06-06 17:26:20 +02:00
Wolfgang Bumiller
49add58c4f bump version to 8.0.0~1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-05 14:46:22 +02:00
Wolfgang Bumiller
d81a9aea7b plugin: simplify and fix create-base-path vs mkdir logic
In the previous code, if `create-base-path` was explicitly
set to false, it would be treated the same as if it was
undef, falling through to whatever 'mkdir' was.

Instead, the new options should always be preferred, and the
logic can be simplified to a single line.

Here's the table showing the difference, 'u' being 'undef':

config: mkdir:  u 0 1 u 0 1 u 0 1
        create: u u u 0 0 0 1 1 1
        =========================
mkpath: old:    1 0 1 0 0 1 1 1 1
        new:    1 0 1 0 0 0 1 1 1

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-05 11:17:27 +02:00
Aaron Lauterer
7c242295c9 deprecate mkdir option for create-base-path and create-subdirs
The `mkdir` option has two meanings[0][1] which are split up in `create-path`
and `create-sub-dirs`.

The `create-base-path` option decides if the path to the storage is
automatically created or not.
The `create-subdirs` options decides if the default directory
structure (dump, images, ...) at the storage location is created.

The `mkdir` option is still working but will trigger a warning in the
logs.

As a side effect, this also fixes #3214 because the `create-base-path` option
is now run after the `is_mountpoint` check in the `activate_storage`
method in DirPlugin.pm.

The 'mkpath' command has been moved into a new helper function that
first determines if the conditions to create the path is true, called
'config_aware_base_mkdir'.

[0] https://lists.proxmox.com/pipermail/pve-devel/2020-December/046575.html
[1] https://lists.proxmox.com/pipermail/pve-devel/2020-December/046576.html

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-06-05 10:56:24 +02:00
Thomas Lamprecht
39d9fb2440 buildsys: derive upload dist automatically
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-24 17:21:02 +02:00
Thomas Lamprecht
ece47540d3 bump version to 7.99.0
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-24 16:23:08 +02:00
Thomas Lamprecht
8f7da880f0 d/control: raise standards version compliance to 4.6.2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-24 16:22:36 +02:00
Thomas Lamprecht
076345b053 d/control: define compat level via build-depends and raise to 13
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-24 16:22:28 +02:00
Thomas Lamprecht
d5b3f148a0 d/control: add missing ${misc:Depends}
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-24 16:22:18 +02:00
Thomas Lamprecht
b35efde24e buildsys: add sbuild convenience target
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-24 16:22:06 +02:00
Thomas Lamprecht
e59a960a53 buildsys: add DSC target
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-24 16:21:53 +02:00
Thomas Lamprecht
d785b69df9 buildsys: make build-dir generation atomic
and factor it out to own target

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-24 16:21:09 +02:00
Thomas Lamprecht
a2242b41fc separate packaging and source build system
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-24 16:20:27 +02:00
Thomas Lamprecht
f5e87de606 makefile: convert to use simple parenthesis
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-24 13:56:17 +02:00
Thomas Lamprecht
8eabe46e14 buildsys: rework doc-gen cleanup and makefile inclusion
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-24 13:53:18 +02:00
Thomas Lamprecht
4059e94bfa bump version to 7.4-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-23 08:46:02 +02:00
Friedrich Weber
4e55684caf api: upload: add pattern to tmpfilename parameter
The `tmpfilename` is generated by pve-http-server and always adheres
to this pattern, so make this explicit to prevent confusion and have
a more complete JSON API schema, useful for e.g., the API viewer.

Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
 [ T: slightly extended commit message ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-23 08:43:30 +02:00
Thomas Lamprecht
5c4d3db5ec fix various perlcritic lints
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-04-11 17:20:20 +02:00
Thomas Lamprecht
529b4f9de9 cephfs: fix variable declared in conditional statement
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-04-11 17:15:41 +02:00
Thomas Lamprecht
c698646f00 rbd: fix variable declared in conditional statement
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-04-11 17:14:55 +02:00
Thomas Lamprecht
f11bdb1731 bump version to 7.4-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-21 16:49:31 +01:00
Leo Nunner
8165ad58c6 feature: enable 'content-dirs' property for cephfs
Signed-off-by: Leo Nunner <l.nunner@proxmox.com>
2023-03-21 14:05:39 +01:00
Stoiko Ivanov
30c0c70852 cifs: use empty string instead of / as default directory
this keeps the mount sources consistent with previous versions
without this patch there is a small regression, which leads to the
storage not being recognized as being mounted on upgrade:
* pvestatd in older version mount the storage with out trailing /
```
//cifsstore/ISO on /mnt/pve/cifsstore type cifs...
```
* the cifs_is_mounted helper does not recognize it as being mounted
(as the source now has a / in the end)
* attempting to mount leads to
```
mount error(16): Device or resource busy
```

noticed after upgrading and having a cifs storage mounted

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2023-03-21 12:38:55 +01:00
Thomas Lamprecht
b2c6227261 bump version to 7.4-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-20 17:04:59 +01:00
Christian Ebner
d70d814ccf api: fix get content call response type for RBD/ZFS/iSCSI volumes
`pvesh get /nodes/{node}/storage/{storage}/content/{volume}` failed for
several storage types, because the respective storage plugins returned
only the volumes `size` on `volume_size_info` calls, while also the format
is required.

This patch fixes the issue by returning also `format` and where possible `used`.

The issue was reported in the forum:
https://forum.proxmox.com/threads/pvesh-get-nodes-node-storage-storage-content-volume-returns-error.123747/

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 [ T: fixup white space error ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-20 16:35:58 +01:00
Thomas Lamprecht
13e1af437a content path overrides: allow single dots and enforce max-lengths
Allow a dot as long as its not followed by another dot and enforce
max component and (a reduced) max path length checking already at
schema level.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-20 15:11:44 +01:00
Leo Nunner
a3c30c6871 config: use relative paths for content overrides
Remove the requirement for paths to start with a /, as it might be
confusing to users.

Signed-off-by: Leo Nunner <l.nunner@proxmox.com>
2023-03-20 14:30:43 +01:00
Thomas Lamprecht
574fb9ad11 directory: fix verify-dir code style
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-16 17:16:04 +01:00
Thomas Lamprecht
97edd11f6b cifs: small line bloat reduction
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-02-07 15:55:55 +01:00
Leo Nunner
362159a831 fix #2641: allow mounting of CIFS subdirectories
CIFS/SMB supports directly mounting subdirectories, so it makes sense to
also allow the --subdir parameter for these storages. The subdir
parameter was moved from CephFSPlugin.pm to Plugin.pm, because it isn't
specific to CephFS anymore.

Signed-off-by: Leo Nunner <l.nunner@proxmox.com>
2023-02-07 15:47:08 +01:00
Fiona Ebner
acff89540a nfs: check connection: support NFSv4-only servers without rpcbind
by simply doing a ping with the expected port as a fallback when the
rpcinfo command fails.

The timeout was chosen to be 2 seconds, because that's what the
existing callers of tcp_ping() in the iSCSI and GlusterFS plugins use.

Alternatively, the existing check could be replaced, but that would
1. Dumb down the check.
2. Risk breakage in some corner case that's yet to be discovered.
3. It would still be necessary to use rpcinfo (or dumb the check down
even further) in case port=0; from 'man 5 nfs' about the NFSv4 'port'
option:
> If the specified port value is 0, then the NFS client uses the NFS
> service port number advertised by the server's rpcbind service.

Reported in the community forum:
https://forum.proxmox.com/threads/118466/post-524449
https://forum.proxmox.com/threads/120774/

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-24 17:17:35 +01:00
Fiona Ebner
eaff3616f8 snapshot delete: add reminder to drop $running parameter with PVE 8.x
For the rationale, see:
https://lists.proxmox.com/pipermail/pve-devel/2022-December/055273.html

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-12 08:28:58 +01:00
Thomas Lamprecht
d324720d4b bump version to 7.3-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-11 16:47:33 +01:00
Fiona Ebner
0756b484d4 fix #4390: rbd: snapshot delete: avoid early return to fix handling TPM drive
The only caller where $running can even be truthy is QemuServer.pm's
qemu_volume_snapshot_delete(). But there, a check if snapshots should
be done with QEMU is already made and the storage function is only
called if snapshots should not be done with QEMU (like for TPM drives
which are not attached directly). So rely on the caller and do not
return early to fix removing snapshots in such cases.

Even if a stray call ends up here (can happen already by changing the
krbd setting while a VM is running to flip the above-mentioned check
and the early return check removed by this patch), it might not even
be problematic. At least a quick test worked fine:
1. take snapshot via a monitor command in QEMU
2. remove snapshot via the storage layer
3. create a new file in the VM
4. take a snapshot with the same name via monitor command in QEMU
5. roll back to the snapshot
6. check that the file in the VM is as expected
Using the storage layer to take the snapshots and QEMU to remove the
snapshot also worked doing the same test. Even if it were problematic,
the check in qemu-server should rather be improved then.

(Trying to issue a snapshot mon command for a krbd-mapped image fails
with an error on the other hand, but that is also not too bad and not
relevant to the storage code. Again, it rather would be necessary to
improve the check in qemu-server).

The fact that the pve-container code didn't even pass $running is the
reason removing snapshots worked for containers on a storage with krbd
disabled (the pve-container code calls map_volume() explicitly, so
containers can work regardless of the krbd setting in the storage
configuration; see commit 841fba6 ("call map_volume before using
volumes.") in pve-container).

For volume_snapshot(), the early return with $running was already
removed (or rather the relevant logic moved to QemuServer.pm) in 2015
by commit f5640e7 ("remove running from Storage and check it in
QemuServer"), even before krbd support was added in RBDPlugin.pm.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-01-11 16:42:42 +01:00
Fiona Ebner
d05015f887 zfs: list zvol: limit recursion depth to 1
To be correct in all cases, it's still necessary to filter by "pool"
in zfs_parse_zvol_list(), because $scfg->{pool} could be e.g.
'foo/vm-123-disk-0' which looks like an image name and would pass the
other "should skip"-checks in zfs_parse_zvol_list().

No change in the result of zfs_list_zvol() is intended.

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-10 14:31:21 +01:00