5
0
mirror of git://git.proxmox.com/git/pve-storage.git synced 2025-01-11 05:18:01 +03:00
Commit Graph

1390 Commits

Author SHA1 Message Date
Fabian Ebner
68f1fc2783 mark PBS storages as shared
Like this, the property will get added when parsing the storage configuration
and PBS storages will correctly show up as shared storages in API results.

AFAICT the only affected PBS operation is free_image via vdisk_free, which will
now be protected by a cluster-wide lock, and that shouldn't hurt.

Another issue this fixes, which is the reason this patch exists, was reported
in the forum[0]. The free space from PBS storages was counted once for each node
that had access to the storage.

[0]: https://forum.proxmox.com/threads/pve-6-3-the-storage-size-was-displayed-incorrectly.83136/

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-01-28 16:43:20 +01:00
Dominic Jäger
d854a71803 lvm: Fix #3159: Show RAID LVs as storage content
LVM RAID logical volumes (including mirrors) can be valid disk images, so they
should show up in storage content listings (for example pvesm list).

Including LV types is safer than excluding, especially because of possible
additional types in the future.

Co-developed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2021-01-26 19:19:02 +01:00
Alwin Antreich
89b9ac961a fix: check connection for nfs v4 only server
the check_connection is done by querying the exports of the nfs server
in question. With nfs v4 those exports aren't listed anymore since nfs
v4 employs a pseudo-filesystem starting from root (/).

rpcinfo allows to query the existence of an nfs v4 service.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2021-01-26 19:15:17 +01:00
Thomas Lamprecht
d7fedf266a bump version to 6.3-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-26 18:37:38 +01:00
Dominik Csapak
26a8f21ab2 add workaround for zfs rollback bug
as described in the zfs bug https://github.com/openzfs/zfs/issues/10931
the kernel keeps around cached data from mmaps after a rollback, thus
having invalid data in files that were allegedly rolled back

to workaround this (until a real fix comes along), we unmount the subvol,
invalidating the kernel cache anyway

the dataset gets mounted on the next 'activate_volume' again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 18:32:47 +01:00
Fabian Grünbichler
d3a5e30963 drop absolute udevadm path
the compat symlink from bin to sbin has been dropped with bullseye, and
we rely on PATH begin set properly in our daemons/CLI tools anyway..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 18:27:14 +01:00
Thomas Lamprecht
a7e0184190 debian/control: use https in homepage link
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-26 14:04:15 +01:00
Dominik Csapak
2c048efdaf Diskmanage: extend wearout detection for SAS disk
for some controllers/disks there the line is
Percentage used endurance indicator: x%

so extend the regex for that possibilty.
We even had a test-case for SAS but did not notice we could extract
that info from there...

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-15 14:42:16 +01:00
Dominik Csapak
a66db4ae55 tests: add ssd sas disk
copied from test 'sas' with rotational set to 0
this has then the type 'ssd', rpm: 0, and health: 'OK'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-15 14:42:16 +01:00
Fabian Ebner
189e67ffa6 fix #3199: by fixing usage of strftime
In a very early version I wanted to parse the date from the backup
name, and when switching to using the ctime and localtime() instead,
I forgot to update the usage of strftime.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-15 14:40:45 +01:00
Thomas Lamprecht
32dbc619a5 drbd: comment that the builtin plugin is depreacated
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-15 12:58:20 +01:00
Thomas Lamprecht
c6e632efb6 d/control: update
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-14 21:58:24 +01:00
Thomas Lamprecht
175d73fa75 d/compat: update to 10
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-14 21:53:13 +01:00
Thomas Lamprecht
b59f5e0eed bump version to 6.3-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-14 16:15:07 +01:00
Fabian Ebner
10dfeb9ef0 prune mark: correctly keep track of already included backups
This needs to happen in a separate loop, because some time intervals are not
subsets of others, i.e. weeks and months. Previously, with a daily backup
schedule, having:
* a backup on Sun, 06 Dec 2020 kept by keep-daily
* a backup on Sun, 29 Nov 2020 kept by keep-weekly
would lead to the backup on Mon, 30 Nov 2020 to be selected for keep-monthly,
because the iteration did not yet reach the backup on Sun, 29 Nov 2020 that
would mark November as being covered.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-12-14 16:11:29 +01:00
Thomas Lamprecht
44fdfb2af6 nfs and cifs: implement backup notes helper
reuse the one from DirPlugin by directing the call to it, but with
the actual $class. This should stay stable, as we provide an ABI and
try to always use $class->helpers.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-07 16:13:09 +01:00
Thomas Lamprecht
f244e2aa7f api: content/backup: handle deletion of notes
Previous to this we did not called the plugins update_volume_notes at
all in the case where a user delted the textarea, which results to
passing a falsy value ('').

Also adapt the currently sole implementation to delete the notes field
in the undef or '' value case. This can be done safely, as we default
to returning an empty string if no notes file exists.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-07 16:10:09 +01:00
Thomas Lamprecht
20471dfd95 dir plugin: code cleanup
mostly re-ordering to improve statement grouping and avoiding the
need for an intermediate variable

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-07 16:09:26 +01:00
Thomas Lamprecht
a8e00d1a5d bump version to 6.3-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-03 17:24:59 +01:00
Thomas Lamprecht
b1f50fb5cc d/control: bump versioned dependency for libpve-common-perl
so the new get_repository helper for PBS is available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-03 16:55:28 +01:00
Dominik Csapak
53003cb5ea PBSPlugin: use get_repository from PVE::PBSClient
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-03 16:53:53 +01:00
Wolfgang Bumiller
ab90c3b1f1 pbs: fix token auth with PVE::APIClient
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-03 16:53:43 +01:00
Thomas Lamprecht
405b247168 api: scan: note that USB is depreacated
It now got moved in /nodes/<node>/hardware/usb as envisioned[0], this
allows to sunset the usb scan API endpoint here and drop it with 7.0

[0]: https://lists.proxmox.com/pipermail/pve-devel/2018-November/034694.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-02 11:31:06 +01:00
Thomas Lamprecht
a57101434b bump version to 6.3-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:27:54 +01:00
Thomas Lamprecht
39fc0c77e2 api/cli: add pbs scan endpoint and command
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
Thomas Lamprecht
11a942a18a pvesm: also map the password param for new style cifs scan command
not only for the old deprecated alias

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
Thomas Lamprecht
b1ccf4f357 api: scan: move over index and usb scan from manager
Add the missing pieces allowing pve-manager to just point the
/nodes/<node>/scan api directory at this module, dropping it's
duplicated copy.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
Thomas Lamprecht
e65abdb458 api: scan cifs: port over NT_STATUS filter from pve-manager
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
Thomas Lamprecht
2892b6739d factor out scan CLI definition to real API module
we have a 1:1 copy of that code in pve-manager's PVE::API2::Scan,
which we can avoid by using a common module form pvesm CLI and the
API.

This is the first basic step of dropping the code duplication in
pve-manager.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
Thomas Lamprecht
2cd10f58be pbs: activate storage: fully validate if storage config works
improves UX of on_update and on_add hooks *a lot*.

This is a bit more expensive than the TCP ping, or even just an
unauthenticated ping, but not as bad as a full datastore status - as
this only reads the datastore config file (which is normally in page
cache anyway).

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
Thomas Lamprecht
8b62ac6a0c pbs: add scan datastore helper
for use in both, the scan API and the on_add/on_update hooks

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
Thomas Lamprecht
2f9eb6dc4c pbs: reuse pve apiclient for api connect helper
it is flexible enough to easily do so, and should do well until we
actually have cheap native bindings (e.g., through wolfgangs rust
permlod magic).

Make it a private helper, we do *not* want to expose it directly for
now.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
Fabian Ebner
f3ccd0ef3d plugin: hooks: add explicit returns
to avoid returning something unexpected. Finish what
afeda18256 already started for all the other
plugins. At least for ZFS's on_add_hook this is necessary (adding a ZFS storage
currently fails as reported here [0]), but it cannot hurt
in the other places either as the only hooks we expect to return something
currently are PBS's on_add_hook and on_update_hook.

[0]: https://forum.proxmox.com/threads/gui-add-zfs-storage-verification-failed-400-config-type-check-object-failed.79734/

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-27 10:45:42 +01:00
Thomas Lamprecht
74dcca3a48 nfs: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-27 10:45:06 +01:00
Thomas Lamprecht
39591d92ec bump version to 6.3-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 23:20:58 +01:00
Thomas Lamprecht
878fe0177e api: content: pass encrypted status for PBS backups
Prefer the fingerprint, fallback to checking the files crypt-mode.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 23:18:05 +01:00
Thomas Lamprecht
3cc2eb738f pbs add/update: save fingerprint in storage config
fallback to the old truthy "1" if not available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 22:09:40 +01:00
Thomas Lamprecht
d2c47b3837 pbs add/update: do basic key value validation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 22:09:15 +01:00
Thomas Lamprecht
478609d3bc pbs: autogen key: rename old one if existing
it could be debated do have some security implications and that
deletion is safer, but key deletion is a pretty hairy thing.

Should be documented, and people just should use delete instead of
autogen if they want to "destroy" a key.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 22:05:23 +01:00
Thomas Lamprecht
7e6a7a7f39 bump version to 6.2-12
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 16:06:03 +01:00
Dominik Csapak
45e93e6dda Storage/PBSPlugin: implement get/update_volume_notes for pbs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 13:42:13 +01:00
Dominik Csapak
e9991d2694 Storage/Plugin: add get/update_volume_comment and implement for dir
and add the appropriate api call to set and get the comment
we need to bump APIVER for this and can bump APIAGE, since
we only use it at this new call that can work with the default
implementation

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 10:23:25 +01:00
Dominik Csapak
3369651885 api2/storage/content: change to volume_size_info and add return properties
'file_size_info' only works for directory based storages, while
'volume_size_info' should work for all

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 10:23:25 +01:00
Dominik Csapak
6fef456c8d rename comment to notes
so that we are more consistent with pbs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 10:23:25 +01:00
Thomas Lamprecht
093d5ad385 bump version to 6.2-11
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-23 19:37:00 +01:00
Fabian Ebner
d4e8a1f27c convert maxfiles to prune_backups when reading the storage configuration
If there are already prune options configured, simply delete the maxfiles
setting. Having set both is invalid from vzdump's perspective anyways, and any
backup job on such a storage failed, meaning a user would've noticed.

If there are no prune options, translate the maxfiles value to keep-last,
except for maxfiles being zero (=unlimited), in which case we use keep-all.

If both are not set, don't set anything, so:
1. Storages don't suddenly have retention options set.
2. People relying on vzdump defaults can still use those.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-23 15:57:30 +01:00
Fabian Ebner
1b87f01388 prune: introduce keep-all option
useful to have an alternative to the old maxfiles = 0. There has to
be a way for vzdump to distinguish between:
1. use the /etc/vzdump.conf default (when no options are configured for the storage)
2. use no limit (when keep-all=1)

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-23 15:27:17 +01:00
Thomas Lamprecht
5045e0b77a perlcritic: don't modify $_ in list functions, use for
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-23 15:27:14 +01:00
Thomas Lamprecht
1207620cf0 perlcritic: avoid conditional variable declaration
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-23 15:24:02 +01:00
Thomas Lamprecht
afeda18256 plugin: hooks: avoid that method param count gets returned
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-23 15:12:58 +01:00