5
0
mirror of git://git.proxmox.com/git/pve-storage.git synced 2024-12-23 17:34:34 +03:00
Commit Graph

91 Commits

Author SHA1 Message Date
Lorenz Stechauner
bba10cf4af factoring out regex for vztmpl
stores the regex definition in PVE::Storage.

One test had to be adapted because it tested obsolete code. Namely:
it expects vztmpl to only end with .tar.gz, but the new regex also
includes .tar.xz, there is nothing against allowing .tar.xz files as
vztmpl files.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-06-23 20:19:09 +02:00
Thomas Lamprecht
ffc31266da tree-wide: fix typos with codespell
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 08:28:48 +02:00
Thomas Lamprecht
0d8ed0a313 test: rbd: improve usage output
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 15:43:42 +02:00
Thomas Lamprecht
23ba4ac2c6 test: rbd: typo fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 15:27:33 +02:00
Thomas Lamprecht
33191d3266 test: rbd: indendation cleanup, allow -d short-opt for debug
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 15:27:26 +02:00
Thomas Lamprecht
0d7c9ca188 test: rbd: actually use debug helper
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 15:27:23 +02:00
Thomas Lamprecht
9407a7a338 tests: rbd ns: exit cleanly on help, and allow short-opt -h
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 14:46:12 +02:00
Thomas Lamprecht
876d268298 tests: rbd ns: avoid underscore in CLI param name
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 14:42:38 +02:00
Aaron Lauterer
9f3d6f3bb9 rbd: add integration test for namespace handling
This test is intended to be run on a hyperconverged PVE cluster to test
the most common operations of VMs using a namespaced Ceph RBD pool.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2021-04-09 12:56:21 +02:00
Fabian Ebner
061c7e5e3d tests: zfs: complain when a first sub-test dies
Previously, $count was not increased and no test failure was reported.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-03-15 13:36:53 +01:00
Fabian Ebner
525b4a6ebe Diskmanage: replace check for zpool binary with a function and mock it
so the test still works when it's not installed.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-19 15:29:04 +01:00
Fabian Ebner
40be5c5cab Diskmanage: introduce get_sysdir_size helper
to be used for partitions as well.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
Fabian Ebner
59c03cd970 Diskmanage: also check for filesystem type when determining usage
Like this, a non-ZFS filesystem living on a whole disk will also be detected
when it is not mounted.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
Dominik Csapak
2c048efdaf Diskmanage: extend wearout detection for SAS disk
for some controllers/disks there the line is
Percentage used endurance indicator: x%

so extend the regex for that possibilty.
We even had a test-case for SAS but did not notice we could extract
that info from there...

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-15 14:42:16 +01:00
Dominik Csapak
a66db4ae55 tests: add ssd sas disk
copied from test 'sas' with rotational set to 0
this has then the type 'ssd', rpm: 0, and health: 'OK'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-15 14:42:16 +01:00
Fabian Ebner
189e67ffa6 fix #3199: by fixing usage of strftime
In a very early version I wanted to parse the date from the backup
name, and when switching to using the ctime and localtime() instead,
I forgot to update the usage of strftime.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-15 14:40:45 +01:00
Fabian Ebner
10dfeb9ef0 prune mark: correctly keep track of already included backups
This needs to happen in a separate loop, because some time intervals are not
subsets of others, i.e. weeks and months. Previously, with a daily backup
schedule, having:
* a backup on Sun, 06 Dec 2020 kept by keep-daily
* a backup on Sun, 29 Nov 2020 kept by keep-weekly
would lead to the backup on Mon, 30 Nov 2020 to be selected for keep-monthly,
because the iteration did not yet reach the backup on Sun, 29 Nov 2020 that
would mark November as being covered.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-12-14 16:11:29 +01:00
Fabian Ebner
f514181d28 prune mark: keep all if all prune options are zero/missing
as an additional safety measure. And add some tests.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-16 10:14:11 +01:00
Jan-Jonas Sämann
641146add8 Update disk_tests/ssd_smart/sde data
Provides recent test data for disk_tests/ssd_smart/sde_smart. The
previous data was created using an older smartmontools version, which
did not yet support the drive and therefore had bogus attribute mapping.

Signed-off-by: Jan-Jonas Sämann <sprinterfreak@binary-kitchen.de>
2020-10-30 15:25:11 +01:00
Thomas Lamprecht
4731eb1118 disk management: set more specific type for nvme
some users are confused, and it's nicer to have the more specific
type presented here.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-09 18:06:38 +02:00
Fabian Ebner
8f26b3910d Add prune_backups to storage API
Implement it for generic storages supporting backups
(i.e. directory-based storages) and add a wrapper for PBS.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-24 15:44:53 +02:00
Fabian Ebner
b1ddc54a93 archive_info: use timelocal correctly
Because we always have 4-digit years, we can simply pass
the year itself to timelocal instead of subtracting 1900.
Like this it will also work for years not in the range 2000-2999.

See also:
https://perldoc.perl.org/Time/Local.html#Year-Value-Interpretation

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-08 10:45:49 +02:00
Fabian Ebner
e34afeb117 Extend archive_info to include filename and logfilename
Only expect the logfilename if the archive has a standard name.
This also gives a mechanism to get an untainted filename.

archive_info can take either a volume ID or a path as it's
currently implemented. This is useful for vzdump when there
is no storage (i.e. for 'vzdump --dumpdir'). Add a test case for this.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-06-30 13:59:23 +02:00
Fabian Ebner
fb821c1828 Expand archive_info to include ctime, vmid and is_std_name
where 'is_std_name' shows whether the backup name uses the standard naming
schema and most likely was created by our tools.

Also adds a '^' to the existing filename matching regex, which
should be fine since basename() is used beforehand.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-06-06 19:38:02 +02:00
Stoiko Ivanov
0f0d99a3e5 fix #2777 create zpools with stable dev paths
when compiling the disk list add a property with a stable
/dev/disk/by-id/ path for a block device when available.

This is needed to create zpools with the stable by-id links

The /dev/disk/by-id/ directory can contain multiple links to the same device
(e.g. when it's used as a LVM PV, or one for the wwn/nvme-eui in addition
to the one with vendor and serial). We take the first one which matches
the bus where the disk is attached. For nvme disks we exclude the one
containing the nvme-eui.

The patch assumes that not all disks need to have such a link (e.g.
virtio-block devices as we pass them to guests).

Additionally the tests were adapted to run successfully.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-06-06 19:32:33 +02:00
Thomas Lamprecht
a6c4705a46 tests: re-add virt "none" negative test
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-15 19:32:21 +02:00
Thomas Lamprecht
5029f978c3 archive info: keep some basic strictness
we want to enforce at least the strictness that our tools can do
something with a backup archive..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-15 19:18:24 +02:00
Thomas Lamprecht
bf5af0fbf5 archive info: include archive name in error message
so that we have some context if users report issues with it..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-15 19:13:39 +02:00
Thomas Lamprecht
c265d42160 tests: archive info: avoid assembling arrays to just de-assemble them directly
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-15 19:11:48 +02:00
Thomas Lamprecht
5df46bf2d5 tests: archive info: make deterministic
nothing worse than jumping test results due to not sorting the tests
and thus making discussions harder (which test # failed) and it may
imply another error by mistake.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-15 19:10:23 +02:00
Thomas Lamprecht
3dde6a9cb9 tests: archive info: code cleanup
We could also do a hash array slice, like:
> my ($archive, $expected, $description) = $tt->@{'archive', 'expected', 'description'};

But as none are optional lets just access values directly..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-15 19:01:39 +02:00
Alwin Antreich
4b26f8140d Fix: backup: relax file name matching regex
The rework of the backup file detection logic missed the non-standard
file name case. This patch allows to restore backups with different file
names. Though the config extraction fails, since the type is unknown.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-05-15 18:12:01 +02:00
Fabian Grünbichler
aa738ee355 test: force ctime to UTC
otherwise this test fails depending on TZ of the testing system

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-05-04 11:10:56 +02:00
Alwin Antreich
2eb7ba261d test: filesystem_path
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
Alwin Antreich
ddd313c95e test: get_subdir
Co-Authored-by: Dominic Jaeger <d.jaeger@proxmox.com>
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
Alwin Antreich
014d36dbbb Fix: #2124 storage: add zstd support
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
Alwin Antreich
73fcb7bfd1 Fix: add missing snippets subdir
since it is a valid content type and adapt the path_to_volume_id_test.
Also adds an extra check if all vtype_subdirs are returned.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
Alwin Antreich
892dc99249 Fix: path_to_volume_id returned wrong content
type for backup files. Patch includes changes of the test as well.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
Alwin Antreich
e4dbfde31f test: path_to_volume_id
Test to reduce the potential for accidental breakage on regex changes.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
Alwin Antreich
40c795e7df Fix: backup: ctime was from stat not file name
The vzdump file was passed with the full path to the regex. That regex
captures the time from the file name, to calculate the epoch.

As the regex didn't match, the ctime from stat was taken instead. This
resulted in the ctime shown when the file was changed, not when the
backup was made.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
Alwin Antreich
6a27e8171e test: list_volumes
Test to reduce the potential for accidental breakage on regex changes.

Co-Authored-by: Dominic Jaeger <d.jaeger@proxmox.com>
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
Alwin Antreich
c48801b52a test: parse_volname
Test to reduce the potential for accidental breakage on regex changes.
And to make sure that all vtype_subdirs are parsed.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
Alwin Antreich
cd554b79d1 storage: test: split archive format/compressor
detection into separate functions so they are reusable and easier
modifiable. This patch also adds the test for archive_info.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
Dominik Csapak
413862d2cb disk_tests: improve nvme smart test
by using an actual percentage for 'Percentage Used' instead of 0%

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-04-15 12:52:45 +02:00
Fabian Ebner
dcefd9dd28 fix #2085: add mountpoint property for non-default ZFS pool MPs
When adding a zfspool storage with 'pvesm add' the mount point is now
added automatically to the storage configuration if it can be
determined.  path() does not assume the default mountpoint anymore,
fixing 2085.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2019-11-21 12:42:38 +01:00
Fabian Grünbichler
56362cfb55 ZFS: refactor waiting for zvol symlinks
and actually do that not just for creating zvols, but also when
activating them. this should fix a range of issues/races that sometimes
occured on bootup, snapshot rollback or similar operations.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-08-06 13:39:43 +02:00
Dominik Csapak
bfb3d42daf Diskmanage: add flag for encrypted osds
we can only do this here, since the ceph cluster is not aware of
osd encryption, only the local node is (via ceph-volume and lv tags)

this way, we are able to show an 'encrypted' flag in the disk gui at least

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:38:24 +02:00
Dominik Csapak
1e6057d1b4 Diskmanage: fix usage test for sdd
sdd1 and sdd2 were listed in the zfs output but were not included
in the partlist, so the zfs detection test was wrong

now it is correct and more in line with the real world

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:37:57 +02:00
Dominik Csapak
b8cbfd362f Diskmanage: add test for lsblk by simulating zfs disk
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:37:57 +02:00
Dominik Csapak
8cd6d7e8e3 Diskmanage: change parttype uuid detection
previously ceph included a udev rule to populate
/dev/disk/by-parttypeuuid/

but not anymore, so we now use 'lsblk --json -o path,parttype' to
get a mapping between parttype uuid and partition

fix the test by simulating empty lsblk output

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:37:57 +02:00