5
0
mirror of git://git.proxmox.com/git/pve-storage.git synced 2025-01-25 06:03:53 +03:00

1318 Commits

Author SHA1 Message Date
Thomas Lamprecht
a9078a7922 bump version to 6.2-9
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-13 11:14:10 +02:00
Stoiko Ivanov
6726a2e037 LIO: drop unused statements
minor cleanup of left-over/unused statements.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-10-13 11:11:05 +02:00
Stoiko Ivanov
d4abdf4e22 LIO: untaint values read from remote config
The LIO backend for ZFS over iSCSI fetches the json-config periodically from
the target.
This patch reduces the stored config values to those which are actually used
and additonally untaints the values read from the remote host's config-file.

Since the LUN index is used in calls to targetcli on the remote host (via
run_command), untainting prevents the call to crash when run with '-T'.

Tested by creating a zfs over iscsi backed VM, starting it, adding disks,
resizing disks, removing disks, creating snapshots, rolling back to a snapshot.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-10-13 11:11:05 +02:00
Stoiko Ivanov
609f117ff2 ZFSPlugin: untaint lun number
ZFS over iSCSI fetches information about the disk-images via ssh, thus
the obtainted data is tainted (perlsec (1)).

Since pvedaemon runs with '-T' enabled trying to start a VM via GUI/API failed,
while it still worked via `qm` or `pvesh`.

The issue surfaced after commit cb9db10c1a9855cf40ff13e81f9dd97d6a9b2698 in
pve-common ('run_command: improve performance for logging and long lines'),
and results from concatenating the original (tainted) buffer to a variable,
instead of a captured subgroup.

Untainting the value in ZFSPlugin should not cause any regressiosn, since the
other 3 target providers already have a match on '\d+' for retrieving the
lun number.

reported via pve-user [0].

reproduced and tested by setting up a LIO-target (on top of a virtual PVE),
adding it as storage and trying to start a guest (with a disk on the
ZFS over iSCSI storage) with `perl -T /usr/sbin/qm start $vmid`

[0] https://lists.proxmox.com/pipermail/pve-user/2020-October/172055.html

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-10-09 18:07:37 +02:00
Thomas Lamprecht
4731eb1118 disk management: set more specific type for nvme
some users are confused, and it's nicer to have the more specific
type presented here.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-09 18:06:38 +02:00
Thomas Lamprecht
d5c80a5bd5 code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-09 18:06:07 +02:00
Fabian Grünbichler
d1f4700063 file_size_info: handle dangling symlinks
and other stat failure modes.

this method returns undef if 'qemu-img info ...' fails to return
information, so callers must handle this already.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-05 12:51:44 +02:00
Thomas Lamprecht
c018887fd3 bump version to 6.2-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-02 16:09:15 +02:00
Thomas Lamprecht
3de423680a PBS: use simple TCP ping for online check for now
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-02 15:55:03 +02:00
Thomas Lamprecht
4133e6e216 PBS: add support to specify port
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-02 15:49:48 +02:00
Thomas Lamprecht
00f1de310e bump version to 6.2-7
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-29 18:56:04 +02:00
Thomas Lamprecht
c9c90349c3 check for service exsitance before enabling zfs-import service
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-29 18:52:32 +02:00
Stoiko Ivanov
f720f6c440 Disks: instantiate import unit for created zpool
When creating a new ZFS storage, also instantiate an import-unit for the pool.
This should help mitigate the case where some pools don't get imported during
boot, because they are not listed in an existing zpool.cache file.

This patch needs the corresponding addition of 'zfs-import@.service' in
the zfsonlinux repository.

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-09-29 18:52:32 +02:00
Stoiko Ivanov
59fdc2b71e fix regression in zfs volume activation
commit 815df2dd08ac4c7295135262e60d64fbb57b8f5c introduced a small issue
when activating linked clone volumes - the volname passed contains
basevol/subvol, which needs to be translated to subvol.

using the path method should be a robust way to get the actual path for
activation.

Found and tested by building the package as root (otherwise the zfs
regressiontests are skipped).

Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-09-29 18:52:32 +02:00
Thomas Lamprecht
e3eb131ec5 zfs pool: clean up use statements
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-29 05:08:57 +02:00
Fabian Ebner
815df2dd08 ZFS: mount subvols in activate_volume
Makes it possible to clone and start a container whose
ZFS subvols are not yet mounted for some reason. If a
subvol cannot be mounted, there's a better error now:
zfs error: cannot mount '/myzpool/subvol-103-disk-0': directory is not empty

Previously, cloning would quietly do an "empty" clone,
and startup would fail with:
mount_autodev: 1074 Permission denied - Failed to create "/dev" directory
lxc_setup: 3238 Failed to mount "/dev"
do_start: 1224 Failed to setup container "103"
__sync_wait: 41 An error occurred in another process (expected sequence number 5)

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-09-29 05:07:15 +02:00
Fabian Grünbichler
d0eaf18571 zfs: rollback: improve error message
we don't even know whether $snap exists at all, so the old variant could
be rather misleading..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-23 15:11:17 +02:00
Fabian Grünbichler
c8eb017867 zfs: handle unexpectedly missing snapshots better
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-23 15:11:17 +02:00
Thomas Lamprecht
48d0cd02c1 fix indentation of $prune_backups_format
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-03 18:21:09 +02:00
Fabian Ebner
8ca00a63f7 prune-backups: improve command description
This is shown in the man page, so it's not important to mention
that this is a wrapper. Also mention the fact that the keep options
from the storage configuration serve as a fallback, which was previously
mentioned in the description of the (now removed) prune-backups parameter.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-09-03 18:20:06 +02:00
Fabian Ebner
a0933d7e16 prune-backups CLI: use keep-options directly
Makes the interface cleaner; e.g. --keep-daily=2 instead of
--prune-backups=keep-daily=2

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-09-03 18:20:06 +02:00
Fabian Ebner
c3e87d0f6e prune_backups CLI: print different message when there's no backups at all
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-08-20 17:28:07 +02:00
Fabian Ebner
7b73d327b5 prune_backups: fix message
For prune selections, it doesn't matter what the current time is,
only the timestamps of the backups matter.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-08-20 17:28:07 +02:00
Fabian Ebner
93d1812e5a storage_migrate: log bandwidth limit
and avoid undefined post-if declaration of @cstream.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-08-04 13:57:24 +02:00
Thomas Lamprecht
b31e971ceb bump version to 6.2-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 19:10:06 +02:00
Thomas Lamprecht
4ef17e1fe0 pbs: check if encryption key could be deleted
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 19:06:09 +02:00
Thomas Lamprecht
18cf6c9f27 config: add hint for set encryption key, die on unlink error
as else the API cannot easily know if this is set, it cannot check
with -f as the key is in a restricted area and we do not want that a
GET runs as protected.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 19:05:25 +02:00
Fabian Ebner
25a9583634 Add API and pvesm call for prune_backups
For the pvesm call use a wrapper and a --dry-run option to redirect
to the correct API call.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-24 15:44:53 +02:00
Fabian Ebner
8f26b3910d Add prune_backups to storage API
Implement it for generic storages supporting backups
(i.e. directory-based storages) and add a wrapper for PBS.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-24 15:44:53 +02:00
Fabian Ebner
3353698f45 Introduce prune-backups property for directory-based storages
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-24 15:44:53 +02:00
Wolfgang Bumiller
0dffc7ba55 bump version to 6.2-5
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-14 13:58:44 +02:00
Wolfgang Bumiller
0ca8eb4f9d pvesm: pass-through 'autogen' special value for encryption key
so that the on-update hook can handle its creation

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-14 13:57:57 +02:00
Wolfgang Bumiller
b3b63fc27d fix another encryption_key vs encryption-key instance
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-14 13:51:26 +02:00
Wolfgang Bumiller
842e23e347 bump version to 6.2-4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-14 13:45:29 +02:00
Wolfgang Bumiller
3ca5000825 param mapping fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-10 10:52:45 +02:00
Thomas Lamprecht
c43655d2ed vdisk_list: skip scanning storages which cannot have images/rootdisks
Do not try to scan (and thus activate) storages which aren't
configured to support (or cannot support) "vdisks" anyway.

Avoids seemingly strange failures of VM migrations due to a backup storage
not being currently online - even if that storage isn't referenced in
the VM config anywhere..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 16:16:23 +02:00
Wolfgang Bumiller
d6fc193816 use ssh -X for upload
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 15:00:38 +02:00
Wolfgang Bumiller
6c67b3e239 bump version to 6.2-3
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 15:00:10 +02:00
Wolfgang Bumiller
02cc5e1033 quick fixup for prune command in vzdump
This is a hack and we should get rid of `run_client_cmd` and
`run_raw_client_cmd` as an API entry!

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 14:55:07 +02:00
Thomas Lamprecht
a08b499571 bump version to 6.2-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 14:03:33 +02:00
Thomas Lamprecht
9c81e528af d/control: depend on proxmox-backup-client
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 12:34:51 +02:00
Thomas Lamprecht
bb0a0f9680 pbs: activate_storage: query status to ensure we can connect
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 11:54:13 +02:00
Thomas Lamprecht
1aeb322bc3 pbs: allow to autogen an encryption key
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 11:53:51 +02:00
Thomas Lamprecht
ce2e273359 pbs: rename encryption_key param to encryption-key
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 11:53:27 +02:00
Wolfgang Bumiller
baf7712010 pvesm: encryption key parameter should load files
also `pvesm set` and `pvesm add` should behave the same with
respect to how configuration options are treated

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 11:49:02 +02:00
Wolfgang Bumiller
72385de9e2 refactor sensitive parameter handling
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 11:49:02 +02:00
Wolfgang Bumiller
76bb5febc7 pbs: encryption support, split "raw client command" API
(And deprecate it...)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 11:49:02 +02:00
Thomas Lamprecht
0f2549edd4 cifs: use more general name for password description
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 11:48:52 +02:00
Fabian Grünbichler
28be2a431b archive_info: relax custom name handling
we already differentiate between standard and non-standard names anyway
and don't detect and return the VMID in the latter case anyway. drop it
from the RE as well to allow names like 'vzdump-qemu-template.vma.lzo'
without the need for a fake VMID.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-07-08 15:26:19 +02:00
Fabian Ebner
b1ddc54a93 archive_info: use timelocal correctly
Because we always have 4-digit years, we can simply pass
the year itself to timelocal instead of subtracting 1900.
Like this it will also work for years not in the range 2000-2999.

See also:
https://perldoc.perl.org/Time/Local.html#Year-Value-Interpretation

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-08 10:45:49 +02:00