5
0
mirror of git://git.proxmox.com/git/pve-storage.git synced 2024-12-22 13:34:16 +03:00
Commit Graph

1919 Commits

Author SHA1 Message Date
Wolfgang Bumiller
2ff6f99c60 esxi: add 'skip-cert-verification' option
following pve-esxi-import-tools's commits:
3ee5c3b ("esxi-folder-fuse: add --insecure option")
c292c67 ("listvms.py: add --insecure parameter, verify cert by
          default")
34c87be ("rename --insecure option to --skip-cert-verification")

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 [ TL: rename 'insecure' to 'skip-cert-verification' to better convey
   what it means ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-13 15:29:35 +01:00
Dominik Csapak
4f50a578f4 import: rework plugin api to not need a blessed object
does not really make sense, and if the plugin wants that, it can still
be done, like we do here for the ESXiPlugin

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
c92d810053 import: drop target parameter
since disks are a separate hash now with no associated storage, we
don't need this anymore

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
10b5874078 esxi: add disk capacity to import metadata
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
1128100569 esxi: warn if the guest was running
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
aba709f247 esxi: add helper to get vm from manifest for vmx path
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
99ace9d447 esxi: let manifest file expire after 60s
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 [ TL: fix wrong comparison with >= and avoid undef warning if file
   does not yet exist at all ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
814bde1958 esxi: drop unused sub methods
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 [ TL: squash removal of both subs into one commit ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-13 15:29:35 +01:00
Thomas Lamprecht
1a94dceb6f api: import-metadata: make warnings structured & merge ignored-volumes
This allows the frontends to translate them and avoids somewhat
duplicated info by having some warnings explicitly (ignored-volumes)
while others are in the warnings array.

By passing along the key and the value the frontend can also show the
warnings in-line, e.g. by marking a disk-entry in a grid as having
potential problems.

Ideally we'd have a central list of known types used for the API
return schema enum and to check when calling the $warn closure, but as
we only got three warnings keep this as is and only add a comment.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-13 15:29:35 +01:00
Thomas Lamprecht
b628964289 api: import-metadata: complete return schema
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-13 15:29:35 +01:00
Thomas Lamprecht
da513e260c esxi plugin: do not support the images content-type
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-13 15:29:35 +01:00
Thomas Lamprecht
dde640eda7 esxi plugin: adapt to new libexec path for helpers
and rename the name of the constant for the FUSE tool, as otherwise it
sounds like this was the mountpoint to be used

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
5e934933ec esxi import: deal with serial devices
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
fd16d98451 add source field to import-metadata
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
203d73baf9 add a warnings array to import-metadata
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
a3271a47ca esxi: improve scsihw detection
we should actually keep the scsihw even with eg. linux VMs as they
might not have virtio/... in their initrd and then fail to boot

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
728c08b2f0 esxi: separate disks and network
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
763a2292f3 import: return skipped/ignored volumes in get_create_args
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
e72845efbf api: add storage/{storage}/import-metadata
This will be used for returning the base meta information of a
external VM that is about to be imported into Proxmox VE.

A front-end can use this endpoint to show the proposed configs with
potential override switches to the user, so that they can adapt the
most important options to ensure that import can work.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 [ TL: add more commit message with some background ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-13 15:29:35 +01:00
Wolfgang Bumiller
a171036789 add ESXi plugin for import and wire-it up to a new get_guest_import plugin fn
The get_guest_import fn is currently a private/internal API and should
not yet be exposed as a public part of the storage API.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-13 15:17:55 +01:00
Wolfgang Bumiller
a02056f8df bump pve-common dep to 8.1.1 for PVE::Systemd::unit_is_active
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-13 12:25:43 +01:00
Wolfgang Bumiller
bf453532b0 storage plugins: add 'import' content type
Add a new 'import' content type which will be the corner stone for a
better API and UI integrated way to import virtual guests into Proxmox
VE.

For starters this will be used to implement a ESXi adapter, so that
those VMs can get imported nicely.

Later we want to integrate the OVF/OVA import skeletons we got in
qemu-server to something more usable here.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 [ TL: add more commit message with some background ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-13 12:25:00 +01:00
Thomas Lamprecht
eaf6995afd bump version to 8.1.0
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-06 18:26:18 +01:00
Thomas Lamprecht
ee283b6d57 buildsys: cleanup all docs and shell comnpletion artefacts
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-05 11:26:30 +01:00
Thomas Lamprecht
8d060d6749 d/copyright: update years
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-01 10:13:24 +01:00
Thomas Lamprecht
71176c2b8c d/control: add dependency relation on libpve-access-control
We actually use it already (partially without perl use statement) and
the use of the new Sys.AccessNetwork privilege should be version
guarded too.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-02-28 15:51:25 +01:00
Thomas Lamprecht
17e0b01b7f fix #5254: api: allow usage of download-url with Sys.AccessNetwork
The download-url API endpoint has some implications that admins are
unaware of, namely that it basically allow to scan the whole network
via HTTP URLs, and potentially even download some image that the user
should not have access to and adding to a VM that the user controls.

That's why in addition to the Datastore.AllocateTemplate privilege on
the storage, the Sys.Modify on the whole Cluster was required to use
the API call. That design was chosen as we were not fully sure if a
separate privilege is warranted, but user feedback has shown that the
(not so big) cost of adding such a new privilege is justified.

Change the permission check to allow the combination of
Datastore.AllocateTemplate on the storage and either 'Sys.Modify' on
/, for backwards compatibility, or the newer 'Sys.AccessNetwork' on
the node that handles the download.
Using a node-specific ACL path allows admins to e.g. prepare one
specific node's firewall so that pveproxy can access only a safe set
of hosts via outgoing HTTP (not stemming from valid connection
tracking to the PVE API), and thus even further limit the privileges
of users or tools that are trusted to download images to a storage.

Buglink: https://bugzilla.proxmox.com/show_bug.cgi?id=5254
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-by: Hannes Duerr <h.duerr@proxmox.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-02-28 15:48:44 +01:00
Fiona Ebner
4abb327843 plugin schema: improve description of 'shared' property
It's not clear to users what this property does otherwise. Latest
report from the community forum:

https://forum.proxmox.com/threads/134393

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-01-30 12:42:36 +01:00
Fiona Ebner
0554c03683 plugin schema: improve description of 'nodes' property
The default description doesn't make it clear what the property is
used for in the context of storages.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-01-30 10:38:38 +01:00
Fiona Ebner
76aa80fc93 zfs: fix duplicate word typo in error message
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-01-30 10:25:41 +01:00
Thomas Lamprecht
a18a8b4b45 bump version to 8.0.5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-21 21:20:57 +01:00
Thomas Lamprecht
5840bd18a8 btrfs: fix calling parent create_base method in fall-back
If we want to forward to the create_base of the directory plugin while
making that use our $class for the operations that call might do, we
cannot use the -> notation (which would resolve the next actual
implementation) but rather pass the class directly.
But, DirPlugin reuses the create_base method from the base Plugin
method, so we also need to call that, because on direct call notation
the inheritance fallback to super methods isn't available.

Reported in the forum:
https://forum.proxmox.com/threads/95684/post-606535

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-19 20:11:36 +01:00
Thomas Lamprecht
3cd71f42c0 bump version to 8.0.4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-13 18:12:48 +01:00
Aaron Lauterer
e1348a2792 disks: get: add osdid-list return parameter
It is possible to run multiple OSD daemons on one disk. The new
'osdid-list' parameter returns an array of all OSD IDs found on the
disk.

The old 'osdid' parameter is kept for compatibility. We might want to
deprecate / remove it in the future.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2023-11-13 16:48:08 +01:00
Thomas Lamprecht
29ca4abaee d/control: bump versioned dependency for libpve-common-perl to >= 8.0.10
to ensure the new tar parameter for the pbs-client is known

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-13 16:44:46 +01:00
Dominik Csapak
96c579a262 api/filerestore: add 'tar' parameter to 'download' api
to be able to download 'tar.zst' archives

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-11-13 16:44:07 +01:00
Thomas Lamprecht
c4f6c2ff7a iscsi: code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-27 13:44:05 +02:00
Thomas Lamprecht
6f072aaaa8 iscsi: always recheck if iscsiadm binary is available
this avoids the need for restarting all services that have pve-storage
modules loaded after the admin installed open-iscsi.

while at it make it a bit more clear that this might die by using
assert in the method name.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-27 13:43:18 +02:00
Yuri Konotopov via pve-devel
90c1b10cc5 fix #254: iscsi: add support for multipath targets
With this patch Proxmox now tries to login to all discovered portals
in case some of them are not logged yet.
In case of multipath configuration when initially configured portal is
missing for some reason Proxmox don't lose iSCSI storage now and can
successfully restore iSCSI connection between reboots.

Signed-off-by: Yuri Konotopov <ykonotopov@gnome.org>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Dominik Csapak <d.csapak@proxmox.com>
2023-10-27 13:18:52 +02:00
Thomas Lamprecht
9bd1c95a71 offline migration: use telling variable for capture group
With that the if/else branches are pretty clear, so drop the rather
noisy comments that describe what one can get from the code already
anyway.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-08-31 15:21:46 +02:00
Leo Nunner
aa82ad5c25 fix #3004: show progress of offline migration in task log
dd supports a 'status' flag, which enables it to show the copied bytes,
duration, and the transfer rate, which then get printed to stderr.

Signed-off-by: Leo Nunner <l.nunner@proxmox.com>
2023-08-31 15:21:11 +02:00
Fiona Ebner
122ccde412 cifs: bubble up NT_STATUS_INVALID_PARAMETER during connection check
instead of claiming that the storage is not online.

Would've made the issue fixed by b27da68 ("cifs: fix check connection
call") more obvious, because (the UI passes along an empty string for
domain if not set and) the smbclient call returns that status with
> -W ''
in Bookworm.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-08-30 11:27:45 +02:00
Wolfgang Bumiller
2a85ebe260 bump version to 8.0.3
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-18 10:40:16 +02:00
Wolfgang Bumiller
d99d45930a whitespace and formatting fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-18 10:12:57 +02:00
Wolfgang Bumiller
444d262fd4 remove unneeded comment
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-18 10:06:26 +02:00
Philipp Hufnagl
2197ff97dc fix #4849: download-url: allow download and decompression of compressed ISOs
adds information for how to decompress isos.

generates the compressor regex from a list of comression formats (to
avoid redundancy)
extends the download_url wtih the functionality to handley compression
for images

Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
2023-08-18 09:58:06 +02:00
Philipp Hufnagl
57ec06621b fix whitespace
Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>

FG: dropped non-indentation parts

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-08-04 13:49:03 +02:00
Philipp Hufnagl
618bdfd927 clean: fix whitspaces and minor code issues
removed Data::Dumper and a newline

Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>

re-added the newline, but fixed up the indentation at that line..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-07-26 14:28:12 +02:00
Thomas Lamprecht
42c08f41cb bump version to 8.0.2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-28 09:32:00 +02:00
Fiona Ebner
b27da68320 cifs: fix check connection call
by not having -W eat the next argument when $scfg->{domain} is defined
but empty.

As reported in the community forum [0], this could lead to issues
adding a storage in the UI.

[0]: https://forum.proxmox.com/threads/44983/post-566842

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-26 20:31:25 +02:00