5
0
mirror of git://git.proxmox.com/git/pve-storage.git synced 2025-02-26 21:57:31 +03:00

516 Commits

Author SHA1 Message Date
Dominik Csapak
571b6f2638 add default rotational value
because if the file does not exist,
we have an perl error for comparing an uninitialized
value

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-10-28 10:20:28 +02:00
Dominik Csapak
865bdbd917 use model from udevadm
we want this, because the model in /sys/block/<device>/device/model
is limited to 16 characters

and since the model is not always in the udevadm output (nvme),
also read the model from the model file as fallback

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-10-28 10:19:40 +02:00
Dominik Csapak
10a48db55c make dir_is_empty a proper sub
this allows us later to mock the sub,
which we need for testing

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-10-28 10:19:40 +02:00
Dominik Csapak
461a9fd8da move directory test into get_sysdir_info
because it logically belongs there, also
this makes the testing easier

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-10-28 10:19:40 +02:00
Dominik Csapak
532e89e7be use /sys/block/ path for udev instead of name
since we iterate over the entries in /sys/block
it makes sense to use this path

this should fix #1099
because udevadm does not take
 -n cciss!c0d0 (because it only looks in dev for this)
but takes
 -p /sys/block/cciss!c0d0

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-10-28 10:19:40 +02:00
Fabian Grünbichler
8b5ccc06b7 fix #1165: only check mount status when is_mountpoint is set
otherwise the status() method returns garbage for non-mount
point directory storages.
2016-10-13 08:29:30 +02:00
Wolfgang Bumiller
d57fb43d9b disks: fix warnings 2016-10-07 09:28:06 +02:00
Dominik Csapak
a0028cf97e allow rbd images < 1M to be detected
without this, having an efidisk on a ceph storage
prevents creating another disk on the same
ceph storage, because it will not be detected
and we try to allocate one with the same name

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-10-07 08:17:40 +02:00
Dominik Csapak
38ddd4ce8e correctly parse cciss devices in get_disks
cciss maps the devices to
/sys/block/cciss!cXdY

and not to

/sys/block/cXdY

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-10-06 07:50:05 +02:00
Dominik Csapak
6965a6701e fix #1135: refactor wear level parsing
refactored the wear level parsing into its
own function, where we can now define a
vendor <-> attribute id
mapping

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-10-06 07:49:30 +02:00
Dominik Csapak
dd902da78e merge get_smart_data/health
instead of parsing the output of smart in two places,
give get_smart_data a flag if we only want health

this fixes a bug (not on the bugtracker), where
an ssd with disabled smart had an empty string as health
in the gui

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-10-06 07:48:57 +02:00
Fabian Grünbichler
c9bd3d22b2 fix #1123: modify NVME device path for SMART support
seems like we need to drop the namespace part before calling
smartctl.

note that NVME devices require smartmontools >= 6.5
2016-10-04 10:35:30 +02:00
Fabian Grünbichler
16bf963b94 add return schema to smart API path 2016-10-04 10:34:47 +02:00
Fabian Grünbichler
dc1311cbe8 fix #1120: non-ATA disks with unstructured smart data
simple parser as fallback for non-ATA disks, just collect
all lines in the "SMART DATA" section.
2016-10-04 10:34:22 +02:00
Fabian Grünbichler
5db2d529ed fix #1120: SCSI SMART health parsing
SCSI disks report their health as "SMART Health Status: OK"
2016-10-04 10:34:06 +02:00
Fabian Grünbichler
7a98a62dbc disks: allow get_disks without smart checks
the smart checks are only needed for the API call(s) that
list all disks and their status, but get_disks is also used
in disk usage checks and in the Ceph code, where the smart
status is completely irrelevant.

drop the implicit skipping of smart checks if $disk is set,
since we have an explicit parameter for this now.
2016-10-04 10:33:49 +02:00
Fabian Grünbichler
1993540bf8 fix #1122: correctly create LUNs for linked clones 2016-09-29 08:42:26 +02:00
Fabian Grünbichler
e3b02ffe6e disks: fix typo 2016-09-29 08:42:15 +02:00
Fabian Grünbichler
1c99955364 disks: parse smart attributes using RE 2016-09-29 08:42:06 +02:00
Fabian Grünbichler
0c486b09df disks: use smartctl -H -A
to only list SMART health and attributes, instead of
"smartctl -a", which prints "all SMART information"
2016-09-29 08:41:31 +02:00
Fabian Grünbichler
acd3d91649 move SMART error handling into get_disks
because we never ever want to die in get_disks because of a
single disk, but the nodes/xyz/disks/smart API path is
allowed to fail if a disk device is unsupported by smartctl
or something else goes wrong.
2016-09-29 08:40:19 +02:00
Wolfgang Bumiller
d547f26c7d Fix #1012: dir storage: add is_mountpoint option
While the mkdir option deals with the case where we don't
want to clobber a mount point with directories (like ZFS,
gluster or NFS), putting a directory storage directly onto a
mount point is still risky:
If the path exists - which it usually does even if not
mounted - the storage will be considered successfully
activated, but empty (or with unexpected content). Some
operations will then lead to unexpected problems: the
free_disk operation for instance only warns if the disk does
not exist, but does not throw an error. In this case the
configuration might be updated without the real disk being
deleted. Once it's mounted back in, later operations which
check existing disks which are not part of the current VM
configuration (like migration) might error unexpectedly.

This adds an 'is_mountpoint' option to directory storages
which assumes the directory is an externally managed mount
point (eg. fstab or zfs) and changes activate_storage() to
throw an error if the path is not mounted.
2016-09-27 09:56:55 +02:00
Wolfgang Bumiller
c7616abcb2 path based storages: improve the mkdir option
So far this only prevented the creation of the toplevel
directory. This does not cover all problem cases,
particularly when said directory is supposed to be a mount
point, including NFS and glusterfs beside ZFS.

The directory based storages we have already use mkpath
whenever they need to create files, and for actions on files
which are supposed to exist it's fine if it errors out.
So it should also be safe to skip the creation of standard
subdirectories in activate_storage().

Additionally NFS and glusterfs storages should also accept
the mkdir option as they otherwise may exhibit similar
issues, eg. when an NFS storage is mounted onto a directory
inside a ZFS subvolume.
2016-09-27 09:54:53 +02:00
Dietmar Maurer
ff3badd83f white space cleanups 2016-09-26 13:40:43 +02:00
Wolfgang Link
a9ef8ffb16 Avoid JavaScript gets a string 0.
If the JavaScript gets a "0" it convents it to a boolean false.
So to ensure the GUI always get valid int we cast the values.
2016-09-26 13:38:56 +02:00
Fabian Grünbichler
787624dfc0 add comments about LVM thin clones 2016-09-15 14:01:07 +02:00
Fabian Grünbichler
5510f5c9f9 fix typo 2016-09-15 13:56:17 +02:00
Fabian Grünbichler
1b83c3d9c7 harmonize list_images code 2016-09-15 13:54:47 +02:00
Fabian Grünbichler
17fb7e4215 move check for existing clones into own method
and change its return type to boolean
2016-09-15 13:52:57 +02:00
Fabian Grünbichler
9924228be1 remove unused method
only used by test case, which should use what the rest of
the codebase uses as well
2016-09-15 13:42:55 +02:00
Dietmar Maurer
cfd58f1fcc code cleanup 2016-09-14 11:31:10 +02:00
Fabian Grünbichler
3718e83ab5 fix error message 2016-09-14 11:24:06 +02:00
Fabian Grünbichler
883d9b81f0 fix indentation 2016-09-14 11:23:52 +02:00
Fabian Grünbichler
9690e55e9b rbd: detect linked clones/base volumes correctly
since the rbd images themselves are named differently than
the volumes in our config files, we need to recreate this
information from the parent relation in the ceph metadata,
otherwise list_images() might return wrong volume names/IDs

since list_images is used by PVE::Storage::vdisk_free() to
check for children still referencing a base image, because
of the wrong volume id RBDPlugin->parse_volname() does not
detect the base image of linked clones and the check fails.
this is thankfully mitigated by the protected status of the
base snapshot, but creates a rather confusing error message.

scenario (VM 701 is a linked clone of template VM 700):

$ qm config 700 | grep virtio0:
virtio0: ceph_qemu:base-700-disk-1,size=2G
$ qm config 701 | grep virtio0:
virtio0: ceph_qemu:base-700-disk-1/vm-701-disk-1,size=2G

before (pvesm list reports wrong volume ID, check fails):

$ pvesm list ceph_qemu
ceph_qemu:base-700-disk-1   raw 2147483648 700
ceph_qemu:vm-701-disk-1     raw 2147483648 701
$ pvesm free ceph_qemu:base-700-disk-1
snap_unprotect: can't unprotect; at least 1 child(ren) in pool rbd
rbd unprotect base-700-disk-1 snap '__base__' error: snap_unprotect: can't unprotect; at least 1 child(ren) in pool rbd

after (correct volume ID, check works as intended):

$ pvesm list ceph_qemu
ceph_qemu:base-700-disk-1                   raw 2147483648 700
ceph_qemu:base-700-disk-1/vm-701-disk-1     raw 2147483648 701
$ pvesm free ceph_qemu:base-700-disk-1
base volume 'base-700-disk-1' is still in use (use by 'base-700-disk-1/vm-701-disk-1')
2016-09-14 11:23:39 +02:00
Fabian Grünbichler
5e6aa346c7 rbd: use correct key to access hash elements 2016-09-14 11:17:16 +02:00
Dominik Csapak
9018a4e639 do not automatically die on smartctl exit code > 0
since smartctl uses the return value to encode
disk health status (such as failure in the past)
we cannot die there, but have to parse the returncode

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-09-08 16:52:33 +02:00
Dietmar Maurer
2622a5ca2d sheepdog 1.0 changed the path from /usr/sbin/dog to /usr/bin/dog 2016-09-07 09:11:43 +02:00
Dominik Csapak
409f8203e0 add api entries for disk management
adds a new class (intended to be used under nodes in pve-manager)
which adds the three api calls: list, smart and init

list being a general list of the available disk with infos
smart being a call to get the smart data from a given device
init being a call to write a gpt header to an unused disk

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-09-05 13:49:28 +02:00
Dominik Csapak
cbba9b5b9c add Diskmanage Utilities
this adds the functions for listing the disks (mostly copied from
the ceph code), checking if a disk is a valid blockdevice, if it
is used/in a zfs pool/as an lvm pv, and an init function (just to add a gpt header;
this is important if one wants to use a fresh disk for ceph journals)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-09-05 11:31:19 +02:00
Dietmar Maurer
6d2b278c51 white space cleanups 2016-09-01 06:28:54 +02:00
Dietmar Maurer
e7ac2d5cf6 rbd_unittobytes: use a local var instead of a sub 2016-09-01 06:24:51 +02:00
Alexandre Derumier
134172255f rbd: allow to use client custom ceph conf for each storeid
If you want to use different ceph storage,
something they have differents values like ms_nocrc = true.(they are also others ones).

The client need to specify theses special options to be able to connect

This patch allow to create a ceph config file for each storeid in

/etc/pve/priv/ceph/$storeid.conf

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2016-09-01 06:21:40 +02:00
Dmitry Petuhov
4dee23d305 Add support for custom storage plugins
PVE team cannot support specialized vendor-specific storage
plugins because of lack of hardware. But we can allow users to
add own plugins for their storages without need to rewrite any
PVE code and thus ease PVE updates to them.

Idea of this patch is to add folder /usr/share/perl5/PVE/Storage/Custom
where user can place his plugins and PVE will automatically load
them on start or warn if it could not and continue. Maybe we could
even load all plugins (except PVE::Storage::Plugin itself) this way,
because current storage plugins are not really plugins, if they
need to be explicitly loaded in PVE code :-).

Custom plugins MUST have api() method returning version for which
it was designed. If API changes from PVE side, module is just not
being registered and warnig message is printed do log, so user have
to update module. Until module update, corresponding storage will
just disappear from PVE, so it shall not impose any data damage
because of API change.

This approach works (with some limitations) if plugin works in
generic PVE way: full control of volumes lifecycle. And will not
currently work for custom plugins like iSCSI, which needs to select
pre-existing volumes. Maybe someone will add more flexible way to
pve-manager to select input elements for storage plugins to target
this.

Currently tested with my NetApp plugin.

Signed-off-by: Dmitry Petuhov <mityapetuhov@gmail.com>
2016-08-26 15:44:49 +02:00
Dominik Csapak
f3b3b2a3b7 remove compression option from lvm migration
ssh(1) mentions that compression is only disirable on slow
connections.

since migration from cluster node to cluster node needs a
fast network anyway, we can drop the compression for
a speed improvement

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-08-05 08:36:13 +02:00
Fabian Grünbichler
82fc923fd4 fix spelling / grammar 2016-07-13 13:59:26 +02:00
Wolfgang Bumiller
e6ccfdeb21 Remove unused pve-storage-monhost format 2016-07-11 13:58:56 +02:00
Wolfgang Bumiller
e858048fc5 rbd: use pve-storage-portal-dns-list for monhost
This way we get parameter verification on monitor addresses
as well as the ability to pass multiple `--monhost`
arguments to `pvesm add`.

Since our '-list' schemas default to using commas we now
need to properly support these, so all uses of the monhost
property now replace all of semicolon, space or comma into
the currently required character.
This should fix the issues reported by Alwin Antreich on the
pve-user list.

Since this schema supports both ipv6+port notations we need
to make sure we convert to the bracket enclosed variant.
Added a helper for this.
2016-07-11 13:58:56 +02:00
Wolfgang Bumiller
33cef4c84e rbd: path: don't build the entire path if we don't use it 2016-07-11 13:58:56 +02:00
Fabian Grünbichler
0423e8c686 fix indentation 2016-06-29 11:42:03 +02:00
Fabian Grünbichler
7a9dd1195d add tagged_only option to LVM storage
to filter volumes by the 'pve-vm-ID' tag, which is set on
all volumes created via the PVE storage layer.
2016-06-29 11:42:03 +02:00