5
0
mirror of git://git.proxmox.com/git/pve-storage.git synced 2025-01-11 05:18:01 +03:00
Commit Graph

999 Commits

Author SHA1 Message Date
Thomas Lamprecht
902a6bb379 bump version 5.0-37
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-02-01 12:39:32 +01:00
Dominik Csapak
4ec588fe92 allow snippets by default for new dir storages
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-01-31 11:04:29 +01:00
Dominik Csapak
d1eb35ea74 enable snippets content type for all directory based storages
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-01-31 11:04:29 +01:00
Dominik Csapak
7c7ae12f43 add new content type 'snippets'
will be used to contain files which can be executed as hookscripts or
contain custom cloud-init configs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-01-31 11:04:29 +01:00
Thomas Lamprecht
931c35cfa0 fix #1598: use glusterfs daemon default port for online check
use the port where the main glusterfs daemon listens on as ping port,
this one is also used by QEMU as default.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-01-29 08:27:27 +01:00
Christian Ebner
712e27f178 Fix #1941: remove empty directories when freeing image on FS based storages
Remove directories if they are empty, which can happen if all images
from a VM got deleted, e.g., after destroying said VM.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-01-24 15:09:20 +01:00
Stoiko Ivanov
4e8de9ad56 Fix #2050: only provide 'conv=sparse' for LvmThin
LVMPlugin->volume_import (used by storage_migrate on either offline
migration with local disks, or online migration with storage-only
referenced disks) passed 'conv=sparse' to `dd`. This can lead to
data-corruption, if the target volume is not zero-initialized.

dropping the sparse argument completely would fix the problem, but
breaks keeping data sparse for LvmThinPlugin.

This patch moves the dd out into (LVM*) plugin specific sub so that
each can control the parameters.

Steps for reproducing the issue:
* create a cluster with (at least) 2 nodes A and B, with a free
  disk-device (/dev/sdx)
* write a recognizable pattern to /dev/sdx on B:
  `dd if=/dev/zero bs=10M | tr '\000' '\255' | dd of=/dev/sdb bs=10M`
  (would be grateful for alternatives to the dd| tr| dd)
* on both A and B create a lvm-vg (pvcreate, vgcreate)
* add it as _not_ shared storage, which is available on nodes A and B
* create a small guest on A
* fill a file in the guest with zeros
  `dd if=/dev/zero of=/zerofil bs=10M`
* stop the guest, migrate it to B
* start the guest - check that the file `/zerofil` contains `ad`
  instead of `00`

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-01-18 10:46:33 +01:00
Thomas Lamprecht
a31cb84fd4 update version to 5.0-36
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-01-15 13:18:28 +01:00
Stoiko Ivanov
628a921a94 LVM: Add '--refresh' when activating volumes
From `man 8 lvchange`:
  --refresh
      If the logical volume is active, reload its metadata. This is not
      necessary in normal operation, but may be useful ... if you're doing
      clustering manually without a clustered lock manager.

Fixes migration in a shared LVM (iscsi) setup, where a disk gets resized on one
node A and the guest is afterwards migrated to another node B: B still presents
the old size to the guest, leading to data corruption.

It is necessary to run `lvchange` twice because the options `-ay` and
`--refresh` are mutually exclusive.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2019-01-15 09:43:07 +01:00
Stoiko Ivanov
955c1f2cf7 fix #2046 add volume_size_info to LVMPlugin
Without volume_size_info a Storage plugin falls back to the Implementation
in PVE/Storage/Plugin.pm, which relies on `qemu-img info`.

`qemu-img info` returns wrong results on a node in the case of shared volume
groups (e.g. when sharing disks via iSCSI), if a disk was resized on another
node (it lseeks to the end of the block-device, and this yields the old size).

Using lvs directly fixes the issue, since the LVM metadata gets updated when
invoking lvs.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2019-01-15 09:43:07 +01:00
Thomas Lamprecht
4b3088a0a8 get_monaddr_list: also ensure that returned 'mon addr' are defined
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-01-09 15:23:44 +01:00
Alwin Antreich
187df8553e ceph: get_monaddr_list: exclude general monitor section
Else, if a general MON section existed in the ceph.conf, we added a
undefined entry and a cephfs storage can't be mounted anymore.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-01-09 13:40:21 +01:00
David Limbeck
5b5534a9d7 fix use of uninitialized value in parse_ceph_config
Signed-off-by: David Limbeck <d.limbeck@proxmox.com>
2019-01-03 10:26:57 +01:00
Thomas Lamprecht
8e3c05c1b9 bump version to 5.0-35
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-12-20 10:50:45 +01:00
Thomas Lamprecht
ab3e0a7fbe d/control: bump pve-cluster version dependency to 5.0-32
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-12-20 10:49:06 +01:00
Dominik Csapak
71be011328 register ceph.conf parser/writer
With this we can use cfs_read_file/cfs_write_file and cfs_lock_file.

Code for writing is mostly copied from pve-managers CephTools.pm,
with the addition of mgr sections.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-12-20 09:26:11 +01:00
Dominik Csapak
4050fcc16b move Storage/CephTools to CephConfig
it is not a storage plugin, and it makes more sense to have it
top-level, but there we cannot name it CephTools because of the
existing ones in pve-manager

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-12-20 09:26:11 +01:00
Thomas Lamprecht
36d4af9ba8 bump version to 5.0-34
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-12-14 12:22:22 +01:00
Wolfgang Bumiller
c3442aa554 Fix #2020: use /sys to map nvmeXnY to nvmeX
`nvmeX` devices nodes are apparently allocated independently
from their namespace block devices `nvmeXnY` and therefore
they are not strictly related by name. For instance:
  $ readlink /sys/block/nvme0n1/device
  ../../nvme1
  $ readlink /sys/block/nvme1n1/device
  ../../nvme0

Here /dev/nvme0n1 is the first namespace of /dev/nvme1 while
/dev/nvme1n1 is the first namespace of /dev/nvme0.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-10 14:54:11 +01:00
Alwin Antreich
d35a0b4b62 Fix #2019: CephFS storage misses maxfiles
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-12-07 13:47:05 +01:00
Thomas Lamprecht
434c91b645 bump version to 5.0-33
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-27 13:23:48 +01:00
Thomas Lamprecht
54e0b0034b cephfs: tell systemd that mount mount requires network
As we mount this manually and thus systemd doesn't know about any
dependency for cephFS mounts, this got umounted only at the last
stage of shutdown, where network wasn't active anymore.

But, CephFS needs to be connected to an active MDS for a clean
unmount so without network this mount would delay shutdown for quite
a bit, until after some minutes systemd gave up and forced unmount.

So tell systemd that this mount requires network, which can be done
with the '_netdev'[0] mount option, that lucky for us can be also
passed to a mount call and isn't only available for fstab.

with this a mount gets, among others:
> Wants=network-online.target
> Before=umount.target remote-fs.target
> After=remote-fs-pre.target system.slice network.target network-online.target -.mount

Which does the trick for us.

[0]: https://www.freedesktop.org/software/systemd/man/systemd.mount.html#_netdev

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-27 12:10:36 +01:00
Thomas Lamprecht
957321a86e pvesm: add scan subcommands
Change to a cleaner sub command interface grouping all scan commands.

Alias to old command names for backward compatibility
Best viewed with the ignore whitespace/indent change '-w' flag from
git

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-22 14:11:38 +01:00
Thomas Lamprecht
95de09ae15 bump version to 5.0-32
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-19 15:27:32 +01:00
Thomas Lamprecht
39581b5240 d/control: bump pve-common dependency and break older pve-manager
To ensure that we have the new SysFSTools available and as previously
we hosted the API scan endoints which pve-manager registered so
record the break of older versions.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-19 14:44:51 +01:00
Dominik Csapak
98bf79f78b remove usb scan code
this is now in PVE::SysFSTools

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-11-19 13:20:04 +01:00
Dominik Csapak
a0965edde7 remove PVE/API2/Storage/Scan.pm
since those are now defined in pve-manager

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-11-19 13:20:04 +01:00
Dominik Csapak
7963ba74bb move Scan API calls from PVE/API2/Storage/Scan.pm to pvesm.pm
since the calls for the real API are defined now in pve-managers Scan.pm

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-11-19 13:20:04 +01:00
Thomas Lamprecht
a0908caa99 APIAGE followup: fix typo and print versions in error message
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-19 12:18:03 +01:00
Wolfgang Bumiller
042dd4be1f plugin loader: add an APIAGE
With the addition of the map/unmap_volume() methods we made
an (actually unnecessary) API version bump.
All current users of these methods fall back to path() when
they return undef, so plugins implementing version 1 are
in fact compatible currently. (In fact, the default
Plugin::map_volume() could fall back to it on its own, but
doesn't currently).

For now let's just allow plugins older plugins to also be
loaded by introducing an API age variable. With it, if we
have a reason to break older plugins, we can have a
deprecation period during which older plugins cause a
warning instead of refusing to load altogether.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-11-19 10:47:46 +01:00
Thomas Lamprecht
49cc7802f7 LVM: lock on volume_resize
This is important for shared LVM storages. As with deletes and
creates of images, as else we may have not the up-to-date metadata
and extents may get reused if another node created an image during
the same time, for example.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-15 10:13:57 +01:00
Thomas Lamprecht
a4bdab17bb fix #862: do not resolve portal adress on storage add
as described in #862:

> I experienced a problem with ISCSI portal when using a hostname and
> not IP.
> The GUI resolves the hostname to an IP and writes it to storage.cfg.
> As my setup requires hostnames, i needed to change the config
> manually back to the hostname which is working fine.
>
> Why is this conversion done? If I enter a hostname, i want to have a
> hostname. If i enter an IP address i want to have an IP address.

This makes sense to me, a feature of using domains is that they
are/should be resolved when actually using (i.e., connecting to them)
so resolving it once on add does not seems like a good idea (if I do
not miss something - as this is a classic "imported from SVN" I do
not have any rationale to look at).

So save the work and pass it as is.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-15 10:06:19 +01:00
Thomas Lamprecht
5124548d5d bump version to 5.0-31
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-09 17:26:34 +01:00
Dietmar Maurer
40d698932e implement map_volume and unmap_volume
This allows to request a mapped device/path explicitly, regardles of
the storage option, eg. krbd option in the RBDplugin.

Bump of the storage ABI => 2

Co-authored-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2018-11-09 17:25:51 +01:00
Thomas Lamprecht
8b6b710265 folowup: fix whitespace errors and s/and/&&/ for consistency
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-09 15:27:08 +01:00
Tim Marx
5b4b715771 storage zfs: removed unused variable
Signed-off-by: Tim Marx <t.marx@proxmox.com>
2018-11-09 14:49:43 +01:00
Tim Marx
32f749b840 storage zfs: changed return value description & optionals
Signed-off-by: Tim Marx <t.marx@proxmox.com>
2018-11-09 14:49:43 +01:00
Tim Marx
a49fc735e5 close #1949: storage zfs: changed zpool command parser
Signed-off-by: Tim Marx <t.marx@proxmox.com>
2018-11-09 14:49:43 +01:00
Tim Marx
b005f2f483 Fix: api zfs: changed return value name to errors
Signed-off-by: Tim Marx <t.marx@proxmox.com>
2018-10-29 10:33:39 +01:00
Thomas Lamprecht
e6c4144fa6 d/control: record pve-cluster build dependency
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-10-18 13:25:34 +02:00
Thomas Lamprecht
db59cd4493 bump version to 5.0-30
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-10-03 17:02:55 +02:00
Thomas Lamprecht
a298e55665 remove outdated unused API2::Storage::Replication module
this was not installed, had references to non-existing modules
(e.g., PVE::ReplicationTools) and the things it probably was intended
for are done in pve-manager, which has full access to all pve perl
libs and API methods (from a dependency POV)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-10-03 16:58:57 +02:00
Thomas Lamprecht
0370861cfd diskmanage: rename check_unused to assert_disk_unused
to have a clear method name for this. check_XYZ suggests also that we
return true if the check was OK, but we don't.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-10-03 14:51:38 +02:00
Thomas Lamprecht
9280153e10 rename check_available to assert_sid_unused
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-10-03 14:49:14 +02:00
Dominik Csapak
4dcb16c0dc fix #1929: only check storage if user want to create one
this is useful if a user wants to create similar storage on each host

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-10-03 14:30:22 +02:00
Dominik Csapak
820bab50b9 add missing storage check in LVM Disk API
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-10-03 14:30:22 +02:00
Dominik Csapak
76c1e57be7 refactor disk/storage checks for Disk API
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-10-03 14:30:22 +02:00
Dietmar Maurer
94836d7b8c pvesm iscsiscan: fix cli parameters
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2018-10-01 10:56:18 +02:00
Thomas Lamprecht
c9fc2bc994 bump version to 5.0-29
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-09-19 11:21:37 +02:00
Thomas Lamprecht
dd9e97ed14 find_free_diskname: fixup regex match operator
Co-developed-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-09-19 11:21:37 +02:00