1
0
mirror of git://sourceware.org/git/lvm2.git synced 2024-12-21 13:34:40 +03:00
Commit Graph

17182 Commits

Author SHA1 Message Date
David Teigland
e4615533f7 tests: for devices file 2021-02-11 17:46:20 -06:00
David Teigland
2adf2f9fc1 tests: allow tests to use real devices
user creates a file listing real devices they want
lvm tests to use, and sets LVM_TEST_DEVICE_LIST.
lvm tests can use these with prepare_real_devs
and get_real_devs.
Other aux functions do not work with these devs.
2021-02-11 17:46:20 -06:00
David Teigland
d61838df49 device usage based on devices file
The devices file /etc/lvm/devices/system.devices is a list of
devices that lvm can use.  This is the default system devices
file, which is specified in lvm.conf devices/devicesfile.

The command option --devicesfile <filename> allows lvm to be
used with a different set of devices.  This allows different
applications to use lvm on different sets of devices, e.g.
system devices do not need to be exposed to an application
using lvm on its own devices, and application devices do not
need to be exposed to the system.

In most cases (with limited exceptions), lvm will not read or
use a device not listed in the devices file.  When the devices
file is used, the regex filter is not used, and the filter
settings in lvm.conf are ignored.  filter-deviceid is used
when the devices file is enabled, and rejects any device that
does not match an entry in the devices file.

Set use_devicesfile=0 in lvm.conf or set --devicesfile ""
on the command line to disable the use of a devices file.
When disabled, lvm will see and use any device on the system
that passes the regex filter (and other standard filters.)

A device ID, e.g. wwid or serial number from sysfs, is a
unique ID that identifies a device.  The device ID is
generally independent of the device content, and lvm can
get the device ID without reading the device.
The device ID is used in the devices file as the primary
method of identifying device entries, and is also included
in VG metadata for PVs.

Each device_id has a device_id_type which indicates where
the device_id comes from, e.g. "sys_wwid" means the device_id
comes from the sysfs wwid file.  Others are sys_serial,
mpath_uuid, loop_file, md_uuid, devname.  (devname is the
device path, which is a fallback when no other proper
device_id_type is available.)

filter-deviceid permits lvm to use only devices on the system
that have a device_id matching a devices file entry.  Using
the device_id, lvm can determine the set of devices to use
without reading any devices, so the devices file will constrain
lvm in two ways:
1. it limits the devices that lvm will read.
2. it limits the devices that lvm will use.

In some uncommon cases, e.g. when devices have no unique ID
and device_id has to fall back to using the devname, lvm may
need to read all devices on the system to determine which
ones correspond to the devices file entries.  In this case,
the devices file does not limit the devices that lvm reads,
but it does limit the devices that lvm uses.

pvcreate/vgcreate/vgextend are not constrained by the devices
file, and will look outside it to find the new PV.  They assign
the new PV a device_id and add it to the devices file.  It is
also possible to explicitly add new PVs to the devices file before
using them in pvcreate/etc, in which case these commands would not
need to look outside the devices file for the new device.

vgimportdevices VG looks at all devices on the system to find an
existing VG and add its devices to the devices file.  The command
is not limited by an existing devices file.  The command will also
add device_ids to the VG metadata if the VG does not yet include
device_ids.  vgimportdevices -a imports devices for all accessible
VGs.  Since vgimportdevices does not limit itself to devices in
an existing devices file, the lvm.conf regex filter applies.
Adding --foreign will import devices for foreign VGs, but device_ids
are not added to foreign VGs.  Incomplete VGs are not imported.

The lvmdevices command manages the devices file.  The primary
purpose is to edit the devices file, but it will read PV headers
to find/check PVIDs.  (It does not read, process or modify VG
metadata.)

lvmdevices
. Displays devices file entries.
lvmdevices --check
. Checks devices file entries.
lvmdevices --update
. Updates devices file entries.
lvmdevices --adddev <devname>
. Adds devices_file entry (reads pv header).
lvmdevices --deldev <devname>
. Removes devices file entry.
lvmdevices --addpvid <pvid>
. Reads pv header of all devices to find <pvid>,
  and if found adds devices file entry.
lvmdevices --delpvid <pvid>
. Removes devices file entry.

The vgimportclone command has a new option --importdevices
that does the equivalent of vgimportdevices with the cloned
devices that are being imported.  The devices are "uncloned"
(new vgname and pvids) while at the same time adding the
devices to the devices file.  This allows cloned PVs to be
imported without duplicate PVs ever appearing on the system.

The command option --devices <devnames> allows a specific
list of devices to be exposed to the lvm command, overriding
the devices file.

TODO:
. device_id_type for other special devices (nbd, drbd, others?)
. dmeventd run commands with --devicesfile dmeventd.devices
. allow operations with duplicate pvs if device id and size match only one dev
2021-02-11 17:46:04 -06:00
David Teigland
834cba000a hints: clean up comment and init settings
no behavior change
2021-02-02 15:14:05 -06:00
David Teigland
cb54d0801d cachevol: allow forced detaching of damaged or invalid cachevol
A cachevol can be forcibly detached when it's missing devices.
Also allow this if it's damaged/invalid and unrepairable.
This would be needed to recover data from the origin LV after
a cachevol is lost or damaged beyond repair.
2021-02-02 14:31:23 -06:00
Zdenek Kabelac
ae2af1d5ed test: check read_only_volume_list tagging works 2021-02-02 21:23:39 +01:00
Zdenek Kabelac
51c83f1483 lvcreate: use lv_passes_readonly_filter
Check if created LV is going to be activated read-only
because such LV cannot be zeroed (equals to use
option '-pr').
2021-02-02 21:23:39 +01:00
David Teigland
8454ce66c5 writecache: let block_size setting override device block sizes
In cases where lvconvert does not detect a fs block size on the
device, it falls back to choosing a writecache block size based
on the device's LBS and PBS (tries to match those.)

If the user specifies a writecache block size on the command
line (--cachesettings block_size=4096|512), lvconvert currently
fails and reports an error if the user-specified value does not
match the value lvconvert would have chosen based on LBS and PBS.

The purpose of allowing a user-specified value on the command line
is to override what lvconvert would otherwise do, so change this
to just print a warning that the user value does not match the
value that would be chosen based on the LBS/PBS, and then take
the user-specified value as the writecache block size.
2021-02-02 13:52:31 -06:00
David Teigland
bee9f4efdd filter-mpath: work with nvme devices
Recognize when a device is nvme, and apply filter-mpath to
nvme devices in addition to scsi devices.
2021-02-02 13:01:20 -06:00
Zdenek Kabelac
48dfc388f7 tests: increase required target version
Require 1.7.0 for raid0.
2021-02-01 20:10:07 +01:00
Zdenek Kabelac
6e8a32e4fd tests: avoid killing test machines with kernel bug
Bug 1916891  still applies:

https://bugzilla.redhat.com/show_bug.cgi?id=1916891
2021-02-01 15:02:38 +01:00
Zdenek Kabelac
bfafd2b4e9 tests: check for raid target support 2021-02-01 15:01:04 +01:00
Zdenek Kabelac
026d94d882 gcc: hide uninitialized warning with older gcc 2021-02-01 14:54:49 +01:00
Zdenek Kabelac
48030389ce libdm: add dm_tree_node_add_thin_pool_target_v1
Supports thin-pool without crop.
2021-02-01 14:52:32 +01:00
Zdenek Kabelac
25b942a8ac aux: update thin_restore_needs_more_volumes
Version 0.8.5-2.el7 needs to return 0;
2021-02-01 12:13:49 +01:00
Zdenek Kabelac
b26e565ee5 tests: check for cache support presence 2021-02-01 12:13:49 +01:00
Zdenek Kabelac
19ddad12ca tests: update thin and cache checked messages
Slightly changed messages for allocation of pool metadata.
2021-02-01 12:13:49 +01:00
Zdenek Kabelac
b17b556e26 tests: check 16G thin pool metadata size 2021-02-01 12:13:49 +01:00
Zdenek Kabelac
9fa4c099fb lvconvert: ensure device_name is defined 2021-02-01 12:13:49 +01:00
Zdenek Kabelac
a9fd207192 pvscan: enhance _count_pvid_files_from_lookup_file
Ensure all vars are always properly defined in all paths.
2021-02-01 12:13:49 +01:00
Zdenek Kabelac
3acf6040b5 wipe: reformat message for failure case
Use the same error message layout to match BLKZEROUT look.
Makes testing easier.
2021-02-01 12:13:49 +01:00
Zdenek Kabelac
8f5695fa71 WHATS_NEW: updates 2021-02-01 12:13:49 +01:00
Zdenek Kabelac
be0bf43d74 allocation: report allocation error instead of crash
Current allocation limitation requires to fit metadata/log LV on
a single PV. This is usually not a big problem, but since
thin-pool and cache-pool is using this for allocating extents
for their metadata LVs it might be eventually causing errors
where the remaining free spaces for large metadata size is spread
over several PV.
2021-02-01 12:13:49 +01:00
Zdenek Kabelac
45f0c48365 pvmove: automatically resolve whole stacked LV
When passing 'pvmove --name arg' try to automatically move
all associated dependencies with given LV.

i.e. 'pvmove --name thinpool vg vgnew'
moves all thins and data and metadata LV into a new VG vgnew.
2021-02-01 12:06:13 +01:00
Zdenek Kabelac
abc9265a06 cache: reuse code for metadata min_max
Use update_pool_metadata_min_max() which is shared with
thin-pool metadata min-max updating.

Gives improved messages when converting volumes to metadata.
2021-02-01 12:06:13 +01:00
Zdenek Kabelac
f96b455506 pool: limit pmspare to 16GiB
There is not much point to let allocate more then this size
even when i.e. converted LV is bigger then 16GiB (%extent_size)
ATM neither thin-pool nor cache-pool supports bigger metadata.
2021-02-01 12:06:13 +01:00
Zdenek Kabelac
b4212be2e7 thin: improve 16g support for thin pool metadata
Initial support for thin-pool used slightly smaller max size 15.81GiB
for thin-pool metadata. However the real limit later settled at 15.88GiB
(difference is ~64MiB - 16448 4K blocks).

lvm2 could not simply increase the size as it has been using hard cropping
of the loaded metadata device to avoid warnings printing warning of kernel
when the size was bigger (i.e. due to bigger extent_size).

This patch adds the new lvm.conf configurable setting:
allocation/thin_pool_crop_metadata
which defaults to 0 -> no crop of metadata beyond 15.81GiB.
Only user with these sizes of metadata will be affected.

Without cropping lvm2 now limits metadata allocation size to 15.88GiB.
Any space beyond is currently not used by thin-pool target.
Even if i.e. bigger LV is used for metadata via lvconvert,
or allocated bigger because of to large extent size.

With cropping enabled (=1) lvm2 preserves the old limitation
15.81GiB and should allow to work in the evironement with
older lvm2 tools (i.e. older distribution).

Thin-pool metadata with size bigger then 15.81G is now using CROP_METADATA
flag within lvm2 metadata, so older lvm2 recognizes an
incompatible thin-pool and cannot activate such pool!

Users should use uncropped version as it is not suffering
from various issues between thin_repair results and allocated
metadata LV as thin_repair limit is 15.88GiB
Users should use cropping only when really needed!

Patch also better handles resize of thin-pool metadata and prevents resize
beoyond usable size 15.88GiB. Resize beyond 15.81GiB automatically
switches pool to no-crop version. Even with existing bigger thin-pool
metadata command 'lvextend -l+1 vg/pool_tmeta' does the change.

Patch gives better controls 'coverted' metadata LV and
reports less confusing message during conversion.

Patch set also moves the code for updating min/max into pool_manip.c
for better sharing with cache_pool code.
2021-02-01 12:06:13 +01:00
Zdenek Kabelac
b218a7cfe7 man: update lvmthin
Add few more notes about thin-pool repair.
Fix couple typos.
2021-02-01 12:06:13 +01:00
David Teigland
a690d16d29 writecache: use cleaner message instead of table reload
When detaching writecache, make the first stage send a message
to dm-writecache to set the cleaner option.  This is instead of
reloading the dm table with the cleaner option set.  Reloading
the table causes udev to process/probe the dm dev, which gets
stalled because of the writeback activity, and the stalled udev
in turn stalls the lvconvert command when it tries to sync with
udev events.

When getting writecache status we do not need to get
open_count or read_head info, which can cause extra steps.
2021-01-28 15:14:25 -06:00
Heinz Mauelshagen
f08ef23856 lvdisplay: enhance LV status output for raid(0)
In case legs of a raid0 LV are removed, the lvdisplay command still
reports 'available' though raid0 is not providing any resilience
compared to the other raid levels.

Also lvdisplay does not display '(partial)' in case of missing raid0
legs as oposed to the lvs command.

Enhance lvdisplay to report "NOT available" for any RaidLV type in case
too many legs are inaccessible hence causing data loss.  I.e. any leg
for raid0, all for raid1, more than 1 for raid4/5, more than 2 for raid6
and in case of completely lost mirror groups for raid10.

Add test/shell/lvdisplay-raid.sh.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1872678
2021-01-27 16:56:22 +01:00
Zdenek Kabelac
665d265349 WHATS_NEW: update 2021-01-22 16:09:38 +01:00
Zdenek Kabelac
fa2fa9f36d pvscan: ensure read buffer ends with 0
Read buffersize - 1 so the last byte is always 0.
Simplify init of 0 buffers.
Check snprintf result for error and report internal error as it could
happen only via bad compile parameters.
2021-01-22 15:30:37 +01:00
Zdenek Kabelac
ce6e74f485 filters: avoid duplicated //
sysfs_dir already goes with '/sys/'.
2021-01-22 15:30:37 +01:00
Zdenek Kabelac
5485ec10e2 pvscan: make ret value defined 2021-01-22 15:30:37 +01:00
Zdenek Kabelac
cddeaed6e9 tests: check support for online vdo rename
kvdo 6.2.3 module version supports online rename.
Add cache and raid stacked renames.
2021-01-22 15:30:37 +01:00
Zdenek Kabelac
8532b1ca97 vdo: support online rename
New VDO targets v6.2.3 corrects support for online rename of VDO device.
If needed if can be disable via new lvm.conf setting:

vdo_disabled_features = [ "online_rename" ]
2021-01-22 15:30:37 +01:00
Zdenek Kabelac
4b8e5ad595 pools: fix removal of spare volume
When removing pool LV from a stacked LV setup, it's been possible
to leak _pmspare and such hidden LV then required manual
user removal.

Fix it by moving automatic removal into _lv_reduce().
2021-01-22 15:30:37 +01:00
Zdenek Kabelac
9ae1935b74 config: avoid printing spaces before end of line
Empty comments were generating unnecessary space char before new line.
2021-01-22 15:30:37 +01:00
David Teigland
74adbec77f pvck: fix warning and exit code for non-4k mda1 offset
Print a warning if mda1 is not 4k, 8k or 64k.
Don't exit with an error for any unexpected mda1 offset.
2021-01-19 11:21:20 -06:00
David Teigland
2ec29d0677 label_scan: fix missing free of filtered_devs
missing free of devl entries on filtered_devs list in
commit 2c9bb67604
2021-01-18 16:26:02 -06:00
Marian Csontos
9757b4726c make: generate 2021-01-18 14:46:22 +01:00
David Teigland
37227b8ad6 devs: remove invalid path name aliases
Make dev_cache_get() verify aliases and drop any
that are invalid before returning a dev for a given
name.
2021-01-15 16:31:50 -06:00
David Teigland
0534723a2d integrity: fix segfault on error path when replacing images
When adding replacement raid+integrity images (lvconvert --repair
after a raid image is lost), various errors can cause the function
to exit with an error.  On this exit path, the function attempts
to revert new images that had been created but not yet used.  The
cleanup failed to account for the fact that not all images needed
to be reverted.
2021-01-13 13:39:33 -06:00
Zdenek Kabelac
ff1324d595 tests: check full zeroing of thin-pool metadata
Check zeroing works on whole thin-pool metadata device.
2021-01-13 12:54:45 +01:00
Zdenek Kabelac
71162b2b4e tests: check thin-pool corner case allocs 2021-01-13 12:54:45 +01:00
Zdenek Kabelac
0b6ee6a912 alloc: enhance estimation of sufficient_pes_free
Since commit 77fdc17d70 always include
log_len size into needed extents - however now we may need sometimes
more extents then necessary - mainly when multiple PVs are involved
into allocation.

Add logs_still_needed into calculation of sufficient_pes_free()
2021-01-13 12:54:45 +01:00
David Teigland
a4c56a3bc3 lvmlockd: sscanf buffer size warnings 2021-01-08 12:13:05 -06:00
Marian Csontos
6308365d48 post-release 2021-01-08 10:08:36 +01:00
Marian Csontos
3e8bd8d1bd pre-release 2021-01-08 10:07:25 +01:00
David Teigland
e9503f257a lvconvert: chunksize option was missing with cachedevice 2021-01-07 13:30:48 -06:00