1
0
mirror of git://sourceware.org/git/lvm2.git synced 2024-12-22 17:35:59 +03:00
Commit Graph

164 Commits

Author SHA1 Message Date
David Teigland
9048565093 devices: rework libudev usage
related to config settings:
  obtain_device_info_from_udev (controls if lvm gets
    a list of devices from readdir /dev or from libudev)
  external_device_info_source (controls if lvm asks
    libudev for device information)

. Make the obtain_device_list_from_udev setting
  affect only the choice of readdir /dev vs libudev.
  The setting no longer controls if udev is used for
  device type checks.

. Change obtain_device_list_from_udev default to 0.
  This helps avoid boot timeouts due to slow libudev
  queries, avoids reported failures from
  udev_enumerate_scan_devices, and avoids delays from
  "device not initialized in udev database" errors.
  Even without errors, for a system booting with 1024 PVs,
  lvm2-pvscan times improve from about 100 sec to 15 sec,
  and the pvscan command from about 64 sec to about 4 sec.

. For external_device_info_source="none", remove all
  libudev device info queries, and use only lvm
  native device info.

. For external_device_info_source="udev", first check
  lvm native device info, then check libudev info.

. Remove sleep/retry loop when attempting libudev
  queries for device info.  udev info will simply
  be skipped if it's not immediately available.

. Only set up a libdev connection if it will be used by
  obtain_device_list_from_udev/external_device_info_source.

. For native multipath component detection, use
  /etc/multipath/wwids.  If a device has a wwid
  matching an entry in the wwids file, then it's
  considered a multipath component.  This is
  necessary to natively detect multipath
  components when the mpath device is not set up.
2021-07-13 11:11:23 -05:00
Zdenek Kabelac
3d26bde1c0 make: cflow target works again
Restored 'make cflow' functinality.
Produces some quick referece of function chaining and usage.
2021-03-20 00:05:01 +01:00
David Teigland
83fe6e720f device usage based on devices file
The LVM devices file lists devices that lvm can use.  The default
file is /etc/lvm/devices/system.devices, and the lvmdevices(8)
command is used to add or remove device entries.  If the file
does not exist, or if lvm.conf includes use_devicesfile=0, then
lvm will not use a devices file.  When the devices file is in use,
the regex filter is not used, and the filter settings in lvm.conf
or on the command line are ignored.

LVM records devices in the devices file using hardware-specific
IDs, such as the WWID, and attempts to use subsystem-specific
IDs for virtual device types.  These device IDs are also written
in the VG metadata.  When no hardware or virtual ID is available,
lvm falls back using the unstable device name as the device ID.
When devnames are used, lvm performs extra scanning to find
devices if their devname changes, e.g. after reboot.

When proper device IDs are used, an lvm command will not look
at devices outside the devices file, but when devnames are used
as a fallback, lvm will scan devices outside the devices file
to locate PVs on renamed devices.  A config setting
search_for_devnames can be used to control the scanning for
renamed devname entries.

Related to the devices file, the new command option
--devices <devnames> allows a list of devices to be specified for
the command to use, overriding the devices file.  The listed
devices act as a sort of devices file in terms of limiting which
devices lvm will see and use.  Devices that are not listed will
appear to be missing to the lvm command.

Multiple devices files can be kept in /etc/lvm/devices, which
allows lvm to be used with different sets of devices, e.g.
system devices do not need to be exposed to a specific application,
and the application can use lvm on its own set of devices that are
not exposed to the system.  The option --devicesfile <filename> is
used to select the devices file to use with the command.  Without
the option set, the default system devices file is used.

Setting --devicesfile "" causes lvm to not use a devices file.

An existing, empty devices file means lvm will see no devices.

The new command vgimportdevices adds PVs from a VG to the devices
file and updates the VG metadata to include the device IDs.
vgimportdevices -a will import all VGs into the system devices file.

LVM commands run by dmeventd not use a devices file by default,
and will look at all devices on the system.  A devices file can
be created for dmeventd (/etc/lvm/devices/dmeventd.devices)  If
this file exists, lvm commands run by dmeventd will use it.

Internal implementaion:

- device_ids_read - read the devices file
  . add struct dev_use (du) to cmd->use_devices for each devices file entry
- dev_cache_scan - get /dev entries
  . add struct device (dev) to dev_cache for each device on the system
- device_ids_match - match devices file entries to /dev entries
  . match each du on cmd->use_devices to a dev in dev_cache, using device ID
  . on match, set du->dev, dev->id, dev->flags MATCHED_USE_ID
- label_scan - read lvm headers and metadata from devices
  . filters are applied, those that do not need data from the device
  . filter-deviceid skips devs without MATCHED_USE_ID, i.e.
    skips /dev entries that are not listed in the devices file
  . read lvm label from dev
  . filters are applied, those that use data from the device
  . read lvm metadata from dev
  . add info/vginfo structs for PVs/VGs (info is "lvmcache")
- device_ids_find_renamed_devs - handle devices with unstable devname ID
  where devname changed
  . this step only needed when devs do not have proper device IDs,
    and their dev names change, e.g. after reboot sdb becomes sdc.
  . detect incorrect match because PVID in the devices file entry
    does not match the PVID found when the device was read above
  . undo incorrect match between du and dev above
  . search system devices for new location of PVID
  . update devices file with new devnames for PVIDs on renamed devices
  . label_scan the renamed devs
- continue with command processing
2021-02-23 16:43:32 -06:00
David Teigland
d9e8895a96 Allow dm-integrity to be used for raid images
dm-integrity stores checksums of the data written to an
LV, and returns an error if data read from the LV does
not match the previously saved checksum.  When used on
raid images, dm-raid will correct the error by reading
the block from another image, and the device user sees
no error.  The integrity metadata (checksums) are stored
on an internal LV allocated by lvm for each linear image.
The internal LV is allocated on the same PV as the image.

Create a raid LV with an integrity layer over each
raid image (for raid levels 1,4,5,6,10):

lvcreate --type raidN --raidintegrity y [options]

Add an integrity layer to images of an existing raid LV:

lvconvert --raidintegrity y LV

Remove the integrity layer from images of a raid LV:

lvconvert --raidintegrity n LV

Settings

Use --raidintegritymode journal|bitmap (journal is default)
to configure the method used by dm-integrity to ensure
crash consistency.

Initialization

When integrity is added to an LV, the kernel needs to
initialize the integrity metadata/checksums for all blocks
in the LV.  The data corruption checking performed by
dm-integrity will only operate on areas of the LV that
are already initialized.  The progress of integrity
initialization is reported by the "syncpercent" LV
reporting field (and under the Cpy%Sync lvs column.)

Example: create a raid1 LV with integrity:

$ lvcreate --type raid1 -m1 --raidintegrity y -n rr -L1G foo
  Creating integrity metadata LV rr_rimage_0_imeta with size 12.00 MiB.
  Logical volume "rr_rimage_0_imeta" created.
  Creating integrity metadata LV rr_rimage_1_imeta with size 12.00 MiB.
  Logical volume "rr_rimage_1_imeta" created.
  Logical volume "rr" created.
$ lvs -a foo
  LV                  VG  Attr       LSize  Origin              Cpy%Sync
  rr                  foo rwi-a-r---  1.00g                     4.93
  [rr_rimage_0]       foo gwi-aor---  1.00g [rr_rimage_0_iorig] 41.02
  [rr_rimage_0_imeta] foo ewi-ao---- 12.00m
  [rr_rimage_0_iorig] foo -wi-ao----  1.00g
  [rr_rimage_1]       foo gwi-aor---  1.00g [rr_rimage_1_iorig] 39.45
  [rr_rimage_1_imeta] foo ewi-ao---- 12.00m
  [rr_rimage_1_iorig] foo -wi-ao----  1.00g
  [rr_rmeta_0]        foo ewi-aor---  4.00m
  [rr_rmeta_1]        foo ewi-aor---  4.00m
2020-04-15 12:10:32 -05:00
David Teigland
b6b4ad8e28 move pv_list code into lib 2020-04-13 10:04:14 -05:00
David Teigland
76dd9b2b51 writecache: move code into new file
put writecache specific code in writecache_manip.c

should be no functional change
2019-09-24 15:51:05 -05:00
David Teigland
6620dc9475 add device hints to reduce scanning
Save the list of PVs in /run/lvm/hints.  These hints
are used to reduce scanning in a number of commands
to only the PVs on the system, or only the PVs in a
requested VG (rather than all devices on the system.)
2019-01-15 10:23:47 -06:00
Zdenek Kabelac
701ecff0ff lvm: drop usage of dl library
Since lvm no longer supports any dlopen-able plugins
(which in practice was never really usable) drop linking
with -ldl.
2018-12-17 10:36:52 +01:00
David Teigland
3ae5569570 Add dm-writecache support
dm-writecache is used like dm-cache with a standard LV
as the cache.

$ lvcreate -n main -L 128M -an foo /dev/loop0

$ lvcreate -n fast -L 32M -an foo /dev/pmem0

$ lvconvert --type writecache --cachepool fast foo/main

$ lvs -a foo -o+devices
  LV            VG  Attr       LSize   Origin        Devices
  [fast]        foo -wi-------  32.00m               /dev/pmem0(0)
  main          foo Cwi------- 128.00m [main_wcorig] main_wcorig(0)
  [main_wcorig] foo -wi------- 128.00m               /dev/loop0(0)

$ lvchange -ay foo/main

$ dmsetup table
foo-main_wcorig: 0 262144 linear 7:0 2048
foo-main: 0 262144 writecache p 253:4 253:3 4096 0
foo-fast: 0 65536 linear 259:0 2048

$ lvchange -an foo/main

$ lvconvert --splitcache foo/main

$ lvs -a foo -o+devices
  LV   VG  Attr       LSize   Devices
  fast foo -wi-------  32.00m /dev/pmem0(0)
  main foo -wi------- 128.00m /dev/loop0(0)
2018-11-06 14:18:41 -06:00
David Teigland
117160b27e Remove lvmetad
Native disk scanning is now both reduced and
async/parallel, which makes it comparable in
performance (and often faster) when compared
to lvm using lvmetad.

Autoactivation now uses local temp files to record
online PVs, and no longer requires lvmetad.

There should be no apparent command-level change
in behavior.
2018-07-11 11:26:42 -05:00
Zdenek Kabelac
a8f84f7801 vdo: introduce segment types and manip functions
Core functionality introducing lvm VDO support.
2018-07-09 15:28:35 +02:00
David Teigland
73b7e6fde7 Remove more code that was only used by liblvm2app 2018-06-08 09:29:11 -05:00
David Teigland
3e781ea446 Remove clvmd and associated code
More code reduction and simplification can follow.
2018-06-05 11:09:13 -05:00
Joe Thornber
232918fb86 build: libbase.a 2018-06-04 13:53:07 +01:00
Joe Thornber
7635df8cce bcache: switch to storing blocks in a radix tree.
Rather than a hash table.  This will make invalidate_fd() more
efficient since we can iterate just those blocks that are on
a particular dev.
2018-05-30 14:17:26 +01:00
Joe Thornber
efad84ebc2 bcache: Move the utils to a separate file.
This makes it clearer that they don't access the cache internals.
2018-05-03 09:34:41 +01:00
David Teigland
24e7745d7a devices: ignore lvm1 and pool devices 2018-05-01 15:18:47 -05:00
David Teigland
c1cd18f21e Remove lvm1 and pool disk formats
There are likely more bits of code that can be removed,
e.g. lvm1/pool-specific bits of code that were identified
using FMT flags.

The vgconvert command can likely be reduced further.

The lvm1-specific config settings should probably have
some other fields set for proper deprecation.
2018-04-30 16:55:02 -05:00
Joe Thornber
2bc896f2a3 build: remove --with-{snapshots,mirrors,raid,thin,cache} options from ./configure
It now behaves as if the were all set as 'internal'
2018-04-30 10:11:23 +01:00
Joe Thornber
da7e13ef88 [lib/device/bcache] Tweaks after Kabi's review 2018-04-20 11:10:45 -05:00
Alasdair G Kergon
f3ae99dcc0 liblvm: Move lib code used exclusively into metadata-liblvm.c
Also remove some redundant function definitions from metadata.h.
2017-10-18 19:29:32 +01:00
David Teigland
6ac1e04b3a replicator: remove the code
It has not been used in a long time and is not
expected to be used further.
2017-10-13 16:20:42 -05:00
Zdenek Kabelac
00fdf01d9d makefiles: cleanups 2017-08-01 11:53:32 +02:00
Zdenek Kabelac
2232e82d25 makefiles: fixing linking
Avoid adding -g more then once for debug builds.
Avoid enabling  DEBUG_MEM when we build multithreaded tools.
Link executables with -fPIE -pie and --export-dynamic LDFLAGS
Introduce PROGS_FLAGS to add option to pass flags for external libs.
Link  lvm2 internally library only when really used.
Link DAEMON_LIBS with daemons.
Pass VALGRIND_CFLAGS internally
Set shell failure mode on couple places.
2017-08-01 11:53:30 +02:00
Alasdair G Kergon
c27963c566 lib: Move lcm and gcd to lib/misc for wider use. 2016-08-18 14:06:13 +01:00
David Teigland
ebd2758dab vgimportclone: add native command
This is cleaner and more efficient than the script.
The args and usage are unchanged.
2016-06-22 13:13:10 -05:00
David Teigland
2d5dc6512e dbus: add notification from commands
When a command modifies a PV or VG, or changes the
activation state of an LV, it will send a dbus
notification when the command is finished.  This
can be enabled/disabled with a config setting.
2016-03-07 10:06:09 -06:00
Zdenek Kabelac
fcbef05aae doc: change fsf address
Hmm rpmlint suggest fsf is using a different address these days,
so lets keep it up-to-date
2016-01-21 12:11:37 +01:00
Lidong Zhong
729f489009 pvcreate: don't support unpartitioned DASD devices with CDL formatted
The former patch(dab3ebce4c) is a little bit strict. For example, it is
OK to create PV on unpartitioned DASD devices with LDL formatted. So
after lvm version containing the patch, LVs created on those devices
could not be found.

Signed-off-by: Lidong Zhong <lzhong@suse.com>
2015-10-27 11:42:47 +01:00
Zdenek Kabelac
8ff43c3705 makefiles: ensure no old .h are left
Ensure make clean  cleans any left-over file from their previous
location so they are not in conflict with new ones.

Also hide error message when .commands file is not present.
2015-09-08 15:41:33 +02:00
Zdenek Kabelac
9403edbb93 configure: relocate generated headers
Relocate generated configure.h and lvm-version.h outside
of compilable .c source tree.

The reason is behind - when compiling in builddir != srcdir
the generated file in lib/misc/configure.h was used for all compiled
source file except ones located in lib/misc dir - those would have used
configure.h file located in this dir - if there have existed one (i.e.
from some other build)

This problem was only visible, when srcdir == buildir was used before
trying to use  srcdri != builddir  (as configure.h appeared then in
srcdir).
2015-09-07 17:40:58 +02:00
Alasdair G Kergon
a28fb37b9e libdm: Add dm_timestamp functions. 2015-07-29 19:21:07 +01:00
David Teigland
fe70b03de2 Add lvmlockd 2015-07-02 15:42:26 -05:00
Peter Rajnoha
f143ad3a93 cleanup: remove unused tags.c file 2015-06-29 09:43:47 +02:00
Ondrej Kozina
e587b0677b lvmpolld: Add standalone polldaemon.
See doc/lvmpolld_overview.txt
2015-05-09 00:59:18 +01:00
Petr Rockai
d571eab3b2 configure: Look for valgrind.h independently of VALGRIND_POOLS. 2015-02-05 13:50:34 +01:00
Peter Rajnoha
1c7a509bed filters: add firmware RAID filter
Just like MD filtering that detects components of software RAID (md),
add detection for firmware RAID.

We're not adding any native code to detect this - there are lots of
firmware RAIDs out there which is just out of LVM scope. However,
with current changes with which we're able to get device info from
external sources (e.g. external_device_info_source="udev"), we can
do this easily if the external device status source has this kind
of information - which is the case of "udev" source where the results
of blkid scans are stored.

This detection should cover all firmware RAIDs that blkid can detect and
which are identified as:
  ID_FS_TYPE = {adaptec,ddf,hpt45x,hpt37x,isw,jmicron,lsi_mega,nvidia,promise_fasttrack,silicon_medley,via}_raid_member
2015-01-30 13:17:12 +01:00
Peter Rajnoha
fbfde21e7c device: add infrastructure to support external device info 2015-01-30 13:01:12 +01:00
Peter Rajnoha
8a843d0d97 filters: add "usable device" filter
Usable device filter is responsible for filtering out unusable DM devices.
The filter has 3 modes of operation:

  - FILTER_MODE_NO_LVMETAD:
    When this mode is used, we check DM device usability by looking:
      - whether device is empty
      - whether device is blocked
      - whether device is suspended (only on devices/ignore_suspended_devices=1)
      - whether device uses an error target
      - whether device name/uuid is reserved

  - FILTER_MODE_PRE_LVMETAD:
    When this mode is used, we check DM device usability by looking:
      - whether device is empty
      - whether device is suspended (only on devices/ignore_suspended_devices=1)
      - whether device uses an error target
      - whether device name/uuid is reserved

  - FILTER_MODE_POST_LVMETAD:
    When this mode is used, we check DM device usability by looking:
      - whether device is blocked
      - whether device is suspended (only on devices/ignore_suspended_devices=1)

These modes will be used by subsequent patch to create different
instances of this filter, depending on lvmetad use.
2014-09-30 13:11:58 +02:00
Alasdair G Kergon
29a3fbf067 locking: Separate out flock and signal code. 2014-05-01 17:37:14 +01:00
Zdenek Kabelac
7a595d7388 makefiles: use BLKID/UDEV_CFLAGS properly
blkid.h needs BLKID_CFLAGS
Do not add UDEV_CFLAGS everywhere and use it only when needed.
2014-03-06 17:30:06 +01:00
Jonathan Brassow
b94a3ee9f6 cache: Add functions that create/remove cache LVs
A cache LV - from LVM's perpective - is a user accessible device that
links the cachepool LV and the origin LV.  The following functions
were added to facilitate the creation and removal of this top-level
LV:
1) 'lv_cache_create' - takes a cachepool and an origin device and links
   them into a new top-level LV of 'cache' segment type.  No allocation
   is necessary in this function, as the sub-LVs contain all of the
   necessary allocated space.  Only the top-level layer needs to be
   created.

2) 'lv_cache_remove' - this function removes the top-level LV of a
   cache LV - promoting the cachepool and origin sub-LVs to top-level
   devices and leaving them exposed to the user.  That is, the
   cachepool is unlinked and free to be used with another origin to
   form a new cache LV; and the origin is no longer cached.
   (Currently, if the cache needs to be flushed, it is done in this
   function and the function waits for it to complete before proceeding.
   This will be taken out in a future patch in favor of polling.)
2014-02-04 07:59:58 -06:00
Jonathan Brassow
90bbed3255 cache: New 'cachepool' segment type
This patch adds the new cachepool segment type - the first of two
necessary to eventually create 'cache' logical volumes.  In addition
to the new segment type, updates to makefiles, configure files, the
lv_segment struct, and some necessary libdevmapper flags.

The cachepool is the LV and corresponding segment type that will hold
all information pertinent to the cache itself - it's size, cachemode,
cache policy, core arguments (like migration_threshold), etc.
2014-01-27 05:27:16 -06:00
Jonathan Brassow
5590448c32 Misc: Move some thin pool functions to a new file
Functions that handle set-up, tear-down and creation of thin pool
volumes will be more generally applicable when more targets exist
that make use of device-mapper's persistent data format.  One of
these targets is the dm-cache target.  I've selected some functions
that will be useful for the cache segment type to be moved, since
they will no longer be thin pool specific but are more broadly
useful to any segment type that makes use of a 'pool' LV.
2014-01-22 10:11:29 -06:00
Alasdair G Kergon
80bcdb93ff filters: check for mpath before opening devs
Split out the partitioned device filter that needs to open the device
and move the multipath filter in front of it.

When a device is multipathed, sending I/O to the underlying paths may
cause problems, the most obvious being I/O errors visible to lvm if a
path is down.

Revert the incorrect <backtrace> messages added when a device doesn't
pass a filter.

Log each filter initialisation to show sequence.

Avoid duplicate 'Using $device' debug messages.
2013-08-13 23:26:58 +01:00
Tony Asleson
9775a19af1 lib/properties: Common code for handling properties.
The common bits from lib/report/properties.[c|h] have
been moved to lib/properties/prop_common.[c|h] to allow
re-use of property handling functionality without
polluting the report handling functionality.

Signed-off-by: Tony Asleson <tasleson@redhat.com>
2013-07-12 16:52:16 -05:00
Peter Rajnoha
1778d34cef refactor: move lib/device/device.c -> lib/device/dev-type.c 2013-06-12 11:54:37 +02:00
Zdenek Kabelac
286cd2006b cleanup: drop unneeded included header files
This headers were not resolving anything used for compiled .c files.
Remove unused util.c file.
2012-08-23 14:37:20 +02:00
Alasdair Kergon
7126d8c21b Reflect new file locations, include file updates etc. 2012-02-28 18:35:04 +00:00
Petr Rockai
dae0822698 The lvmetad client-side integration. Only active when use_lvmetad = 1 is set in
lvm.conf *and* lvmetad is running.
2012-02-23 13:11:07 +00:00