1
0
mirror of git://sourceware.org/git/lvm2.git synced 2024-12-22 17:35:59 +03:00
Commit Graph

124 Commits

Author SHA1 Message Date
David Teigland
96b777167c cov: clean up pvid and vgid usage
pvid and vgid are sometimes a null-terminated string, and
other times a 'struct id', and the two types were often
cast between each other.  When a struct id was cast to a char
pointer, the resulting string would not necessarily be null
terminated.  Casting a null-terminated string id to a
struct id is fine, but is still avoided when possible.

A struct id is:  int8_t uuid[ID_LEN]
A string id is:  char pvid[ID_LEN + 1]

A convention is introduced to help distinguish them:

- variables and struct fields named "pvid" or "vgid"
  should be null-terminated strings.

- variables and struct fields named "pv_id" or "vg_id"
  should be struct id's.

- examples:
  char pvid[ID_LEN + 1];
  char vgid[ID_LEN + 1];
  struct id pv_id;
  struct id vg_id;

Function names also attempt to follow this convention.

Avoid casting between the two types as much as possible,
with limited exceptions when known to be safe and clearly
commented.

Avoid using variations of strcpy and strcmp, and instead
use memcpy/memcmp with ID_LEN (with similar limited
exceptions possible.)
2021-08-16 11:31:15 -05:00
David Teigland
83fe6e720f device usage based on devices file
The LVM devices file lists devices that lvm can use.  The default
file is /etc/lvm/devices/system.devices, and the lvmdevices(8)
command is used to add or remove device entries.  If the file
does not exist, or if lvm.conf includes use_devicesfile=0, then
lvm will not use a devices file.  When the devices file is in use,
the regex filter is not used, and the filter settings in lvm.conf
or on the command line are ignored.

LVM records devices in the devices file using hardware-specific
IDs, such as the WWID, and attempts to use subsystem-specific
IDs for virtual device types.  These device IDs are also written
in the VG metadata.  When no hardware or virtual ID is available,
lvm falls back using the unstable device name as the device ID.
When devnames are used, lvm performs extra scanning to find
devices if their devname changes, e.g. after reboot.

When proper device IDs are used, an lvm command will not look
at devices outside the devices file, but when devnames are used
as a fallback, lvm will scan devices outside the devices file
to locate PVs on renamed devices.  A config setting
search_for_devnames can be used to control the scanning for
renamed devname entries.

Related to the devices file, the new command option
--devices <devnames> allows a list of devices to be specified for
the command to use, overriding the devices file.  The listed
devices act as a sort of devices file in terms of limiting which
devices lvm will see and use.  Devices that are not listed will
appear to be missing to the lvm command.

Multiple devices files can be kept in /etc/lvm/devices, which
allows lvm to be used with different sets of devices, e.g.
system devices do not need to be exposed to a specific application,
and the application can use lvm on its own set of devices that are
not exposed to the system.  The option --devicesfile <filename> is
used to select the devices file to use with the command.  Without
the option set, the default system devices file is used.

Setting --devicesfile "" causes lvm to not use a devices file.

An existing, empty devices file means lvm will see no devices.

The new command vgimportdevices adds PVs from a VG to the devices
file and updates the VG metadata to include the device IDs.
vgimportdevices -a will import all VGs into the system devices file.

LVM commands run by dmeventd not use a devices file by default,
and will look at all devices on the system.  A devices file can
be created for dmeventd (/etc/lvm/devices/dmeventd.devices)  If
this file exists, lvm commands run by dmeventd will use it.

Internal implementaion:

- device_ids_read - read the devices file
  . add struct dev_use (du) to cmd->use_devices for each devices file entry
- dev_cache_scan - get /dev entries
  . add struct device (dev) to dev_cache for each device on the system
- device_ids_match - match devices file entries to /dev entries
  . match each du on cmd->use_devices to a dev in dev_cache, using device ID
  . on match, set du->dev, dev->id, dev->flags MATCHED_USE_ID
- label_scan - read lvm headers and metadata from devices
  . filters are applied, those that do not need data from the device
  . filter-deviceid skips devs without MATCHED_USE_ID, i.e.
    skips /dev entries that are not listed in the devices file
  . read lvm label from dev
  . filters are applied, those that use data from the device
  . read lvm metadata from dev
  . add info/vginfo structs for PVs/VGs (info is "lvmcache")
- device_ids_find_renamed_devs - handle devices with unstable devname ID
  where devname changed
  . this step only needed when devs do not have proper device IDs,
    and their dev names change, e.g. after reboot sdb becomes sdc.
  . detect incorrect match because PVID in the devices file entry
    does not match the PVID found when the device was read above
  . undo incorrect match between du and dev above
  . search system devices for new location of PVID
  . update devices file with new devnames for PVIDs on renamed devices
  . label_scan the renamed devs
- continue with command processing
2021-02-23 16:43:32 -06:00
David Teigland
87ee401eea md component detection changes
Move extra md component detection into the label scan phase.
It had been in set_pv_devices which was deep within the vg_read
phase, which wasn't a good place (better to detect that earlier.)
Now that pv metadata info is available in the scan phase, the pv
details (size and device_hint) can be used for extra md checking.

Use the device_hint from the pv metadata to trigger a full md
component check if the device_hint begins with /dev/md.

Stop triggering full md component checks based on missing
udev info for a dev.

Changes to tests to reflect that the code is now detecting
md components in some test case that it wasn't before.
2021-02-05 16:23:51 -06:00
David Teigland
b3cdf0d881 lvmcache: add lvmcache_get_dev_mda
for future patch
2020-10-21 16:24:16 -05:00
David Teigland
450f272b31 devices: support printing the filter that rejects a device
Use of this new message function needs to be added
to various commands to improve the output.
2020-10-01 12:00:09 -05:00
David Teigland
1404e5ee61 metadata: open rw fd before closing ro fd
lvm opens devices readonly to scan them, but
needs to open then readwrite to update the metadata.
Previously, the ro fd was closed before the rw fd
was opened, leaving a small gap where the dev was
not held open, and during which the dev could
possibly change which storage it referred to.

With the bcache_change_fd() interface, lvm opens a
rw fd on a device to be written, tells bcache to
change to the new rw fd, and closes the ro fd.

. open dev ro
. read dev with the ro fd (label_scan)
. lock vg (ex for writing)
. open dev rw
. close ro fd
. rescan dev to check if the metadata changed
  between the scan and the lock
. if the metadata did change, reread in full
. write the metadata
2020-09-18 15:10:11 -05:00
David Teigland
d79afd4084 lvmcache: rework handling of VGs with duplicate vgnames
The previous method of managing duplicate vgnames prevented
vgreduce from working if a foreign vg with the same name
existed.
2020-04-21 14:40:34 -05:00
David Teigland
cc4051eec0 pass cmd struct through more functions
no functional change
2020-04-21 10:58:05 -05:00
David Teigland
3854931aea lvmcache_get_mda: remove unused function 2020-04-21 10:58:05 -05:00
David Teigland
0c1316cda8 scanning: optimize by checking text offset and checksum
After the VG lock is taken for vg_read, reread the mda_header
and compare the metadata text offset and checksum to what was
seen during label scan.  If it is unchanged, then the metadata
has not changed since the label scan, and the metadata does not
need to be reread under the lock for command processing.

For commands that do not make changes (e.g. reporting), the
mda_header is reread and checked on one mda to decide if the
full metadata rereading can be skipped.  For other commands
(e.g. modifying the vg) the mda_header is reread and checked
from all PVs.  (These could probably just check one mda also.)
2019-11-26 16:52:28 -06:00
David Teigland
bd21736e8b vgck: let updatemetadata repair mismatched metadata
Let vgck --updatemetadata repair cases where different mdas
hold indepedently valid but unmatching copies of the metadata,
i.e. different text metadata checksums or text metadata sizes.
2019-10-11 12:57:39 -05:00
David Teigland
f3084ee2e5 scan: add PV summary info to lvmcache
Expand the lvmcache info that is saved by the scan to
include PV info from the metadata.
2019-09-30 11:38:10 -05:00
David Teigland
e01fddc578 improve duplicate pv handling for md components
Eliminate md components at the start so they don't
interfere with actual duplicates, and don't need
to be removed later.  This also allows for choosing
no copy of a PVID if they all happen to be md
components.
2019-08-16 13:26:12 -05:00
David Teigland
677833ce6f lvmcache: renaming functions and variables
related to duplicates, no functional changes.
2019-08-16 13:26:11 -05:00
David Teigland
d16142f90f scanning: open devs rw when rescanning for write
When vg_read rescans devices with the intention of
writing the VG, the label rescan can open the devs
RW so they do not need to be closed and reopened
RW in dev_write_bytes.
2019-06-21 10:57:49 -05:00
David Teigland
4bb7d3da0e lvmcache: remove wrapper around lvmcache_get_vgnameids
This was left over from when there was an lvmetad
version of the function.
2019-06-11 14:10:14 -05:00
David Teigland
49b8846567 lvmcache: remove unused function
Drop lvmcache_fmt_from_vgname(), the way it was called made
it identical to the existing lvmcache_vginfo_from_vgname().
2019-06-10 10:38:32 -05:00
David Teigland
5036244ce8 lvmcache: remove unused code 2019-06-10 10:38:32 -05:00
David Teigland
45b164f62c create separate lvmcache update functions for read and write
The vg read and vg write cases need to update lvmcache
differently, so create separate functions for them.

The read case now handles checking for outdated mdas
and moves them aside into a new list to be repaired in
a subsequent commit.
2019-06-07 15:54:04 -05:00
David Teigland
86d831b916 change args for text label read function
Have the caller pass the label_sector to the read
function so the read function can set the sector
field in the label struct, instead of having the
read function return a pointer to the label for
the caller to set the sector field.

Also have the read function return a flag indicating
to the caller that the scanned device was identified
as a duplicate pv.
2019-06-07 15:54:04 -05:00
David Teigland
889b5d3183 add mda arg to add_mda
Allow the caller of lvmcache_add_mda() to have the
new mda returned.
2019-06-07 15:54:04 -05:00
David Teigland
b2447e3538 keep track of which mdas have old metadata in lvmcache
This will be used for more advanced repair in a
subsequent commit.
2019-06-07 15:54:04 -05:00
David Teigland
0b18c25d93 ability to keep track of outdated pvs in lvmcache
Outdated PVs hold metadata for VG from which they
have been removed.  Add the ability to keep track
of these in lvmcache.
This will be used for more advanced repair in a
subsequent commit.
2019-06-07 15:54:04 -05:00
David Teigland
650524b955 ability to keep track of bad mdas in lvmcache
mda's that cannot be processed by lvm because of
some corruption can be kept on a separate list.
These will be used for more advanced repair in a
subsequent commit.
2019-06-07 15:54:04 -05:00
David Teigland
52586b1039 pvck: new dump option to extract metadata
The new command 'pvck --dump metadata PV' will extract
the current version of VG metadata from a PV for testing
and debugging.  --dump metadata_area extracts the entire
text metadata area.
2019-05-23 11:49:06 -05:00
David Teigland
3584e0c0d5 io: warn when metadata size approaches io memory size
When a single copy of metadata gets within 1MB of the
current io_memory_size value, begin printing a warning
that the io_memory_size should be increased.
2019-03-04 12:13:09 -06:00
David Teigland
6620dc9475 add device hints to reduce scanning
Save the list of PVs in /run/lvm/hints.  These hints
are used to reduce scanning in a number of commands
to only the PVs on the system, or only the PVs in a
requested VG (rather than all devices on the system.)
2019-01-15 10:23:47 -06:00
David Teigland
117160b27e Remove lvmetad
Native disk scanning is now both reduced and
async/parallel, which makes it comparable in
performance (and often faster) when compared
to lvm using lvmetad.

Autoactivation now uses local temp files to record
online PVs, and no longer requires lvmetad.

There should be no apparent command-level change
in behavior.
2018-07-11 11:26:42 -05:00
David Teigland
db741e75a2 pvscan: autoactivate without lvmetad
When lvmetad is not used, use temporary files to record
which PVs have appeared.  Use these temp files to determine
when a VG is complete, to trigger autoactivation.

This change allows us to remove lvmetad while keeping the
same autoactivation behavior that lvmetad provides.

The temp files are created in /run/lvm/pvs_online/ and are
named for the PVID of the PV.  The files contain the
major:minor of the device the PV was read from.

e.g. if VG foo has dev1 and dev2, then:

. pvscan --cache -aay dev1
  reads vg metadata from dev1
  creates /run/lvm/pvs_online/<pvid-of-dev1>
  checks if all vg->pvs are online: no

. pvscan --cache -aay dev2
  reads vg metadata from dev2
  creates /run/lvm/pvs_online/<pvid-of-dev2>
  checks if all vg->pvs are online: yes
  autoactivates vg

A 'pvscan --cache dev' (without -aay) still records that
dev is online.

A 'pvscan --cache --major X --minor Y' after a device is
gone will remove the temp file for it.

A 'pvscan --cache [-aay]' (no devs) resets the state of
temp files by removing them all, then scanning all devs
and creating temp files for PVs that are found.

If no online files exist, the first pvscan --cache scans
all devs and creates temp files for any PVs found.

The scope of the temp files is only pvscan, and they are only
used for pvscan-based autoactivation.  No other commands are
concerned with or aware of these temp files.  When lvm creates
or removes PVs, no attempt is made to update the temp files.
2018-07-09 16:11:24 -05:00
David Teigland
c4153a8dfc Remove checking for locked VGs
A few places were calling a function to check if a
VG lock was held.  The only place it was actually
needed is for pvcreate which wants to do its own
locking (and scanning) around process_each_pv.

The locking/scanning exceptions for pvcreate in
process_each_pv/vg_read can be enabled by just passing
a couple of flags instead of checking if the VG is
already locked.  This also means that these special
cases won't be enabled unknowingly in other places
where they shouldn't be used.
2018-06-12 09:46:04 -05:00
David Teigland
669b1295ae Remove header declarations for removed functions 2018-06-08 10:01:05 -05:00
David Teigland
e7aa51c70f Remove VG lock ordering check
Four commands lock two VGs at a time:

- vgsplit and vgmerge already have their own logic to
  acquire the locks in the correct order.

- vgimportclone and vgrename disable this ordering check.
2018-06-07 16:17:04 +01:00
Joe Thornber
dbba1e9b93 Merge branch 'master' into 2018-05-11-fork-libdm 2018-06-01 13:04:12 +01:00
David Teigland
3c9ed33f83 scan: move warnings about duplicate devices
We have been warning about duplicate devices (and disabling lvmetad)
immediately when the dup was detected (during label_scan).  Move the
warnings (and the disabling) to happen later, after label_scan is
finished.

This lets us avoid an unwanted warning message about duplicates
in the special case were md components are eliminated during the
duplicate device resolution.
2018-05-21 16:48:02 -05:00
Joe Thornber
89fdc0b588 Merge branch 'master' into 2018-05-11-fork-libdm 2018-05-16 13:43:02 +01:00
Joe Thornber
7f97c7ea9a build: Don't generate symlinks in include/ dir
As we start refactoring the code to break dependencies (see doc/refactoring.txt),
I want us to use full paths in the includes (eg, #include "base/data-struct/list.h").
This makes it more obvious when we're breaking abstraction boundaries, eg, including a file in
metadata/ from base/
2018-05-14 10:30:20 +01:00
David Teigland
73578e36fa dev_cache: remove the lvmcache check when closing fd
This is no longer used since devices are not held
open in dev_cache.
2018-05-11 14:30:10 -05:00
David Teigland
c016b573ee clvmd: separate saved_vg from vginfo
The clvmd saved_vg data is independent from the normal lvm
lvmcache vginfo data, so separate saved_vg from vginfo.
Normal lvm doesn't need to use save_vg at all, and in clvmd,
lvmcache changes on vginfo can be made without worrying
about unwanted effects on saved_vg.
2018-05-03 14:54:48 -05:00
David Teigland
a5e13f2eef clvmd: defer freeing saved vgs
To avoid the chance of freeing a saved vg while another
code path is using it, defer freeing saved vgs until
all the lvmcache content is dropped for the vg.
2018-05-03 14:54:48 -05:00
David Teigland
1fec86571f clvmd: reuse a vg struct for sequential LV operations
After reading a VG, stash it in lvmcache as "saved_vg".
Before reading the VG again, try to use the saved_vg.
The saved_vg is dropped on VG lock operations.
2018-04-25 16:39:43 -05:00
David Teigland
f8616ac2d8 lvmcache: rename suspended_vg to saved_vg
The copy of the VG which clvmd stashes in lvmcache should
not only be used between suspend and resume, but between
sequential LV operations in clvmd, so that clvmd does not
need to reread the VG for each one.  Prepare for that by
renaming the stashed VG as "saved_vg".
2018-04-25 16:39:43 -05:00
David Teigland
aee27dc7ba scan: skip device rescan in vg_read
For reporting commands (pvs,vgs,lvs,pvdisplay,vgdisplay,lvdisplay)
we do not need to repeat the label scan of devices in vg_read if
they all had matching metadata in the initial label scan.  The
data read by label scan can just be reused for the vg_read.
This cuts the amount of device i/o in half, from two reads of
each device to one.  We have to be careful to avoid repairing
the VG if we've skipped rescanning.  (The VG repair code is very
poor, and will be redone soon.)
2018-04-20 11:23:14 -05:00
David Teigland
9b6a62f944 lvmcache: simplify
Recent changes allow some major simplification of the way
lvmcache works and is used.  lvmcache_label_scan is now
called in a controlled fashion at the start of commands,
and not via various unpredictable side effects.  Remove
various calls to it from other places.  lvmcache_label_scan
should not be called from anywhere during a command, because
it produces an incorrect representation of PVs with no MDAs,
and misclassifies them as orphans.  This has been a long
standing problem.  The invalid flag and rescanning based on
that is no longer used and removed.  The 'force' variation is
no longer needed and removed.
2018-04-20 11:22:48 -05:00
David Teigland
1717d4cb17 lvmcache: add shorter way to delete dev info
Don't make the caller look up the info first.
2018-04-20 11:22:47 -05:00
David Teigland
f17c2cf7c6 pvremove: device check doesn't require label_read
It just needs to check if the device was found during
the scan, which means checking if it exists in lvmcache.
2018-04-20 11:22:45 -05:00
David Teigland
d9a77e8bb4 lvmcache: simplify metadata cache
The copy of VG metadata stored in lvmcache was not being used
in general.  It pretended to be a generic VG metadata cache,
but was not being used except for clvmd activation.  There
it was used to avoid reading from disk while devices were
suspended, i.e. in resume.

This removes the code that attempted to make this look
like a generic metadata cache, and replaces with with
something narrowly targetted to what it's actually used for.

This is a way of passing the VG from suspend to resume in
clvmd.  Since in the case of clvmd one caller can't simply
pass the same VG to both suspend and resume, suspend needs
to stash the VG somewhere that resume can grab it from.
(resume doesn't want to read it from disk since devices
are suspended.)  The lvmcache vginfo struct is used as a
convenient place to stash the VG to pass it from suspend
to resume, even though it isn't related to the lvmcache
or vginfo.  These suspended_vg* vginfo fields should
not be used or touched anywhere else, they are only to
be used for passing the VG data from suspend to resume
in clvmd.  The VG data being passed between suspend and
resume is never modified, and will only exist in the
brief period between suspend and resume in clvmd.

suspend has both old (current) and new (precommitted)
copies of the VG metadata.  It stashes both of these in
the vginfo prior to suspending devices.  When vg_commit
is successful, it sets a flag in vginfo as before,
signaling the transition from old to new metadata.

resume grabs the VG stashed by suspend.  If the vg_commit
happened, it grabs the new VG, and if the vg_commit didn't
happen it grabs the old VG.  The VG is then used to resume
LVs.

This isolates clvmd-specific code and usage from the
normal lvm vg_read code, making the code simpler and
the behavior easier to verify.

Sequence of operations:

- lv_suspend() has both vg_old and vg_new
  and stashes a copy of each onto the vginfo:
  lvmcache_save_suspended_vg(vg_old);
  lvmcache_save_suspended_vg(vg_new);

- vg_commit() happens, which causes all clvmd
  instances to call lvmcache_commit_metadata(vg).
  A flag is set in the vginfo indicating the
  transition from the old to new VG:
  vginfo->suspended_vg_committed = 1;

- lv_resume() needs either vg_old or vg_new
  to use in resuming LVs.  It doesn't want to
  read the VG from disk since devices are
  suspended, so it gets the VG stashed by
  lv_suspend:
  vg = lvmcache_get_suspended_vg(vgid);

If the vg_commit did not happen, suspended_vg_committed
will not be set, and in this case, lvmcache_get_suspended_vg()
will return the old VG instead of the new VG, and it will
resume LVs based on the old metadata.
2018-04-20 11:22:45 -05:00
David Teigland
79c4971210 label_scan: remove extra label scan and read for orphan PVs
When process_each_pv() calls vg_read() on the orphan VG, the
internal implementation was doing an unnecessary
lvmcache_label_scan() and two unnecessary label_read() calls
on each orphan.  Some of those unnecessary label scans/reads
would sometimes be skipped due to caching, but the code was
always doing at least one unnecessary read on each orphan.

The common format_text case was also unecessarily calling into
the format-specific pv_read() function which actually did nothing.

By analyzing each case in which vg_read() was being called on
the orphan VG, we can say that all of the label scans/reads
in vg_read_orphans are unnecessary:

1. reporting commands: the information saved in lvmcache by
the original label scan can be reported.  There is no advantage
to repeating the label scan on the orphans a second time before
reporting it.

2. pvcreate/vgcreate/vgextend: these all share a common
implementation in pvcreate_each_device().  That function
already rescans labels after acquiring the orphan VG lock,
which ensures that the command is using valid lvmcache
information.
2018-04-20 11:22:45 -05:00
David Teigland
4507ba3596 scan: use new label_scan for lvmcache_label_scan
To do label scanning, lvm code calls lvmcache_label_scan().
Change lvmcache_label_scan() to use the new label_scan()
based on bcache.

Also add lvmcache_label_rescan_vg() which calls the new
label_scan_devs() which does label scanning on only the
specified devices.  This is for a subsequent commit and
is not yet used.
2018-04-20 11:19:32 -05:00
Joe Thornber
00f1b208a1 [io paths] Unpick agk's aio stuff 2018-04-20 11:03:58 -05:00
Alasdair G Kergon
6210c1ec28 device: Mark read-only device buffers const. 2018-01-10 19:57:10 +00:00