1
0
mirror of git://sourceware.org/git/lvm2.git synced 2024-12-21 13:34:40 +03:00
Commit Graph

176 Commits

Author SHA1 Message Date
David Teigland
2b241eb1f6 pvck: use new dump routines for old output
Use the recently added dump routines to produce the
old/traditional pvck output, and remove the code that
had been used for that.

The validation/checking done by the new routines means
that new lines prefixed with CHECK are printed for
incorrect values.
2019-06-05 16:28:52 -05:00
David Teigland
dc1e12dcd4 scan: expand and update label scan comments 2019-05-21 12:02:40 -05:00
David Teigland
60bf9c9f33 hints: exclude md components
In some cases md components could be included in
the hints, so add a check to hint creation to make
sure they are excluded.
2019-05-21 11:58:01 -05:00
David Teigland
99de816a1b scan: remove comments about lvmetad 2019-05-02 13:32:30 -05:00
David Teigland
0046c4e7a7 use memcpy for constant ondisk strings
Use memcpy/memcmp for on disk strings which are not
null terminated: FMTT_MAGIC, LVM2_LABEL and LABEL_ID.
Quiets compile warnings.
2019-05-02 12:59:50 -05:00
David Teigland
c3e385c108 hints: skip hint flock if nolocking option is set 2019-04-29 13:01:15 -05:00
David Teigland
8c87dda195 locking: unify global lock for flock and lockd
There have been two file locks used to protect lvm
"global state": "ORPHANS" and "GLOBAL".

Commands that used the ORPHAN flock in exclusive mode:
  pvcreate, pvremove, vgcreate, vgextend, vgremove,
  vgcfgrestore

Commands that used the ORPHAN flock in shared mode:
  vgimportclone, pvs, pvscan, pvresize, pvmove,
  pvdisplay, pvchange, fullreport

Commands that used the GLOBAL flock in exclusive mode:
  pvchange, pvscan, vgimportclone, vgscan

Commands that used the GLOBAL flock in shared mode:
  pvscan --cache, pvs

The ORPHAN lock covers the important cases of serializing
the use of orphan PVs.  It also partially covers the
reporting of orphan PVs (although not correctly as
explained below.)

The GLOBAL lock doesn't seem to have a clear purpose
(it may have eroded over time.)

Neither lock correctly protects the VG namespace, or
orphan PV properties.

To simplify and correct these issues, the two separate
flocks are combined into the one GLOBAL flock, and this flock
is used from the locking sites that are in place for the
lvmlockd global lock.

The logic behind the lvmlockd (distributed) global lock is
that any command that changes "global state" needs to take
the global lock in ex mode.  Global state in lvm is: the list
of VG names, the set of orphan PVs, and any properties of
orphan PVs.  Reading this global state can use the global lock
in sh mode to ensure it doesn't change while being reported.

The locking of global state now looks like:

lockd_global()
  previously named lockd_gl(), acquires the distributed
  global lock through lvmlockd.  This is unchanged.
  It serializes distributed lvm commands that are changing
  global state.  This is a no-op when lvmlockd is not in use.

lockf_global()
  acquires an flock on a local file.  It serializes local lvm
  commands that are changing global state.

lock_global()
  first calls lockf_global() to acquire the local flock for
  global state, and if this succeeds, it calls lockd_global()
  to acquire the distributed lock for global state.

Replace instances of lockd_gl() with lock_global(), so that the
existing sites for lvmlockd global state locking are now also
used for local file locking of global state.  Remove the previous
file locking calls lock_vol(GLOBAL) and lock_vol(ORPHAN).

The following commands which change global state are now
serialized with the exclusive global flock:

pvchange (of orphan), pvresize (of orphan), pvcreate, pvremove,
vgcreate, vgextend, vgremove, vgreduce, vgrename,
vgcfgrestore, vgimportclone, vgmerge, vgsplit

Commands that use a shared flock to read global state (and will
be serialized against the prior list) are those that use
process_each functions that are based on processing a list of
all VG names, or all PVs.  The list of all VGs or all PVs is
global state and the shared lock prevents those lists from
changing while the command is processing them.

The ORPHAN lock previously attempted to produce an accurate
listing of orphan PVs, but it was only acquired at the end of
the command during the fake vg_read of the fake orphan vg.
This is not when orphan PVs were determined; they were
determined by elimination beforehand by processing all real
VGs, and subtracting the PVs in the real VGs from the list
of all PVs that had been identified during the initial scan.
This is fixed by holding the single global lock in shared mode
while processing all VGs to determine the list of orphan PVs.
2019-04-29 13:01:05 -05:00
David Teigland
ccd1386070 wipe_lv: initially open LV in writable mode
wipe_lv knows it's going to write the device, so it
can open rw from the start.  It was opening readonly,
and then dev_write needed to reopen it readwrite.
2019-04-26 14:49:27 -05:00
David Teigland
d0b869e46a hints: fix non-empty hints list when not using hints
When hints are invalid and ignored, the list of hints
could be non-empty (from additions before an invalid
hint was found).  This confused the calling code which
was checking for an empty list to see if hints were used.
Ensure the list is empty when hints are not used.
2019-04-11 11:58:51 -05:00
David Teigland
0cc80ccfd5 hints: fix case of error getting device size
When checking hints, if there's an error getting
the device size, that should be equivalent to
seeing zero size.
2019-04-11 10:32:28 -05:00
David Teigland
6f18186bfd pvscan: print more reasons for ignoring devices 2019-04-05 15:48:12 -05:00
David Teigland
7edbf8a441 io: increase the default io memory from 4 to 8 MiB
This is the default bcache size that is created at the
start of the command.  It needs to be large enough to
hold a single copy of metadata for a given VG, or the
VG cannot be read or written (since the entire VG would
not fit into available memory.)

Increasing the default reduces the chances of anyone
needing to increase the default to use their VG.

The size can be set in lvm.conf global/io_memory_size;
the lower limit is 4 MiB and the upper limit is 128 MiB.
2019-03-04 12:14:06 -06:00
David Teigland
3584e0c0d5 io: warn when metadata size approaches io memory size
When a single copy of metadata gets within 1MB of the
current io_memory_size value, begin printing a warning
that the io_memory_size should be increased.
2019-03-04 12:13:09 -06:00
David Teigland
dd8d083795 config: add new setting io_memory_size
which defines the amount of memory that lvm will allocate
for bcache.  Increasing this setting is required if it is
smaller than a single copy of VG metadata.
2019-03-04 11:36:21 -06:00
David Teigland
0aa51a2f61 hints: fix recreating hints from pvscan
When aay was included in the pvscan --cache command,
the activation part was complaining about the unusual
state of the hint file since it had been recreated
just prior.
2019-02-13 15:23:43 -06:00
David Teigland
5f102b3421 hints: invalidate when pvscan --cache sees a new PV
An idea from Zdenek for better ensuring valid hints by invalidating
them when pvscan --cache <device> sees a new PV, which is a case
where we know that hints should be invalidated.  This is triggered
from systemd/udev logic, and there may be some cases where it would
invalidate hints that the existing methods wouldn't detect.
2019-01-16 15:34:20 -06:00
David Teigland
7b5abc3fb1 hints: fix hint flock when using lvm shell
also cmd->use_hints needs to be set for each shell command
2019-01-15 12:23:16 -06:00
David Teigland
6620dc9475 add device hints to reduce scanning
Save the list of PVs in /run/lvm/hints.  These hints
are used to reduce scanning in a number of commands
to only the PVs on the system, or only the PVs in a
requested VG (rather than all devices on the system.)
2019-01-15 10:23:47 -06:00
Zdenek Kabelac
0b19387dae headers: use configure.h as 1st. header
Ensure configure.h is always 1st. included header.
Maybe we could eventually introduce gcc -include option, but for now
this better uses dependency tracking.

Also move _REENTRANT and _GNU_SOURCE into configure.h so it
doesn't need to be present in various source files.
This ensures consistent compilation of headers like stdio.h since
it may produce different declaration.
2018-12-14 15:09:13 +01:00
David Teigland
3d2fd95af7 remove unused full filter
it's the same as cmd->filter
2018-12-04 14:06:46 -06:00
Zdenek Kabelac
5a5e3bcf15 gcc: ensure sector is initilized
Some older gcc errnously report the variable can be used uninitlized.
Quite warning by explicit initalization.
2018-12-01 01:07:01 +01:00
David Teigland
ca66d52032 io: use sync io if aio fails
io_setup() for aio may fail if a system has reached the
aio request limit.  In this case, fall back to using
sync io.  Also, lvm use of aio can be disabled entirely
with config setting global/use_aio=0.

The system limit for aio requests can be seen from
  /proc/sys/fs/aio-max-nr

The current usage of aio requests can be seen from
  /proc/sys/fs/aio-nr

The system limit for aio requests can be increased by
setting fs.aio-max-nr using sysctl.

Also add last-byte limit to the sync io code.
2018-11-20 09:13:20 -06:00
Zdenek Kabelac
d3ebb18f40 cov: avoid unsing unchecked label_scan_open
Drop extra call too label_scan_open() without checking return value,
and let code go through next call bellow.
2018-11-05 17:25:11 +01:00
Zdenek Kabelac
1951e0db0f label: add stack trace for failing dev_set_last_byte
Temporarily add check for failure, but whole function
needs to be likely traced for error result.

FIXME
2018-11-03 16:09:36 +01:00
David Teigland
aecf542126 metadata: prevent writing beyond metadata area
lvm uses a bcache block size of 128K.  A bcache block
at the end of the metadata area will overlap the PEs
from which LVs are allocated.  How much depends on
alignments.  When lvm reads and writes one of these
bcache blocks to update VG metadata, it can also be
reading and writing PEs that belong to an LV.

If these overlapping PEs are being written to by the
LV user (e.g. filesystem) at the same time that lvm
is modifying VG metadata in the overlapping bcache
block, then the user's updates to the PEs can be lost.

This patch is a quick hack to prevent lvm from writing
past the end of the metadata area.
2018-10-29 16:53:17 -05:00
Zdenek Kabelac
06a4a356db cov: avoid selfrecursive inclusion of toolcontext.h 2018-10-15 17:49:44 +02:00
Zdenek Kabelac
fdd76da33d cov: drop uneeded header files 2018-10-15 17:49:44 +02:00
Zdenek Kabelac
84f00f5058 cov: add missing error path check for label_scan_open 2018-10-15 17:49:44 +02:00
Zdenek Kabelac
b57e73a0f1 cov: make sure label scans valid lvinfo 2018-10-15 17:49:44 +02:00
David Teigland
763219611c vgcreate: close exclusive fd after pvcreate
When vgcreate does an automatic pvcreate, it opens the
dev with O_EXCL to ensure no other subsystem is using
the device.  This exclusive fd remained in bcache and
prevented activation parts of lvm from using the dev.

This appeared with vgcreate of a sanlock VG because of
the unique combination where the dev is not yet a PV,
so pvcreate is needed, and the vgcreate also creates
and activates an internal LV for sanlock.

Fix this by closing the exclusive fd after it's used
by pvcreate so that it won't interfere with other
bits of lvm that may try to use the device.
2018-08-01 11:22:23 -05:00
David Teigland
117160b27e Remove lvmetad
Native disk scanning is now both reduced and
async/parallel, which makes it comparable in
performance (and often faster) when compared
to lvm using lvmetad.

Autoactivation now uses local temp files to record
online PVs, and no longer requires lvmetad.

There should be no apparent command-level change
in behavior.
2018-07-11 11:26:42 -05:00
David Teigland
42f7caf1c2 scan: work around udev problems by avoiding open RDWR
udev creates a train wreck of events if we open devices
with RDWR.  Until we can fix/disable/scrap udev, work around
this by opening RDONLY and then closing/reopening RDWR when
a write is needed.  This invalidates the bcache blocks for
the device before writing so it can trigger unnecessary
rereading.
2018-06-20 14:08:12 -05:00
David Teigland
3fd75d1bcd scan: use full md filter when md 1.0 devices are present
The md filter can operate in two native modes:
- normal: reads only the start of each device
- full: reads both the start and end of each device

md 1.0 devices place the superblock at the end of the device,
so components of this version will only be identified and
excluded when lvm uses the full md filter.

Previously, the full md filter was only used in commands
that could write to the device.  Now, the full md filter
is also applied when there is an md 1.0 device present
on the system.  This means the 'pvs' command can avoid
displaying md 1.0 components (at the cost of doubling
the i/o to every device on the system.)

(The md filter can operate in a third mode, using udev,
but this is disabled by default because there have been
problems with reliability of the info returned from udev.)
2018-06-15 12:21:25 -05:00
David Teigland
8eab37593e Add cmd arg to more functions
so that it can be used in the filter code
2018-06-15 11:03:55 -05:00
Joe Thornber
d5da55ed85 device_mapper: remove dbg_malloc.
I wrote dbg_malloc before we had valgrind.  These days there's just
no need.
2018-06-08 13:40:53 +01:00
David Teigland
1539e51721 devices: clean up io error messages
Remove the io error message from bcache.c since it is not
very useful without the device path.

Make the io error messages from dev_read_bytes/dev_write_bytes
more user friendly.
2018-06-07 16:17:04 +01:00
David Teigland
7b5b1a9b6f scan: clean exit for alloc failure 2018-06-01 13:15:22 -05:00
Joe Thornber
dbba1e9b93 Merge branch 'master' into 2018-05-11-fork-libdm 2018-06-01 13:04:12 +01:00
David Teigland
6d14d5d16b scan: removed failed paths for devices
Drop a device path when the scan fails to open it.
2018-05-30 09:05:18 -05:00
David Teigland
28c8e95d19 scan: refresh paths and retry open
If scanning fails to open any devices, refresh the
device paths in dev cache, and retry the opens.
2018-05-25 13:09:07 -05:00
David Teigland
28d35e5c59 scan: fix missing close in lib
lib was using dev_test_excl which wasn't closing the device.
Switch code to new io layer with excl open.
Also use exclusive open in some other places.
2018-05-16 14:48:30 -05:00
Joe Thornber
89fdc0b588 Merge branch 'master' into 2018-05-11-fork-libdm 2018-05-16 13:43:02 +01:00
David Teigland
3bbc17a670 scan: use up to 1024 max bcache blocks
Create bcache with one block per device that
will be scanned up to 1024 max blocks.
2018-05-15 15:17:31 -05:00
David Teigland
517d6cc418 scan: add some missing frees
some objects had been moved out of mem pools.
2018-05-14 13:38:16 -05:00
Joe Thornber
7f97c7ea9a build: Don't generate symlinks in include/ dir
As we start refactoring the code to break dependencies (see doc/refactoring.txt),
I want us to use full paths in the includes (eg, #include "base/data-struct/list.h").
This makes it more obvious when we're breaking abstraction boundaries, eg, including a file in
metadata/ from base/
2018-05-14 10:30:20 +01:00
David Teigland
5c9dcd99fd scan: remove unused args from label_read 2018-05-11 14:16:49 -05:00
David Teigland
bbb8040456 dev_cache: drop open_list
devices are now held open only in bcache,
so drop the dev_cache list of open devices
which is unused.
2018-05-11 12:47:56 -05:00
David Teigland
4362013872 bcache: disable fallback to old io
All io has been converted to bcache.
2018-05-11 11:35:56 -05:00
David Teigland
228ed56455 pvck: allow checking at user specified offsets
with the --labelsector option.  We probably don't
need all this code to support any value for this
option; it's unclear how, when, why it would be
used.
2018-05-11 11:23:51 -05:00
David Teigland
57bb46c5e7 filter: use bcache for filter reads
Filters are still applied before any device reading or
the label scan, but any filter checks that want to read
the device are skipped and the device is flagged.

After bcache is populated, but before lvm looks for
devices (i.e. before label scan), the filters are
reapplied to the devices that were flagged above.
The filters will then find the data they need in
bcache.
2018-05-10 16:03:19 -05:00