1
0
mirror of git://sourceware.org/git/lvm2.git synced 2025-01-03 05:18:29 +03:00
Commit Graph

299 Commits

Author SHA1 Message Date
David Teigland
bbb8040456 dev_cache: drop open_list
devices are now held open only in bcache,
so drop the dev_cache list of open devices
which is unused.
2018-05-11 12:47:56 -05:00
David Teigland
57bb46c5e7 filter: use bcache for filter reads
Filters are still applied before any device reading or
the label scan, but any filter checks that want to read
the device are skipped and the device is flagged.

After bcache is populated, but before lvm looks for
devices (i.e. before label scan), the filters are
reapplied to the devices that were flagged above.
The filters will then find the data they need in
bcache.
2018-05-10 16:03:19 -05:00
David Teigland
f4a60fe004 clvmd: saved_vg code and comment formatting 2018-05-03 14:54:48 -05:00
David Teigland
822a8b62be clvmd: don't save cft and buf for saved_vg 2018-05-03 14:54:48 -05:00
David Teigland
c016b573ee clvmd: separate saved_vg from vginfo
The clvmd saved_vg data is independent from the normal lvm
lvmcache vginfo data, so separate saved_vg from vginfo.
Normal lvm doesn't need to use save_vg at all, and in clvmd,
lvmcache changes on vginfo can be made without worrying
about unwanted effects on saved_vg.
2018-05-03 14:54:48 -05:00
David Teigland
a5e13f2eef clvmd: defer freeing saved vgs
To avoid the chance of freeing a saved vg while another
code path is using it, defer freeing saved vgs until
all the lvmcache content is dropped for the vg.
2018-05-03 14:54:48 -05:00
David Teigland
a418f88b76 lvmcache: fix typo in lvmcache_get_saved_vg 2018-05-01 09:06:57 -05:00
David Teigland
c1cd18f21e Remove lvm1 and pool disk formats
There are likely more bits of code that can be removed,
e.g. lvm1/pool-specific bits of code that were identified
using FMT flags.

The vgconvert command can likely be reduced further.

The lvm1-specific config settings should probably have
some other fields set for proper deprecation.
2018-04-30 16:55:02 -05:00
David Teigland
89935ace29 clvmd: keep old saved_vg if it matches new
There is no need to release the old saved_vg
if it matches the new version.
2018-04-30 13:03:15 -05:00
David Teigland
5b6e62dc1f clvmd: drop old saved_vg when returning new saved_vg
In some pvmove tests, clvmd uses the new (precommitted)
saved_vg, but then requests the old saved_vg, and
expects that the new saved_vg be returned instead of
the old.  So, when returning the new saved_vg, forget
the old one so we don't return it again.
2018-04-26 14:57:45 -05:00
David Teigland
cdb8400de2 scan: refresh filters before scan
The filters save information about devices that should
be ignored, so if we need to repeat a scan  (unusual,
but happens in clvmd), we need to update the filters.
2018-04-26 14:48:13 -05:00
David Teigland
4670e9f698 skip some clvmd-specific code in common cases
This, or something like it, can probably be done
in many other places.
2018-04-25 16:40:08 -05:00
David Teigland
1fec86571f clvmd: reuse a vg struct for sequential LV operations
After reading a VG, stash it in lvmcache as "saved_vg".
Before reading the VG again, try to use the saved_vg.
The saved_vg is dropped on VG lock operations.
2018-04-25 16:39:43 -05:00
David Teigland
f8616ac2d8 lvmcache: rename suspended_vg to saved_vg
The copy of the VG which clvmd stashes in lvmcache should
not only be used between suspend and resume, but between
sequential LV operations in clvmd, so that clvmd does not
need to reread the VG for each one.  Prepare for that by
renaming the stashed VG as "saved_vg".
2018-04-25 16:39:43 -05:00
David Teigland
aee27dc7ba scan: skip device rescan in vg_read
For reporting commands (pvs,vgs,lvs,pvdisplay,vgdisplay,lvdisplay)
we do not need to repeat the label scan of devices in vg_read if
they all had matching metadata in the initial label scan.  The
data read by label scan can just be reused for the vg_read.
This cuts the amount of device i/o in half, from two reads of
each device to one.  We have to be careful to avoid repairing
the VG if we've skipped rescanning.  (The VG repair code is very
poor, and will be redone soon.)
2018-04-20 11:23:14 -05:00
David Teigland
9b6a62f944 lvmcache: simplify
Recent changes allow some major simplification of the way
lvmcache works and is used.  lvmcache_label_scan is now
called in a controlled fashion at the start of commands,
and not via various unpredictable side effects.  Remove
various calls to it from other places.  lvmcache_label_scan
should not be called from anywhere during a command, because
it produces an incorrect representation of PVs with no MDAs,
and misclassifies them as orphans.  This has been a long
standing problem.  The invalid flag and rescanning based on
that is no longer used and removed.  The 'force' variation is
no longer needed and removed.
2018-04-20 11:22:48 -05:00
David Teigland
c0973e70a5 dev_cache: clean up scan
Pull out all of the twisted logic and simply call dev_cache_scan
at the start of the command prior to label scan.
2018-04-20 11:22:48 -05:00
David Teigland
ae21305ee7 scan: drop bcache between lvm shell commands
A running lvm shell keeps all lvm devices open
unless the bcache is dropped.
2018-04-20 11:22:48 -05:00
David Teigland
1717d4cb17 lvmcache: add shorter way to delete dev info
Don't make the caller look up the info first.
2018-04-20 11:22:47 -05:00
David Teigland
28255e3eee scan: always setup bcache for commands using lvmetad
Do this at the start of the command so that it doesn't
need to be checked and set up in every function that
could need it.
2018-04-20 11:22:46 -05:00
David Teigland
f17c2cf7c6 pvremove: device check doesn't require label_read
It just needs to check if the device was found during
the scan, which means checking if it exists in lvmcache.
2018-04-20 11:22:45 -05:00
David Teigland
d9a77e8bb4 lvmcache: simplify metadata cache
The copy of VG metadata stored in lvmcache was not being used
in general.  It pretended to be a generic VG metadata cache,
but was not being used except for clvmd activation.  There
it was used to avoid reading from disk while devices were
suspended, i.e. in resume.

This removes the code that attempted to make this look
like a generic metadata cache, and replaces with with
something narrowly targetted to what it's actually used for.

This is a way of passing the VG from suspend to resume in
clvmd.  Since in the case of clvmd one caller can't simply
pass the same VG to both suspend and resume, suspend needs
to stash the VG somewhere that resume can grab it from.
(resume doesn't want to read it from disk since devices
are suspended.)  The lvmcache vginfo struct is used as a
convenient place to stash the VG to pass it from suspend
to resume, even though it isn't related to the lvmcache
or vginfo.  These suspended_vg* vginfo fields should
not be used or touched anywhere else, they are only to
be used for passing the VG data from suspend to resume
in clvmd.  The VG data being passed between suspend and
resume is never modified, and will only exist in the
brief period between suspend and resume in clvmd.

suspend has both old (current) and new (precommitted)
copies of the VG metadata.  It stashes both of these in
the vginfo prior to suspending devices.  When vg_commit
is successful, it sets a flag in vginfo as before,
signaling the transition from old to new metadata.

resume grabs the VG stashed by suspend.  If the vg_commit
happened, it grabs the new VG, and if the vg_commit didn't
happen it grabs the old VG.  The VG is then used to resume
LVs.

This isolates clvmd-specific code and usage from the
normal lvm vg_read code, making the code simpler and
the behavior easier to verify.

Sequence of operations:

- lv_suspend() has both vg_old and vg_new
  and stashes a copy of each onto the vginfo:
  lvmcache_save_suspended_vg(vg_old);
  lvmcache_save_suspended_vg(vg_new);

- vg_commit() happens, which causes all clvmd
  instances to call lvmcache_commit_metadata(vg).
  A flag is set in the vginfo indicating the
  transition from the old to new VG:
  vginfo->suspended_vg_committed = 1;

- lv_resume() needs either vg_old or vg_new
  to use in resuming LVs.  It doesn't want to
  read the VG from disk since devices are
  suspended, so it gets the VG stashed by
  lv_suspend:
  vg = lvmcache_get_suspended_vg(vgid);

If the vg_commit did not happen, suspended_vg_committed
will not be set, and in this case, lvmcache_get_suspended_vg()
will return the old VG instead of the new VG, and it will
resume LVs based on the old metadata.
2018-04-20 11:22:45 -05:00
David Teigland
79c4971210 label_scan: remove extra label scan and read for orphan PVs
When process_each_pv() calls vg_read() on the orphan VG, the
internal implementation was doing an unnecessary
lvmcache_label_scan() and two unnecessary label_read() calls
on each orphan.  Some of those unnecessary label scans/reads
would sometimes be skipped due to caching, but the code was
always doing at least one unnecessary read on each orphan.

The common format_text case was also unecessarily calling into
the format-specific pv_read() function which actually did nothing.

By analyzing each case in which vg_read() was being called on
the orphan VG, we can say that all of the label scans/reads
in vg_read_orphans are unnecessary:

1. reporting commands: the information saved in lvmcache by
the original label scan can be reported.  There is no advantage
to repeating the label scan on the orphans a second time before
reporting it.

2. pvcreate/vgcreate/vgextend: these all share a common
implementation in pvcreate_each_device().  That function
already rescans labels after acquiring the orphan VG lock,
which ensures that the command is using valid lvmcache
information.
2018-04-20 11:22:45 -05:00
David Teigland
748f29b42a scan: do scanning at the start of a command
Move the location of scans to make it clearer and avoid
unnecessary repeated scanning.  There should be one scan
at the start of a command which is then used through the
rest of command processing.

Previously, the initial label scan was called as a side effect
from various utility functions.  This would lead to it being called
unnecessarily.  It is an expensive operation, and should only be
called when necessary.  Also, this is a primary step in the
function of the command, and as such it should be called prominently
at the top level of command processing, not as a hidden side effect
of a utility function.  lvm knows exactly where and when the
label scan needs to be done.  Because of this, move the label scan
calls from the internal functions to the top level of processing.

Other specific instances of lvmcache_label_scan() are still called
unnecessarily or unclearly by specific commands that do not use
the common process_each functions.  These will be improved in
future commits.

During the processing phase, rescanning labels for devices in a VG
needs to be done after the VG lock is acquired in case things have
changed since the initial label scan.  This was being done by way
of rescanning devices that had the INVALID flag set in lvmcache.
This usually approximated the right set of devices, but it was not
exact, and obfuscated the real requirement.  Correct this by using
a new function that rescans the devices in the VG:
lvmcache_label_rescan_vg().

Apart from being inexact, the rescanning was extremely well hidden.
_vg_read() would call ->create_instance(), _text_create_text_instance(),
_create_vg_text_instance() which would call lvmcache_label_scan()
which would call _scan_invalid() which repeats the label scan on
devices flagged INVALID.  lvmcache_label_rescan_vg() is now called
prominently by _vg_read() directly.
2018-04-20 11:21:38 -05:00
David Teigland
4507ba3596 scan: use new label_scan for lvmcache_label_scan
To do label scanning, lvm code calls lvmcache_label_scan().
Change lvmcache_label_scan() to use the new label_scan()
based on bcache.

Also add lvmcache_label_rescan_vg() which calls the new
label_scan_devs() which does label scanning on only the
specified devices.  This is for a subsequent commit and
is not yet used.
2018-04-20 11:19:32 -05:00
Joe Thornber
00f1b208a1 [io paths] Unpick agk's aio stuff 2018-04-20 11:03:58 -05:00
Zdenek Kabelac
b6e7a0b490 cleanup: more usage of dm_strncpy
Use existing wrapper function arournd  strncpy + buf[] = 0;
2018-03-06 15:40:34 +01:00
Alasdair G Kergon
d6cabbbc53 device: Fix basic async I/O error handling 2018-02-08 20:19:21 +00:00
Alasdair G Kergon
db41fe6c5d lvmcache: Use asynchronous I/O when scanning devices. 2018-02-08 20:15:29 +00:00
Alasdair G Kergon
7a9af3cd0e device: Add flag to indicate that a code path can support AIO
Until the whole source supports AIO, library code can check for
AIO_SUPPORTED_CODE_PATH to determine whether or not it is OK
to use AIO.
2018-02-06 01:11:00 +00:00
Alasdair G Kergon
9194610f42 device: Add ioflags parameter to transfer additional state.
Flags are set on the initial I/O and passed to any callbacks that
may in turn issue further I/O using the inherited flags.
2018-01-21 21:10:23 +00:00
Alasdair G Kergon
35cdd9cf48 label: Clean up storing of device and label sector.
No longer use the external 'result' pointer internally to set up the
cached label.  The callback _set_label_read_result() is now given the
internal label pointer directly

Callers that don't need the result are no longer required to pass a
label pointer into label_read().
2018-01-11 02:54:00 +00:00
Alasdair G Kergon
6210c1ec28 device: Mark read-only device buffers const. 2018-01-10 19:57:10 +00:00
Alasdair G Kergon
4d568b709c device: Free cached device bufs when metadata invalid or dev closed. 2018-01-10 15:48:03 +00:00
Alasdair G Kergon
4b02d4e22e label: Add label_read callback. 2018-01-08 23:30:50 +00:00
Alasdair G Kergon
22b6c482ec config: Split config buffer processing into new fn.
Wrap its parameters into struct process_config_file_params allocated
from a mempool now passed into the config_file_read* fns.
2018-01-02 21:10:46 +00:00
Alasdair G Kergon
6bf0f04ae2 log: Improve various device-related messages
- Use 'lvmcache' consistently instead of 'metadata cache'
- Always use 5 characters for source line number
- Remember to convert uuids into printable form
- Use <no name> rather than (null) when VG has no name.
2017-11-13 19:45:33 +00:00
Zdenek Kabelac
1fd8785ff3 tidy: drop unneeded return 2017-07-20 11:20:22 +02:00
Zdenek Kabelac
f7e62bc55c cleanup: drop extra compare
dm_free() already validates for NULL itself.
2017-07-17 12:32:18 +02:00
Zdenek Kabelac
9fe4f2337b cleanup: drop assign before use
Drop unneeded assigns singe vars are set later in code before
their first use (Coverity).
2016-10-03 17:49:55 +02:00
Peter Rajnoha
f1cad4c710 config: use config_tree_from_string_without_dup_node_check throughout code to construct metadata trees 2016-09-21 18:18:15 +02:00
Zdenek Kabelac
37a33d7414 cleanup: warns from older gcc 2016-07-01 00:44:48 +02:00
David Teigland
86f9271457 vgcreate, pvcreate, vgextend: don't use a device with duplicates
If duplicate orphan PVs exist, don't allow one of them to be
used for vgcreate/pvcreate/vgextend.
2016-06-07 15:21:07 -05:00
David Teigland
199b7b55c2 lvmcache: fix duplicate handling with multiple scans
Some commands scan labels to populate lvmcache multiple
times, i.e. lvmcache_init, scan labels to fill lvmcache,
lvmcache_destroy, then later repeat

Each time labels are scanned, duplicates are detected,
and preferred devices are chosen.  Each time this is done
within a single command, we want to choose the same
preferred devices.  So, check for existing preferences
when choosing preferred devices.

This also fixes a problem with the list of unused duplicate
devs when run in an lvm shell.  The devs had been allocated
from cmd memory, resulting in invalid list entries between
commands.
2016-06-07 15:15:51 -05:00
David Teigland
01156de6f7 lvmcache: add optional dev arg to lvmcache_info_from_pvid
A number of places are working on a specific dev when they
call lvmcache_info_from_pvid() to look up an info struct
based on a pvid.  In those cases, pass the dev being used
to lvmcache_info_from_pvid().  When a dev is specified,
lvmcache_info_from_pvid() will verify that the cached
info it's using matches the dev being processed before
returning the info.  Calling code will not mistakenly
get info for the wrong dev when duplicate devs exist.

This confusion was happening when scanning labels when
duplicate devs existed.  label_read for the first dev
would add an info struct to lvmcache for that dev/pvid.
label_read for the second dev would see the pvid in
lvmcache from first dev, and mistakenly conclude that
the label_read from the second dev can be skipped
because it's already been done.  By verifying that the
dev for the cached pvid matches the dev being read,
this mismatch is avoided and the label is actually read
from the second duplicate.
2016-06-07 15:15:47 -05:00
Peter Rajnoha
48877e215d coverity: missing check for id_write_format return value 2016-05-31 09:56:10 +02:00
David Teigland
ba9b7b69d9 pvremove: allow clearing a duplicate PV
Add a special case to allow modifying a duplicate PV
to erase it with pvremove -ff.
2016-05-16 14:40:43 -05:00
David Teigland
49d9582bed lvmcache: use active LVs and device sizes to choose between duplicates
If duplicate devices exist for a PV, and one device's
size matches the PV size, but the other doesn't, then
prefer the matching device.

If one device is used by an active LV, prefer that device.
2016-05-06 09:00:00 -05:00
David Teigland
d4e434d1e6 pvs: new attr and field for unchosen duplicate device
When there are duplicate devices for a PV, one device
is preferred and chosen to exist in the VG.  The other
devices are not used by lvm, but are displayed by pvs
with a new PV attr "d", indicating that they are
unchosen duplicate PVs.

The "duplicate" reporting field is set to "duplicate"
when the PV is an unchosen duplicate, and that field
is blank for the chosen PV.
2016-05-06 09:00:00 -05:00
David Teigland
d3d13e134a lvmcache: process duplicate PVs directly
Previously, duplicate PVs were processed as a side effect
of processing the "chosen" PV in lvmcache.  The duplicate
PV would be hacked into lvmcache temporarily in place of
the chosen PV.

In the old way, we had to always process the "chosen" PV
device, even if a duplicate of it was named on the command
line.  This meant we were processing a different device than
was asked for.  This could be worked around by naming
multiple duplicate devs on the command line in which case
they were swapped in and out of lvmcache for processing.

Now, the duplicate devs are processed directly in their
own processing loop.  This means we can remove the old
hacks related to processing dups as a side effect of
processing the chosen device.  We can now simply process
the device that was named on the command line.

When the same PVID exists on two or more devices, one device
is preferred and used in the VG, and the others are duplicates
and are not used in the VG.  The preferred device exists in
lvmcache as usual.  The duplicates exist in a specical list
of unused duplicate devices.

The duplicate devs have the "d" attribute and the "duplicate"
reporting field displays "duplicate" for them.

'pvs' warns about duplicates, but the formal output only
includes the single preferred PV.

'pvs -a' has the same warnings, and the duplicate devs are
included in the output.

'pvs <path>' has the same warnings, and displays the named
device, whether it is preferred or a duplicate.
2016-05-06 09:00:00 -05:00
David Teigland
8b7a78c728 lvmcache: improve duplicate PV handling
Wait to compare and choose alternate duplicate devices until
after all devices are scanned.  During scanning, the first
duplicate dev is kept in lvmcache, and others are kept in a
new list (_found_duplicate_devs).

After all devices are scanned, compare all the duplicates
available for a given PVID and decide which is best.

If the dev used in lvmcache is changed, drop the old dev
from lvmcache entirely and rescan the replacement dev.
Previously the VG metadata from the old dev was kept in
lvmcache and only the dev was replaced.

A new config setting devices/allow_changes_with_duplicate_pvs
can be set to 0 which disallows modifying a VG or activating
LVs in it when the VG contains PVs with duplicate devices.
Set to 1 is the old behavior which allowed the VG to be
changed.

The logic for which of two devs is preferred has changed.
The primary goal is to choose a device that is currently
in use if the other isn't, e.g. by an active LV.

. prefer dev with fs mounted if the other doesn't, else
. prefer dev that is dm if the other isn't, else
. prefer dev in subsystem if the other isn't

If neither device is preferred by these rules, then don't
change devices in lvmcache, leaving the one that was found
first.

The previous logic for preferring a device was:

. prefer dev in subsystem if the other isn't, else
. prefer dev without holders if the other has holders, else
. prefer dev that is dm if the other isn't
2016-05-06 09:00:00 -05:00
David Teigland
67da017fd2 lvmetad: remove client side altdev code
This is no longer used since lvmetad no longer
keeps track of alternate devices for duplicate PVs,
but is simply disabled when duplicates appear.
2016-05-06 09:00:00 -05:00
David Teigland
9539ee8098 lvmetad: set disabled flag in lvmetad if duplicate PVs are found
When devices are being scanned, if duplicate PVs are seen,
tell lvmetad to set its disabled flag because of duplicate PVs.
2016-05-06 08:59:59 -05:00
David Teigland
5e9e43074a lvmetad: rework command connection setup and checking
The lvmetad connection is created within the
init_connections() path during command startup,
rather than via the old lvmetad_active() check.

The old lvmetad_active() checks are replaced
with lvmetad_used() which is a simple check that
tests if the command is using/connected to lvmetad.

The old lvmetad_set_active(cmd, 0) calls, which
stopped the command from using lvmetad (to revert to
disk scanning), are replaced with lvmetad_make_unused(cmd).
2016-04-19 14:00:02 -05:00
Zdenek Kabelac
a28c81cbae debug: unify some tracing messages
Introduce  FMTVGID - although it might be possibly better to ensure
vgid is always \0 ended string.

Unify some lvmcache reported messages.
2016-04-12 13:06:16 +02:00
Zdenek Kabelac
af148a9d77 cleanup: avoid gcc warns
Some older systems may had global declaration to cause gcc warning.
Rename for cases we don't care...
2016-04-08 20:20:16 +02:00
Zdenek Kabelac
1a9f743eef cleanup: drop unneeded test
dmfree tests for NULL
2016-02-25 23:30:25 +01:00
Peter Rajnoha
42ae586fa7 lvmcache: fix missing free of vginfo->system_id causing mem leak 2016-02-25 16:25:27 +01:00
Peter Rajnoha
71ea2e1602 lvmcache/lvmetad: cache PV extension version
Store PV extension version in lvmcache/lvmetad for use throughout the code.
2016-02-15 12:44:46 +01:00
Peter Rajnoha
7593221f94 lvmcache/lvmetad: cache PV extension flags
Store PV extension flags in lvmcache/lvmetad for use throughout the code.
2016-02-15 12:44:46 +01:00
Peter Rajnoha
1f5dfb7369 lvmcache: invalidate all cached dev sizes if all VGs got unlocked 2016-01-22 14:16:00 +01:00
Zdenek Kabelac
fcbef05aae doc: change fsf address
Hmm rpmlint suggest fsf is using a different address these days,
so lets keep it up-to-date
2016-01-21 12:11:37 +01:00
David Teigland
e710bac03d Revert "lvmcache: skip drop when vg_write lock is not held"
This reverts e28e22b9e1
The problem that that commit was fixing (pytest failure)
no longer appears with the current code, so the commit is
not needed.

That commit is a problem for pvchange, because it prevents
lvmcache from retaining VG metadata even while the global
lock is held.  pvchange holds the global lock to ensure
that VG metadata is kept in lvmcache throughout processing.
If the cache is not kept, a PV with zero MDAs will appear
first in its actual VG and then appear again in the orphan VG.
It wrongly appears a second time in the orphan VG only if
the actual VG is dropped from lvmcache.
2016-01-14 13:34:36 -06:00
David Teigland
796461a912 vgrename: use process_each_vg
Use process_each_vg() to lock and read the old VG,
and then call the main vgrename code.

When real VG names are used (not a UUID in place of the
old name), the command still pre-locks the new name
(when strcmp wants it locked first), before calling
process_each_vg on the old name.

In the case where the old name is replaced with a UUID,
process_each_vg now translates that UUID into the real
VG name, which it locks and reads.  In this case, we
cannot do pre-locking to maintain lock ordering because
the old name is unknown.  So, in this case the strcmp
based lock ordering is suppressed and the old name is
always locked first.  This opens a remote chance for
lock ordering conflict between racing vgrenames between
two names where one or both commands use the UUID.
2015-12-14 14:26:47 -06:00
David Teigland
92e1422707 process_each_pv: do full scan earlier to find new devices
Before commit c1f246fedf,
_get_all_devices() did a full device scan before
get_vgnameids() was called.  The full scan in
_get_all_devices() is from calling dev_iter_create(f, 1).
The '1' arg forces a full scan.

By doing a full scan in _get_all_devices(), new devices
were added to dev-cache before get_vgnameids() began
scanning labels.  So, labels would be read from new devices.
(e.g. by the first 'pvs' command after the new device appeared.)

After that commit, _get_all_devices() was called
after get_vgnameids() was finished scanning labels.
So, new devices would be missed while scanning labels.
When _get_all_devices() saw the new devices (after
labels were scanned), those devices were added to
the .cache file.  This meant that the second 'pvs'
command would see the devices because they would be
in .cache.

Now, the full device scan is factored out of
_get_all_devices() and called by itself at the
start of the command so that new devices will
be known before get_vgnameids() scans labels.
2015-12-14 10:02:29 -06:00
David Teigland
3bcdf5d14b lvmcache: change duplicate VG name warnings to verbose
When two different VGs with the same name exist,
they are both stored in lvmcache using the vginfo->next
list.  Previously, the code would print warnings (sometimes)
when adding VGs to this list.  Now the duplicate VG names
are handled by higher level code, so this list no longer
needs to print warnings about duplicate VG names being found.
2015-12-01 09:30:23 -06:00
David Teigland
88cef47b18 vg_read: look up vgid from name
After recent changes to process_each, vg_read() is usually
given both the vgname and vgid for the intended VG.

However, in some cases vg_read() is given a vgid with
no vgname, or is given a vgname with no vgid.

When given a vgid with no vgname, vg_read() uses lvmcache
to look up the vgname using the vgid.  If the vgname is
not found, vg_read() fails.

When given a vgname with no vgid, vg_read() should also
use lvmcache to look up the vgid using the vgname.
If the vgid is not found, vg_read() fails.

If the lvmcache lookup finds multiple vgids for the
vgname, then the lookup fails, causing vg_read() to fail
because the intended VG is uncertain.

Usually, both vgname and vgid for the intended VG are passed
to vg_read(), which means the lvmcache translations
between vgname and vgid are not done.
2015-12-01 09:18:48 -06:00
David Teigland
eb22f7c8f7 lvmcache: new function to check if VG is foreign
When not using lvmetad, this uses the system_id field in
the cached vginfo structs that are populated during a scan.

When using lvmetad, this requests the VG from lvmetad, and
checks the system_id field in the returned metadata.
2015-11-30 11:54:56 -06:00
David Teigland
d3ca18e489 lvmcache: include system_id in vginfo cache
Save system_id just like creation_host and lock_type
strings in vginfo cache.
2015-11-30 11:32:17 -06:00
David Teigland
4103896ca0 lvmcache: change log_verbose to log_warn
Without this, some cases miss printing a
warning for duplicate PVs.
2015-11-03 13:37:31 -06:00
Peter Rajnoha
3a42c13ccf lvmcache: update cached info properly when moving from VG to orphan while lvmetad is used
When lvmetad is used and lvmcache update function (lvmcache_update_vgname_and_id)
was called to update existing lvmcache records, a condition was met
which made to retun from the update function immediately, effectively
making it NOOP.

It seems there's no reason for such condition and lvmcache should be
update appropriately even when lvmetad used as lvmcache may be reused,
most notably in lvm shell.

It's possible this is a remnant of the lvmetad development code which
didn't get removed for some reason and the bug didn't get spotted
because lvm shell is not used often (the condition dates back to 2012
or so).

Example, lvmetad and lvm shell used:

lvm> pvs
  PV         VG   Fmt  Attr PSize   PFree
  /dev/sda   vg   lvm2 a--  124.00m 124.00m

Before this patch:
==================
lvm> vgremove vg
  Volume group "vg" successfully removed

lvm> pvs

With this patch applied:
========================

lvm> vgremove vg
  Volume group "vg" successfully removed

lvm> pvs
  PV         VG   Fmt  Attr PSize   PFree
  /dev/sda        lvm2 ---  128.00m 128.00m
2015-10-23 15:58:31 +02:00
David Teigland
83d475626a pvscan: use process_each_pv
The old code made two loops through the PVs: in the first
loop it found the max PV and VG name lengths, and in the
second loop it printed each PV using the name lengths as
field widths for aligning columns.

The new code uses process_each_pv() which makes one loop
through the PVs.  In the *first* call to pvscan_single(),
the max name lengths are found by looping through the
lvmcache entries which have been populated by the generic
process_each code prior to calling any _single functions.
Subsequent calls to pvscan_single() reuse the max lengths
that were found by the first call.
2015-10-19 16:15:51 -05:00
Alasdair G Kergon
ce80d73684 lvmcache: Remove verbose msg when rescanning dev. 2015-09-21 19:51:15 +01:00
Zdenek Kabelac
28b4fa3e27 Revert "lvmcache: check for too long pvid"
This reverts commit 70db1d523d.
Since we use 'strncpy' even for case where it exactly matches
the buffer size and \0 is not expected to be added there.
2015-08-18 15:22:13 +02:00
Zdenek Kabelac
70db1d523d lvmcache: check for too long pvid 2015-08-18 14:53:36 +02:00
David Teigland
772b54a08b vgcreate: improve checks for existing global lock
This tries harder to avoid creating duplicate global locks in
sanlock VGs by refusing to create a new sanlock VG with a
global lock if other sanlock VGs exist that may have a gl.
2015-07-29 14:27:32 -05:00
David Teigland
e593213b87 lvmcache: add lock_type to VG summary and info structs
vgsummary information contains provisional VG information
that is obtained without holding the VG lock.  This info
can be used to lock the VG, and then read it with vg_read().
After the VG is read properly, the vgsummary info should
be verified.

Add the VG lock_type to the vgsummary.  It needs to be
known before the VG can be locked and read.
2015-07-29 14:27:32 -05:00
David Teigland
074295245b pvcreate: remove recent warning message
log_warn was added recently because no known code used
the given condition, but running pvcreate on an existing
PV uses this case, and should not produce a warning.
2015-07-09 15:26:32 -05:00
David Teigland
be23fae488 lvmcache: set device in label when switching devs V2
This is an alternative/equivalent to commit
ca67cf84df

The problem (wrong label->dev after a new preferred
duplicate device is chosen) was isolated to the lvmetad
case (non-lvmetad worked fine), and this fixes the problem
by setting the new label->dev in the lvmetad-specific
code rather than in the general lvmcache code.
2015-05-08 17:10:53 -05:00
David Teigland
ca67cf84df lvmcache: set device in label when switching devs
When using lvmetad, the reporter code was not reporting
the new preferred device because the new preferred dev
was not being set in the label struct.
2015-05-08 14:41:05 -05:00
David Teigland
8e509b5dd5 toollib: avoid repeated lvmetad vg_lookup
In process_each_{vg,lv,pv} when no vgname args are given,
the first step is to get a list of all vgid/vgname on the
system.  This is exactly what lvmetad returns from a
vg_list request.  The current code is doing a vg_lookup
on each VG after the vg_list and populating lvmcache with
the info for each VG.  These preliminary vg_lookup's are
unnecessary, because they will be done again when the
processing functions call vg_read.  This patch eliminates
the initial round of vg_lookup's, which can roughly cut in
half the number of lvmetad requests and save a lot of extra work.
2015-05-08 11:44:55 -05:00
David Teigland
f25132b354 lvmcache: choose preferred device once
Once the preferred duplicate devices have been set in
lvmcache, don't attempt to pick preferred duplicates
again.
2015-05-08 11:44:49 -05:00
Zdenek Kabelac
bf5cb4af8e lvmcache: copy just 32bytes
Copy only bytes which fits.

vginfo->vgid  is  [ID_LEN + 1]
vgsummary->vgid has only [ID_LEN]

Reported by Coverity.
2015-05-08 13:31:59 +02:00
David Teigland
beb229e1e8 devices: improve handling of duplicate PVs
Example:

/dev/loop0 and /dev/loop1 are duplicates,
created by copying one backing file to the
other.

'identity /dev/loopX' creates an identity
mapping for loopX named idmloopX, which
adds a duplicate for the named device.

The duplicate selection code for lvmetad is
incomplete, and lvmetad is disabled for this
example.

[~]# losetup -f loopfile0
[~]# pvs
  PV         VG           Fmt  Attr PSize   PFree
  /dev/loop0 foo          lvm2 a--  308.00m 296.00m

[~]# losetup -f loopfile1
[~]# pvs
  Found duplicate PV LnSOEqzEYED3RvIOa5PZP2s7uyuBLmAV: using /dev/loop1 not /dev/loop0
  Using duplicate PV /dev/loop1 which is more recent, replacing /dev/loop0
  PV         VG           Fmt  Attr PSize   PFree
  /dev/loop1 foo          lvm2 a--  308.00m 308.00m

[~]# ./identity /dev/loop0
[~]# pvs
  Found duplicate PV LnSOEqzEYED3RvIOa5PZP2s7uyuBLmAV: using /dev/loop1 not /dev/loop0
  Using duplicate PV /dev/loop1 without holders, replacing /dev/loop0
  Found duplicate PV LnSOEqzEYED3RvIOa5PZP2s7uyuBLmAV: using /dev/mapper/idmloop0 not /dev/loop1
  Using duplicate PV /dev/mapper/idmloop0 from subsystem DM, replacing /dev/loop1
  PV                   VG           Fmt  Attr PSize   PFree
  /dev/mapper/idmloop0 foo          lvm2 a--  308.00m 296.00m

[~]# ./identity /dev/loop1
[~]# pvs
  WARNING: duplicate PV LnSOEqzEYED3RvIOa5PZP2s7uyuBLmAV is being used from both devices /dev/loop0 and /dev/loop1
  Found duplicate PV LnSOEqzEYED3RvIOa5PZP2s7uyuBLmAV: using /dev/loop1 not /dev/loop0
  Using duplicate PV /dev/loop1 which is more recent, replacing /dev/loop0
  Found duplicate PV LnSOEqzEYED3RvIOa5PZP2s7uyuBLmAV: using /dev/mapper/idmloop0 not /dev/loop1
  Using duplicate PV /dev/mapper/idmloop0 from subsystem DM, replacing /dev/loop1
  Found duplicate PV LnSOEqzEYED3RvIOa5PZP2s7uyuBLmAV: using /dev/mapper/idmloop1 not /dev/mapper/idmloop0
  Using duplicate PV /dev/mapper/idmloop1 which is more recent, replacing /dev/mapper/idmloop0
  PV                   VG           Fmt  Attr PSize   PFree
  /dev/mapper/idmloop1 foo          lvm2 a--  308.00m 308.00m
2015-04-28 10:41:32 -05:00
David Teigland
3fc9615d15 Improve duplicate PV handling
Make the processing of duplicate PVs the
same with and without lvmetad.
2015-04-22 13:22:14 -05:00
Alasdair G Kergon
a9d48bae2f cache: Set correct vgid when changing PV header.
pv_write is called both to write orphans and to rewrite PV headers
of PVs in VGs.  It needs to select the correct VG id so that the
internal cache state gets updated correctly.

It only affected commands that involved further steps after
the pv_write and was often masked because the metadata would
be re-read off disk and correct itself.

"Incorrect metadata area header checksum" warnings appeared.

Example:
  Create vg1 containing dev1, dev2 and dev3.
  Hide dev1 and dev2 from the system.
  Fix up vg1 with vgreduce --removemissing.
  Bring back dev1 and dev2.
  In a single operation reinstate dev1 and dev2 into vg1 (vgextend).
Done as separate operations (automatically fix-up dev1 and dev2 as orphans,
then vgextend) it worked, but done all in one go the internal cache got
corrupted and warnings about checksum errors appeared.
2015-04-09 21:13:55 +01:00
Zdenek Kabelac
e28e22b9e1 lvmcache: skip drop when vg_write lock is not held
Commit 80f4b4b803
introduced undesirable side-effects for lvm2app user
which happens to be our own python binding.

It appear obtaing pvs list keeps global lock.

So restricting this to VG_GLOBAL READ locks and skip
the drop skip if WRITE lock is held.
2015-04-02 13:38:32 +02:00
Alasdair G Kergon
6407d184d1 cache: Store metadata size and checksum.
Refactor the recent metadata-reading optimisation patches.

Remove the recently-added cache fields from struct labeller
and struct format_instance.

Instead, introduce struct lvmcache_vgsummary to wrap the VG information
that lvmcache holds and add the metadata size and checksum to it.

Allow this VG summary information to be looked up by metadata size +
checksum.  Adjust the debug log messages to make it clear when this
shortcut has been successful.

(This changes the optimisation slightly, and might be extendable
further.)

Add struct cached_vg_fmtdata to format-specific vg_read calls to
preserve state alongside the VG across separate calls and indicate
if the details supplied match, avoiding the need to read and
process the VG metadata again.
2015-03-18 23:43:02 +00:00
Alasdair G Kergon
80f4b4b803 cache: Retain orphans while global lock held.
Fixes segfault when 'pvs' encounters two different PVs sharing the same
uuid but one an orphan, the other in a VG.

If VG_GLOBAL is held, there seems no point in doing a full scan more
than once.

If undesirable side-effects show up, we can try restricting this to
VG_GLOBAL READ locks.  The original code dates back to 2.02.40.
2015-03-18 23:20:09 +00:00
David Teigland
b64da4d8b5 toollib: search for duplicate PVs only when needed
A full search for duplicate PVs in the case of pvs -a
is only necessary when duplicates have previously been
detected in lvmcache.  Use a global variable from lvmcache
to indicate that duplicate PVs exist, so we can skip the
search for duplicates when none exist.
2015-01-14 14:47:08 -06:00
David Teigland
57d74a45a0 toollib: override the PV device with duplicates
When multiple duplicate devices are specified on the
command line, the PV is processed once for each of them,
but pv->dev is the device used each time.

This overrides the PV device to reflect the duplicate
device that was specified on the command line.  This is
done by hacking the lvmcache to replace pv->dev with the
device of the duplicate being processed.  (It would be
preferable to override pv->dev without munging the content
of the cache, and without sprinkling special cases throughout
the code.)

This override only applies when multiple duplicate devices are
specified on the command line.  When only a single duplicate
device of pv->dev is specified, the priority is to display the
cached pv->dev, so pv->dev is not overridden by the named
duplicate device.

In the examples below, loop3 is the cached device referenced
by pv->dev, and is given priority for processing.  Only after
loop3 is processed/displayed, will other duplicate devices
loop0/loop1 appear (when requested on the command line.)

With two duplicate devices, loop0 and loop3:

  # pvs
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop3 not /dev/loop0
  PV         VG           Fmt  Attr PSize   PFree
  /dev/loop3 loopa        lvm2 a--   12.00m  12.00m

  # pvs /dev/loop3
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop3 not /dev/loop0
  PV         VG    Fmt  Attr PSize  PFree
  /dev/loop3 loopa lvm2 a--  12.00m 12.00m

  # pvs /dev/loop0
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop3 not /dev/loop0
  PV         VG    Fmt  Attr PSize  PFree
  /dev/loop3 loopa lvm2 a--  12.00m 12.00m

  # pvs -o+dev_size /dev/loop0 /dev/loop3
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop3 not /dev/loop0
  PV         VG    Fmt  Attr PSize  PFree  DevSize
  /dev/loop0 loopa lvm2 a--  12.00m 12.00m  16.00m
  /dev/loop3 loopa lvm2 a--  12.00m 12.00m  32.00m

With three duplicate devices, loop0, loop1, loop3:

  # pvs -o+dev_size
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop1 not /dev/loop0
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop3 not /dev/loop1
  PV         VG           Fmt  Attr PSize   PFree   DevSize
  /dev/loop3 loopa        lvm2 a--   12.00m  12.00m  32.00m

  # pvs -o+dev_size /dev/loop3
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop1 not /dev/loop0
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop3 not /dev/loop1
  PV         VG    Fmt  Attr PSize  PFree  DevSize
  /dev/loop3 loopa lvm2 a--  12.00m 12.00m  32.00m

  # pvs -o+dev_size /dev/loop0
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop1 not /dev/loop0
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop3 not /dev/loop1
  PV         VG    Fmt  Attr PSize  PFree  DevSize
  /dev/loop3 loopa lvm2 a--  12.00m 12.00m  32.00m

  # pvs -o+dev_size /dev/loop1
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop1 not /dev/loop0
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop3 not /dev/loop1
  PV         VG    Fmt  Attr PSize  PFree  DevSize
  /dev/loop3 loopa lvm2 a--  12.00m 12.00m  32.00m

  # pvs -o+dev_size /dev/loop3 /dev/loop0
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop1 not /dev/loop0
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop3 not /dev/loop1
  PV         VG    Fmt  Attr PSize  PFree  DevSize
  /dev/loop0 loopa lvm2 a--  12.00m 12.00m  16.00m
  /dev/loop3 loopa lvm2 a--  12.00m 12.00m  32.00m

  # pvs -o+dev_size /dev/loop3 /dev/loop1
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop1 not /dev/loop0
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop3 not /dev/loop1
  PV         VG    Fmt  Attr PSize  PFree  DevSize
  /dev/loop1 loopa lvm2 a--  12.00m 12.00m  32.00m
  /dev/loop3 loopa lvm2 a--  12.00m 12.00m  32.00m

  # pvs -o+dev_size /dev/loop0 /dev/loop1
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop1 not /dev/loop0
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop3 not /dev/loop1
  PV         VG    Fmt  Attr PSize  PFree  DevSize
  /dev/loop1 loopa lvm2 a--  12.00m 12.00m  32.00m
  /dev/loop3 loopa lvm2 a--  12.00m 12.00m  32.00m

  # pvs -o+dev_size /dev/loop0 /dev/loop1 /dev/loop3
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop1 not /dev/loop0
  Found duplicate PV XhLbpVo0hmuwrMQLjfxuAvPFUFZqD4vr: using /dev/loop3 not /dev/loop1
  PV         VG    Fmt  Attr PSize  PFree  DevSize
  /dev/loop0 loopa lvm2 a--  12.00m 12.00m  16.00m
  /dev/loop1 loopa lvm2 a--  12.00m 12.00m  32.00m
  /dev/loop3 loopa lvm2 a--  12.00m 12.00m  32.00m
2015-01-14 11:57:29 -06:00
David Teigland
c1f246fedf toollib: handle duplicate pvs in process_in_pv
Processes a PV once for each time a device with its PV ID
exists on the command line.

This fixes a regression in the case where:

. devices /dev/sdA and /dev/sdB where clones (same PV ID)

. the cached VG references /dev/sdA

. before the regression, the command: pvs /dev/sdB
  would display the cached device clone /dev/sdA

. after the regression, pvs /dev/sdB would display nothing,
  causing vgimportclone /dev/sdB to fail.

. with this fix, pvs /dev/sdB displays /dev/sdA

Also, pvs /dev/sdA /dev/sdB will report two lines, one for each
device on the command line, but /dev/sdA is displayed for each.

This only works without lvmetad.
2015-01-14 11:57:29 -06:00
Alasdair G Kergon
aeb4d3b740 lvmcache: Skip VG_GLOBAL cache lock state update.
Avoids message:
  Metadata cache has no info for vgname: "#global"
2014-10-07 01:15:43 +01:00
Peter Rajnoha
5011cac9cf filters: add cmd->full_filter - composite of cmd->filter and cmd->lvmetad_filter
There are actually three filter chains if lvmetad is used:
  - cmd->lvmetad_filter used when when scanning devices for lvmetad
  - cmd->filter used when processing lvmetad responses
  - cmd->full_fiilter (which is just cmd->lvmetad_filter + cmd->filter chained together) used
    for remaining situations

This patch adds the third one - "cmd->full_filter" - currently this is
used if device processing does not fall into any of the groups before,
for example, devices which does not have the PV label yet and we're just
creating a new one or we're processing the devices where the list of the
devices (PVs) is not returned by lvmetad initially.

Currently, the cmd->full_filter is used exactly in these functions:
  - lvmcache_label_scan
  - _pvcreate_check
  - pvcreate_vol
  - lvmdiskscan
  - pvscan
  - _process_each_label

If lvmetad is used, then simply cmd->full_filter == cmd->filter because
cmd->lvmetad_filter is NULL in this case.
2014-10-02 13:06:46 +02:00
Zdenek Kabelac
9d2c445d0a cleanup: just safely copy string
Keep analyzers happier and use constrained strcpy.
2014-07-04 12:31:17 +02:00
Petr Rockai
ee200ddfc3 pvremove: Update lvmcache => avoid spurious error messages. 2014-06-08 22:57:04 +02:00
Zdenek Kabelac
e5382c063d lvmcache: fix debug trace
Recent debug tracing commit introduce read of uninitialized memory,
since VGID is not really a proper string which ends with '\0'.
Enforce at most 32 (ID_LEN) chars are read from vgid.
(in release fix)
2014-03-21 22:29:22 +01:00
Zdenek Kabelac
fc280bcc42 lvmcache: handle reinit without error
Since commit f12ee43f2e call destroy,
it start to check all VGs are unlocked. However when we become_daemon,
we simply reset locking (since lock is still kept by parent process).
So implement a simple 'reset' flag.
2014-03-21 22:29:21 +01:00
Zdenek Kabelac
599a05f658 lvmcache: add mode debug prints
Decorate NULL returns with debug_cache output so the
debug log doesn't contain spurios <bactrace> line without
any reason for it.

Add internal errors when cache is misused.
2014-03-18 10:28:08 +01:00
Zdenek Kabelac
0118d6aa48 cleanup: spelling 2014-02-24 21:13:35 +01:00