shaba/lvm2
1
0
Fork 0
mirror of git://sourceware.org/git/lvm2.git synced 2024-12-21 13:34:40 +03:00
Code Releases Activity
abc9265a06
lvm2/lib/metadata/metadata.c

5319 lines
142 KiB
C
Raw Normal View History

lvactivate checkpoint commit
2001-09-25 16:49:28 +04:00
/*
Some whitespace tidy-ups.
2008-01-30 17:00:02 +03:00
* Copyright (C) 2001-2004 Sistina Software, Inc. All rights reserved.
Check all tags and LV names are in a valid form in vg_validate.
2012-02-23 04:11:01 +04:00
* Copyright (C) 2004-2012 Red Hat, Inc. All rights reserved.
lvactivate checkpoint commit
2001-09-25 16:49:28 +04:00
*
Update copyright notices.
2004-03-30 23:35:44 +04:00
* This file is part of LVM2.
*
* This copyrighted material is made available to anyone wishing to use,
* modify, copy, or redistribute it subject to the terms and conditions
Fix inconsistent licence notices: executables are GPLv2; libraries LGPLv2.1.
2007-08-21 00:55:30 +04:00
* of the GNU Lesser General Public License v.2.1.
Update copyright notices.
2004-03-30 23:35:44 +04:00
*
Fix inconsistent licence notices: executables are GPLv2; libraries LGPLv2.1.
2007-08-21 00:55:30 +04:00
* You should have received a copy of the GNU Lesser General Public License
Update copyright notices.
2004-03-30 23:35:44 +04:00
* along with this program; if not, write to the Free Software Foundation,
doc: change fsf address Hmm rpmlint suggest fsf is using a different address these days, so lets keep it up-to-date
2016-01-21 13:49:46 +03:00
* Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
lvactivate checkpoint commit
2001-09-25 16:49:28 +04:00
*/
build: Don't generate symlinks in include/ dir As we start refactoring the code to break dependencies (see doc/refactoring.txt), I want us to use full paths in the includes (eg, #include "base/data-struct/list.h"). This makes it more obvious when we're breaking abstraction boundaries, eg, including a file in metadata/ from base/
2018-05-14 12:30:20 +03:00
#include "lib/misc/lib.h"
#include "lib/device/device.h"
#include "lib/metadata/metadata.h"
#include "lib/commands/toolcontext.h"
#include "lib/misc/lvm-string.h"
#include "lib/misc/lvm-file.h"
#include "lib/cache/lvmcache.h"
#include "lib/mm/memlock.h"
#include "lib/datastruct/str_list.h"
#include "lib/metadata/pv_alloc.h"
#include "lib/metadata/segtype.h"
#include "lib/activate/activate.h"
#include "lib/display/display.h"
#include "lib/locking/locking.h"
#include "lib/format_text/archiver.h"
move wipe_outdated_pvs to vg_write and implement it based on a device, not based on a pv struct (which is not available when the device is not a part of the vg.) currently only the vgremove command wipes outdated pvs until more advanced recovery is added in a subsequent commit
2019-02-06 21:32:26 +03:00
#include "lib/format_text/format-text.h"
#include "lib/format_text/layout.h"
#include "lib/format_text/import-export.h"
build: Don't generate symlinks in include/ dir As we start refactoring the code to break dependencies (see doc/refactoring.txt), I want us to use full paths in the includes (eg, #include "base/data-struct/list.h"). This makes it more obvious when we're breaking abstraction boundaries, eg, including a file in metadata/ from base/
2018-05-14 12:30:20 +03:00
#include "lib/config/defaults.h"
#include "lib/locking/lvmlockd.h"
#include "lib/notify/lvmnotify.h"
lvactivate checkpoint commit
2001-09-25 16:49:28 +04:00
move wipe_outdated_pvs to vg_write and implement it based on a device, not based on a pv struct (which is not available when the device is not a part of the vg.) currently only the vgremove command wipes outdated pvs until more advanced recovery is added in a subsequent commit
2019-02-06 21:32:26 +03:00
#include <time.h>
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
#include <math.h>
Fix PE_ALIGN for pagesize over 32KB.
2006-08-17 23:53:36 +04:00
Some whitespace tidy-ups.
2008-01-30 17:00:02 +03:00
static struct physical_volume *_pv_read(struct cmd_context *cmd,
label_scan: remove extra label scan and read for orphan PVs When process_each_pv() calls vg_read() on the orphan VG, the internal implementation was doing an unnecessary lvmcache_label_scan() and two unnecessary label_read() calls on each orphan. Some of those unnecessary label scans/reads would sometimes be skipped due to caching, but the code was always doing at least one unnecessary read on each orphan. The common format_text case was also unecessarily calling into the format-specific pv_read() function which actually did nothing. By analyzing each case in which vg_read() was being called on the orphan VG, we can say that all of the label scans/reads in vg_read_orphans are unnecessary: 1. reporting commands: the information saved in lvmcache by the original label scan can be reported. There is no advantage to repeating the label scan on the orphans a second time before reporting it. 2. pvcreate/vgcreate/vgextend: these all share a common implementation in pvcreate_each_device(). That function already rescans labels after acquiring the orphan VG lock, which ensures that the command is using valid lvmcache information.
2017-11-06 21:09:52 +03:00
const struct format_type *fmt,
struct volume_group *vg,
struct lvmcache_info *info);
Add wrappers to functions related to pv commands in preparation for exported LVM lib
2007-06-11 22:29:30 +04:00
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
static int _check_pv_ext(struct cmd_context *cmd, struct volume_group *vg)
{
struct lvmcache_info *info;
uint32_t ext_version, ext_flags;
struct pv_list *pvl;
if (vg_is_foreign(vg))
return 1;
if (vg_is_shared(vg))
return 1;
dm_list_iterate_items(pvl, &vg->pvs) {
if (is_missing_pv(pvl->pv))
continue;
/* is_missing_pv doesn't catch NULL dev */
if (!pvl->pv->dev)
continue;
if (!(info = lvmcache_info_from_pvid(pvl->pv->dev->pvid, pvl->pv->dev, 0)))
continue;
ext_version = lvmcache_ext_version(info);
if (ext_version < PV_HEADER_EXTENSION_VSN) {
log_warn("WARNING: PV %s in VG %s is using an old PV header, modify the VG to update.",
dev_name(pvl->pv->dev), vg->name);
continue;
}
ext_flags = lvmcache_ext_flags(info);
if (!(ext_flags & PV_EXT_USED)) {
log_warn("WARNING: PV %s in VG %s is missing the used flag in PV header.",
dev_name(pvl->pv->dev), vg->name);
}
}
return 1;
}
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
/*
* Historically, DEFAULT_PVMETADATASIZE was 255 for many years,
* but that value was only used if default_data_alignment was
* disabled. Using DEFAULT_PVMETADATASIZE 255, pe_start was
* rounded up to 192KB from aligning it with 64K
* (DEFAULT_PE_ALIGN_OLD 128 sectors). Given a 4KB mda_start,
* and 192KB pe_start, the mda_size between the two was 188KB.
* This metadata area size was too small to be a good default,
* and disabling default_data_alignment, with no other change,
* does not imply that the default mda_size or pe_start should
* change.
*/
int get_default_pvmetadatasize_sectors(void)
Change default alignment of pe_start to 1MB. The new standard in the storage industry is to default alignment of data areas to 1MB. fdisk, parted, and mdadm have all been updated to this default. Update LVM to align the PV's data area start (pe_start) to 1MB. This provides a more useful default than the previous default of 64K (which generally ended up being a 192K pe_start once the first metadata area was created). Before this patch: # pvs -o name,vg_mda_size,pe_start PV VMdaSize 1st PE /dev/sdd 188.00k 192.00k After this patch: # pvs -o name,vg_mda_size,pe_start PV VMdaSize 1st PE /dev/sdd 1020.00k 1.00m The heuristic for setting the default alignment for LVM data areas is: - If the default value (1MB) is a multiple of the detected alignment then just use the default. - Otherwise, use the detected value. In practice this means we'll almost always use 1MB -- that is unless: - the alignment was explicitly specified with --dataalignment - or MD's full stripe width, or the {minimum,optimal}_io_size exceeds 1MB - or the specified/detected value is not a power-of-2
2010-08-12 08:11:48 +04:00
{
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
int pagesize = lvm_getpagesize();
/*
* This returns the default size of the metadata area in units of
* 512 byte sectors.
*
* We want the default pe_start to consistently be 1 MiB (1024 KiB),
* (even if default_data_alignment is disabled.)
*
* The mda start is at pagesize offset from the start of the device.
*
* The metadata size is the space between mda start and pe_start.
*
* So, if set set default metadata size to 1024 KiB - <pagesize> KiB,
* it will consistently produce pe_start of 1 MiB.
*
* pe_start 1024 KiB = 2048 sectors.
*
* pagesizes:
* 4096 = 8 sectors.
* 8192 = 16 sectors.
* 65536 = 128 sectors.
*/
switch (pagesize) {
case 4096:
return 2040;
case 8192:
return 2032;
case 65536:
return 1920;
}
log_warn("Using metadata size 960 KiB for non-standard page size %d.", pagesize);
return 1920;
Change default alignment of pe_start to 1MB. The new standard in the storage industry is to default alignment of data areas to 1MB. fdisk, parted, and mdadm have all been updated to this default. Update LVM to align the PV's data area start (pe_start) to 1MB. This provides a more useful default than the previous default of 64K (which generally ended up being a 192K pe_start once the first metadata area was created). Before this patch: # pvs -o name,vg_mda_size,pe_start PV VMdaSize 1st PE /dev/sdd 188.00k 192.00k After this patch: # pvs -o name,vg_mda_size,pe_start PV VMdaSize 1st PE /dev/sdd 1020.00k 1.00m The heuristic for setting the default alignment for LVM data areas is: - If the default value (1MB) is a multiple of the detected alignment then just use the default. - Otherwise, use the detected value. In practice this means we'll almost always use 1MB -- that is unless: - the alignment was explicitly specified with --dataalignment - or MD's full stripe width, or the {minimum,optimal}_io_size exceeds 1MB - or the specified/detected value is not a power-of-2
2010-08-12 08:11:48 +04:00
}
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
#define ONE_MB_IN_SECTORS 2048 /* 2048 * 512 = 1048576 */
void set_pe_align(struct physical_volume *pv, uint64_t data_alignment_sectors)
wrap PE_ALIGN
2006-08-17 23:30:59 +04:00
{
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
uint64_t default_data_alignment_mb;
uint64_t pe_align_sectors;
uint64_t temp_pe_align_sectors;
uint32_t page_size_sectors;
Change default alignment of pe_start to 1MB. The new standard in the storage industry is to default alignment of data areas to 1MB. fdisk, parted, and mdadm have all been updated to this default. Update LVM to align the PV's data area start (pe_start) to 1MB. This provides a more useful default than the previous default of 64K (which generally ended up being a 192K pe_start once the first metadata area was created). Before this patch: # pvs -o name,vg_mda_size,pe_start PV VMdaSize 1st PE /dev/sdd 188.00k 192.00k After this patch: # pvs -o name,vg_mda_size,pe_start PV VMdaSize 1st PE /dev/sdd 1020.00k 1.00m The heuristic for setting the default alignment for LVM data areas is: - If the default value (1MB) is a multiple of the detected alignment then just use the default. - Otherwise, use the detected value. In practice this means we'll almost always use 1MB -- that is unless: - the alignment was explicitly specified with --dataalignment - or MD's full stripe width, or the {minimum,optimal}_io_size exceeds 1MB - or the specified/detected value is not a power-of-2
2010-08-12 08:11:48 +04:00
adjust pe_align for md chunk size
2008-09-19 09:19:09 +04:00
if (pv->pe_align)
goto out;
Pass struct physical_volume to pe_align.
2008-09-19 08:28:58 +04:00
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
if (data_alignment_sectors) {
/* Always use specified alignment */
log_debug("Requested PE alignment is %llu sectors", (unsigned long long)data_alignment_sectors);
pe_align_sectors = data_alignment_sectors;
pv->pe_align = data_alignment_sectors;
Update heuristic used for default and detected data alignment. Add "devices/default_data_alignment" to lvm.conf to control the internal default that LVM2 uses: 0==64k, 1==1MB, 2==2MB, etc. If --dataalignment (or lvm.conf's "devices/data_alignment") is specified then it is always used to align the start of the data area. This means the md_chunk_alignment and data_alignment_detection are disabled if set. (Same now applies to pvcreate --dataalignmentoffset, the specified value will be used instead of the result from data_alignment_offset_detection) set_pe_align() still looks to use the determined default alignment (based on lvm.conf's default_data_alignment) if the default is a multiple of the MD or topology detected values.
2010-08-21 00:59:05 +04:00
goto out;
}
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
/*
* By default the first PE is placed at 1 MiB.
*
* If default_data_alignment is 2, then the first PE
* is placed at 2 * 1 MiB.
*
* If default_data_alignment is 3, then the first PE
* is placed at 3 * 1 MiB.
*/
Update heuristic used for default and detected data alignment. Add "devices/default_data_alignment" to lvm.conf to control the internal default that LVM2 uses: 0==64k, 1==1MB, 2==2MB, etc. If --dataalignment (or lvm.conf's "devices/data_alignment") is specified then it is always used to align the start of the data area. This means the md_chunk_alignment and data_alignment_detection are disabled if set. (Same now applies to pvcreate --dataalignmentoffset, the specified value will be used instead of the result from data_alignment_offset_detection) set_pe_align() still looks to use the determined default alignment (based on lvm.conf's default_data_alignment) if the default is a multiple of the MD or topology detected values.
2010-08-21 00:59:05 +04:00
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
default_data_alignment_mb = find_config_tree_int(pv->fmt->cmd, devices_default_data_alignment_CFG, NULL);
if (default_data_alignment_mb)
pe_align_sectors = default_data_alignment_mb * FIRST_PE_AT_ONE_MB_IN_SECTORS;
Cleanup comment and some whitespace.
2009-10-06 20:00:38 +04:00
else
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
pe_align_sectors = FIRST_PE_AT_ONE_MB_IN_SECTORS;
pv->pe_align = pe_align_sectors;
log_debug("Standard PE alignment is %llu sectors", (unsigned long long)pe_align_sectors);
Update heuristic used for default and detected data alignment. Add "devices/default_data_alignment" to lvm.conf to control the internal default that LVM2 uses: 0==64k, 1==1MB, 2==2MB, etc. If --dataalignment (or lvm.conf's "devices/data_alignment") is specified then it is always used to align the start of the data area. This means the md_chunk_alignment and data_alignment_detection are disabled if set. (Same now applies to pvcreate --dataalignmentoffset, the specified value will be used instead of the result from data_alignment_offset_detection) set_pe_align() still looks to use the determined default alignment (based on lvm.conf's default_data_alignment) if the default is a multiple of the MD or topology detected values.
2010-08-21 00:59:05 +04:00
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
page_size_sectors = lvm_getpagesize() >> SECTOR_SHIFT;
if (page_size_sectors > pe_align_sectors) {
/* This shouldn't happen */
log_debug("Increasing PE alignment to page size %u sectors", page_size_sectors);
pe_align_sectors = page_size_sectors;
pv->pe_align = page_size_sectors;
}
adjust pe_align for md chunk size
2008-09-19 09:19:09 +04:00
Fix conversion of md chunk size into sectors.
2008-10-03 18:22:18 +04:00
if (!pv->dev)
goto out;
Add --dataalignment to pvcreate to specify alignment of data area. (mbroz) This patch is not fully tested and leaves some related bugs unfixed. Intended behaviour of the code now: pe_start in the lvm2 format PV label header is set only by pvcreate (or vgconvert -M2) and then preserved in *all* operations thereafter. In some specialist cases, after the PV is added to a VG, the pe_start field in the VG metadata may hold a different value and if so, it overrides the other one for as long as the PV is in such a VG. Currently, the field storing the size of the data area in the PV label header always holds 0. As it only has meaning in the context of a volume group, it is calculated whenever the PV is added to a VG (and can be derived from extent_size and pe_count in the VG metadata).
2009-02-22 22:00:26 +03:00
/*
Use the MD device's stripe-width, instead of chunk_size, to align the data blocks of a Physical Volume that is placed directly upon an MD device.
2009-07-06 23:04:24 +04:00
* Align to stripe-width of underlying md device if present
Add --dataalignment to pvcreate to specify alignment of data area. (mbroz) This patch is not fully tested and leaves some related bugs unfixed. Intended behaviour of the code now: pe_start in the lvm2 format PV label header is set only by pvcreate (or vgconvert -M2) and then preserved in *all* operations thereafter. In some specialist cases, after the PV is added to a VG, the pe_start field in the VG metadata may hold a different value and if so, it overrides the other one for as long as the PV is in such a VG. Currently, the field storing the size of the data area in the PV label header always holds 0. As it only has meaning in the context of a volume group, it is calculated whenever the PV is added to a VG (and can be derived from extent_size and pe_count in the VG metadata).
2009-02-22 22:00:26 +03:00
*/
config: add profile arg to find_config_tree_bool
2013-06-25 14:31:53 +04:00
if (find_config_tree_bool(pv->fmt->cmd, devices_md_chunk_alignment_CFG, NULL)) {
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
temp_pe_align_sectors = dev_md_stripe_width(pv->fmt->cmd->dev_types, pv->dev);
if (temp_pe_align_sectors && (pe_align_sectors % temp_pe_align_sectors)) {
log_debug("Adjusting PE alignment from %llu sectors to md stripe width %llu sectors for %s",
(unsigned long long)pe_align_sectors,
(unsigned long long)temp_pe_align_sectors,
dev_name(pv->dev));
pe_align_sectors = temp_pe_align_sectors;
pv->pe_align = temp_pe_align_sectors;
}
Change default alignment of pe_start to 1MB. The new standard in the storage industry is to default alignment of data areas to 1MB. fdisk, parted, and mdadm have all been updated to this default. Update LVM to align the PV's data area start (pe_start) to 1MB. This provides a more useful default than the previous default of 64K (which generally ended up being a 192K pe_start once the first metadata area was created). Before this patch: # pvs -o name,vg_mda_size,pe_start PV VMdaSize 1st PE /dev/sdd 188.00k 192.00k After this patch: # pvs -o name,vg_mda_size,pe_start PV VMdaSize 1st PE /dev/sdd 1020.00k 1.00m The heuristic for setting the default alignment for LVM data areas is: - If the default value (1MB) is a multiple of the detected alignment then just use the default. - Otherwise, use the detected value. In practice this means we'll almost always use 1MB -- that is unless: - the alignment was explicitly specified with --dataalignment - or MD's full stripe width, or the {minimum,optimal}_io_size exceeds 1MB - or the specified/detected value is not a power-of-2
2010-08-12 08:11:48 +04:00
}
adjust pe_align for md chunk size
2008-09-19 09:19:09 +04:00
Add devices/data_alignment_detection to lvm.conf. Adds 'data_alignment_detection' config option to the devices section of lvm.conf. If your kernel provides topology information in sysfs (linux >= 2.6.31) for the Physical Volume, the start of data area will be aligned on a multiple of the ’minimum_io_size’ or ’optimal_io_size’ exposed in sysfs. minimum_io_size is used if optimal_io_size is undefined (0). If both md_chunk_alignment and data_alignment_detection are enabled the result of data_alignment_detection is used. Signed-off-by: Mike Snitzer <snitzer@redhat.com>
2009-08-01 21:08:43 +04:00
/*
* Align to topology's minimum_io_size or optimal_io_size if present
* - minimum_io_size - the smallest request the device can perform
* w/o incurring a read-modify-write penalty (e.g. MD's chunk size)
* - optimal_io_size - the device's preferred unit of receiving I/O
* (e.g. MD's stripe width)
*/
config: add profile arg to find_config_tree_bool
2013-06-25 14:31:53 +04:00
if (find_config_tree_bool(pv->fmt->cmd, devices_data_alignment_detection_CFG, NULL)) {
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
temp_pe_align_sectors = dev_minimum_io_size(pv->fmt->cmd->dev_types, pv->dev);
if (temp_pe_align_sectors && (pe_align_sectors % temp_pe_align_sectors)) {
log_debug("Adjusting PE alignment from %llu sectors to mininum io size %llu sectors for %s",
(unsigned long long)pe_align_sectors,
(unsigned long long)temp_pe_align_sectors,
dev_name(pv->dev));
pe_align_sectors = temp_pe_align_sectors;
pv->pe_align = temp_pe_align_sectors;
}
temp_pe_align_sectors = dev_optimal_io_size(pv->fmt->cmd->dev_types, pv->dev);
Add devices/data_alignment_detection to lvm.conf. Adds 'data_alignment_detection' config option to the devices section of lvm.conf. If your kernel provides topology information in sysfs (linux >= 2.6.31) for the Physical Volume, the start of data area will be aligned on a multiple of the ’minimum_io_size’ or ’optimal_io_size’ exposed in sysfs. minimum_io_size is used if optimal_io_size is undefined (0). If both md_chunk_alignment and data_alignment_detection are enabled the result of data_alignment_detection is used. Signed-off-by: Mike Snitzer <snitzer@redhat.com>
2009-08-01 21:08:43 +04:00
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
if (temp_pe_align_sectors && (pe_align_sectors % temp_pe_align_sectors)) {
log_debug("Adjusting PE alignment from %llu sectors to optimal io size %llu sectors for %s",
(unsigned long long)pe_align_sectors,
(unsigned long long)temp_pe_align_sectors,
dev_name(pv->dev));
pe_align_sectors = temp_pe_align_sectors;
pv->pe_align = temp_pe_align_sectors;
}
Add devices/data_alignment_detection to lvm.conf. Adds 'data_alignment_detection' config option to the devices section of lvm.conf. If your kernel provides topology information in sysfs (linux >= 2.6.31) for the Physical Volume, the start of data area will be aligned on a multiple of the ’minimum_io_size’ or ’optimal_io_size’ exposed in sysfs. minimum_io_size is used if optimal_io_size is undefined (0). If both md_chunk_alignment and data_alignment_detection are enabled the result of data_alignment_detection is used. Signed-off-by: Mike Snitzer <snitzer@redhat.com>
2009-08-01 21:08:43 +04:00
}
Update heuristic used for default and detected data alignment. Add "devices/default_data_alignment" to lvm.conf to control the internal default that LVM2 uses: 0==64k, 1==1MB, 2==2MB, etc. If --dataalignment (or lvm.conf's "devices/data_alignment") is specified then it is always used to align the start of the data area. This means the md_chunk_alignment and data_alignment_detection are disabled if set. (Same now applies to pvcreate --dataalignmentoffset, the specified value will be used instead of the result from data_alignment_offset_detection) set_pe_align() still looks to use the determined default alignment (based on lvm.conf's default_data_alignment) if the default is a multiple of the MD or topology detected values.
2010-08-21 00:59:05 +04:00
out:
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
log_debug("Setting PE alignment to %llu sectors for %s.",
(unsigned long long)pv->pe_align, dev_name(pv->dev));
wrap PE_ALIGN
2006-08-17 23:30:59 +04:00
}
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
void set_pe_align_offset(struct physical_volume *pv, uint64_t data_alignment_offset_sectors)
Add --dataalignmentoffset to pvcreate to shift start of aligned data area Adds pe_align_offset to 'struct physical_volume'; is initialized with set_pe_align_offset(). After pe_start is established pe_align_offset is added to it. Signed-off-by: Mike Snitzer <snitzer@redhat.com>
2009-07-30 21:45:28 +04:00
{
if (pv->pe_align_offset)
goto out;
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
if (data_alignment_offset_sectors) {
Update heuristic used for default and detected data alignment. Add "devices/default_data_alignment" to lvm.conf to control the internal default that LVM2 uses: 0==64k, 1==1MB, 2==2MB, etc. If --dataalignment (or lvm.conf's "devices/data_alignment") is specified then it is always used to align the start of the data area. This means the md_chunk_alignment and data_alignment_detection are disabled if set. (Same now applies to pvcreate --dataalignmentoffset, the specified value will be used instead of the result from data_alignment_offset_detection) set_pe_align() still looks to use the determined default alignment (based on lvm.conf's default_data_alignment) if the default is a multiple of the MD or topology detected values.
2010-08-21 00:59:05 +04:00
/* Always use specified data_alignment_offset */
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
pv->pe_align_offset = data_alignment_offset_sectors;
Update heuristic used for default and detected data alignment. Add "devices/default_data_alignment" to lvm.conf to control the internal default that LVM2 uses: 0==64k, 1==1MB, 2==2MB, etc. If --dataalignment (or lvm.conf's "devices/data_alignment") is specified then it is always used to align the start of the data area. This means the md_chunk_alignment and data_alignment_detection are disabled if set. (Same now applies to pvcreate --dataalignmentoffset, the specified value will be used instead of the result from data_alignment_offset_detection) set_pe_align() still looks to use the determined default alignment (based on lvm.conf's default_data_alignment) if the default is a multiple of the MD or topology detected values.
2010-08-21 00:59:05 +04:00
goto out;
}
Add --dataalignmentoffset to pvcreate to shift start of aligned data area Adds pe_align_offset to 'struct physical_volume'; is initialized with set_pe_align_offset(). After pe_start is established pe_align_offset is added to it. Signed-off-by: Mike Snitzer <snitzer@redhat.com>
2009-07-30 21:45:28 +04:00
if (!pv->dev)
goto out;
config: add profile arg to find_config_tree_bool
2013-06-25 14:31:53 +04:00
if (find_config_tree_bool(pv->fmt->cmd, devices_data_alignment_offset_detection_CFG, NULL)) {
refactor: make device type recognition code common for general use Changes: - move device type registration out of "type filter" (filter.c) to a separate and new dev-type.[ch] for common use throughout the code - the structure for keeping the major numbers detected for available device types and available partitioning available is stored in "dev_types" structure now - move common partitioning detection code to dev-type.[ch] as well together with other device-related functions bound to dev_types (see dev-type.h for the interface) The dev-type interface contains all common functions used to detect subsystems/device types, signature/superblock recognition code, type-specific device properties and other common device properties (bound to dev_types), including partitioning support. - add dev_types instance to cmd context as cmd->dev_types for common use - use cmd->dev_types throughout as a central point for providing information about device types
2013-06-12 14:08:56 +04:00
int align_offset = dev_alignment_offset(pv->fmt->cmd->dev_types, pv->dev);
Handle a misaligned device that reports a -1 alignment_offset. The kernel's blk_stack_limits() function may flag a device as 'misaligned'. If it does the alignment_offset will be -1. Update set_pe_align_offset() to accommodate this corner case.
2010-03-03 00:56:14 +03:00
/* must handle a -1 alignment_offset; means dev is misaligned */
if (align_offset < 0)
align_offset = 0;
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
pv->pe_align_offset = align_offset;
Handle a misaligned device that reports a -1 alignment_offset. The kernel's blk_stack_limits() function may flag a device as 'misaligned'. If it does the alignment_offset will be -1. Update set_pe_align_offset() to accommodate this corner case.
2010-03-03 00:56:14 +03:00
}
Add devices/data_alignment_offset_detection to lvm.conf. If the pvcreate --dataalignmentoffset option is not specified the start of a PV's aligned data area will be shifted by the associated 'alignment_offset' exposed in sysfs (unless devices/data_alignment_offset_detection is disabled in lvm.conf). Signed-off-by: Mike Snitzer <snitzer@redhat.com>
2009-08-01 21:07:36 +04:00
Update heuristic used for default and detected data alignment. Add "devices/default_data_alignment" to lvm.conf to control the internal default that LVM2 uses: 0==64k, 1==1MB, 2==2MB, etc. If --dataalignment (or lvm.conf's "devices/data_alignment") is specified then it is always used to align the start of the data area. This means the md_chunk_alignment and data_alignment_detection are disabled if set. (Same now applies to pvcreate --dataalignmentoffset, the specified value will be used instead of the result from data_alignment_offset_detection) set_pe_align() still looks to use the determined default alignment (based on lvm.conf's default_data_alignment) if the default is a multiple of the MD or topology detected values.
2010-08-21 00:59:05 +04:00
out:
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
log_debug("Setting PE alignment offset to %llu sectors for %s.",
(unsigned long long)pv->pe_align_offset, dev_name(pv->dev));
Add --dataalignmentoffset to pvcreate to shift start of aligned data area Adds pe_align_offset to 'struct physical_volume'; is initialized with set_pe_align_offset(). After pe_start is established pe_align_offset is added to it. Signed-off-by: Mike Snitzer <snitzer@redhat.com>
2009-07-30 21:45:28 +04:00
}
Add add_pvl_to_vgs() - helper function to add a pv to a vg list. Small refactor of main places in the code where a pv is added to a vg into a small function which adds the pv to the list and updates the vg counts. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-06 18:04:54 +04:00
void add_pvl_to_vgs(struct volume_group *vg, struct pv_list *pvl)
{
dm_list_add(&vg->pvs, &pvl->list);
vg->pv_count++;
Add pv->vg to solidify link between a pv and a vg. lvm2app needs a link back to the vg in order to use the vg handle for memory allocations as well as other things. This patch adds the field to struct physical_volume, and sets pv->vg when reading a vg from disk or extending a vg by using the helper function previously added, add_pvl_to_vgs(). Moves and renames are handled with separate code inside move_pv() and vgmerge(). Add pv->vg check to vg_validate(). A NULL value in pv->vg signifies membership in the orphan VG. Note though in the case of pv_read() on a device with metadatacopies == 0, more devices may need to be read for an authoritative answer. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-13 21:26:36 +04:00
pvl->pv->vg = vg;
Use only vg_set_fid and new pv_set_fid fn to assign the format instance. This is essential for proper format instance ref_count support. We must use these functions to set the fid everywhere from now on, even the NULL value!
2011-03-11 17:50:13 +03:00
pv_set_fid(pvl->pv, vg->fid);
Add add_pvl_to_vgs() - helper function to add a pv to a vg list. Small refactor of main places in the code where a pv is added to a vg into a small function which adds the pv to the list and updates the vg counts. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-06 18:04:54 +04:00
}
Add del_pvl_from_vgs() and move prototypes into metadata-exported.h Add a delete function to manage the vg->pvs list. NOTE: It may be possible to do further cleanup to these add/del functions by passing a 'pv' as input instead of 'pv_list'. The pv_list is used for functions which do allocations (lvcreate) while other places in the code just manage a list of 'pv' (e.g. import functions, vgextend, etc). Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-13 21:25:44 +04:00
void del_pvl_from_vgs(struct volume_group *vg, struct pv_list *pvl)
{
Keep a global (per-format) orphan_vg and keep any and all orphan PVs linked to it. Avoids the need for FMT_INSTANCE_PV and enables further simplifications. No functional change, internal refactor only.
2012-02-10 06:53:03 +04:00
struct lvmcache_info *info;
Use only vg_set_fid and new pv_set_fid fn to assign the format instance. This is essential for proper format instance ref_count support. We must use these functions to set the fid everywhere from now on, even the NULL value!
2011-03-11 17:50:13 +03:00
Add del_pvl_from_vgs() and move prototypes into metadata-exported.h Add a delete function to manage the vg->pvs list. NOTE: It may be possible to do further cleanup to these add/del functions by passing a 'pv' as input instead of 'pv_list'. The pv_list is used for functions which do allocations (lvcreate) while other places in the code just manage a list of 'pv' (e.g. import functions, vgextend, etc). Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-13 21:25:44 +04:00
vg->pv_count--;
dm_list_del(&pvl->list);
Use only vg_set_fid and new pv_set_fid fn to assign the format instance. This is essential for proper format instance ref_count support. We must use these functions to set the fid everywhere from now on, even the NULL value!
2011-03-11 17:50:13 +03:00
Keep a global (per-format) orphan_vg and keep any and all orphan PVs linked to it. Avoids the need for FMT_INSTANCE_PV and enables further simplifications. No functional change, internal refactor only.
2012-02-10 06:53:03 +04:00
pvl->pv->vg = vg->fid->fmt->orphan_vg; /* orphan */
lvmcache: add optional dev arg to lvmcache_info_from_pvid A number of places are working on a specific dev when they call lvmcache_info_from_pvid() to look up an info struct based on a pvid. In those cases, pass the dev being used to lvmcache_info_from_pvid(). When a dev is specified, lvmcache_info_from_pvid() will verify that the cached info it's using matches the dev being processed before returning the info. Calling code will not mistakenly get info for the wrong dev when duplicate devs exist. This confusion was happening when scanning labels when duplicate devs existed. label_read for the first dev would add an info struct to lvmcache for that dev/pvid. label_read for the second dev would see the pvid in lvmcache from first dev, and mistakenly conclude that the label_read from the second dev can be skipped because it's already been done. By verifying that the dev for the cached pvid matches the dev being read, this mismatch is avoided and the label is actually read from the second duplicate.
2016-06-06 22:04:17 +03:00
if ((info = lvmcache_info_from_pvid((const char *) &pvl->pv->id, pvl->pv->dev, 0)))
Keep a global (per-format) orphan_vg and keep any and all orphan PVs linked to it. Avoids the need for FMT_INSTANCE_PV and enables further simplifications. No functional change, internal refactor only.
2012-02-10 06:53:03 +04:00
lvmcache_fid_add_mdas(info, vg->fid->fmt->orphan_vg->fid,
(const char *) &pvl->pv->id, ID_LEN);
pv_set_fid(pvl->pv, vg->fid->fmt->orphan_vg->fid);
Add del_pvl_from_vgs() and move prototypes into metadata-exported.h Add a delete function to manage the vg->pvs list. NOTE: It may be possible to do further cleanup to these add/del functions by passing a 'pv' as input instead of 'pv_list'. The pv_list is used for functions which do allocations (lvcreate) while other places in the code just manage a list of 'pv' (e.g. import functions, vgextend, etc). Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-13 21:25:44 +04:00
}
Turn _add_pv_to_vg() into external library function add_pv_to_vg()
2007-07-12 09:04:42 +04:00
/**
* add_pv_to_vg - Add a physical volume to a volume group
* @vg - volume group to add to
* @pv_name - name of the pv (to be removed)
* @pv - physical volume to add to volume group
*
* Returns:
* 0 - failure
* 1 - success
* FIXME: remove pv_name - obtain safely from pv
*/
liblvm: Move lib code used exclusively into metadata-liblvm.c Also remove some redundant function definitions from metadata.h.
2017-10-18 21:29:32 +03:00
int add_pv_to_vg(struct volume_group *vg, const char *pv_name,
struct physical_volume *pv, int new_pv)
o vgcreate
2001-10-12 18:25:53 +04:00
{
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
struct pv_list *pvl;
Convert _add_pv_to_vg to usable LVM library function.
2007-07-03 01:48:30 +04:00
struct format_instance *fid = vg->fid;
Properly release VG memory pool in metadata manipulation code.
2009-04-10 14:01:08 +04:00
struct dm_pool *mem = vg->vgmem;
Use __attribute__ consistently throughout.
2010-07-09 19:34:40 +04:00
char uuid[64] __attribute__((aligned(8)));
pv: check for the PV_EXT_USED flag and deny pvcreate/pvchange/pvremove/vgcreate on such PV (unless forced) Make sure we won't use a PV that is already marked as used. Normally, VG metadata would stop us from doing that, but we can run into a situation where such metadata is missing because PVs with MDAs are missing and the PVs left are the ones with 0 MDAs. (/dev/sda in this example has 0 MDAs and it belongs to a VG, but other PVs with MDA are missing) $ pvs -o pv_name,pv_mda_count /dev/sda PV #PMda /dev/sda 0 $ pvcreate /dev/sda PV '/dev/sda' is marked as belonging to a VG but its metadata is missing. Can't initialize PV '/dev/sda' without -ff. $ pvchange -u /dev/sda PV '/dev/sda' is marked as belonging to a VG but its metadata is missing. Can't change PV '/dev/sda' without -ff. Physical volume /dev/sda not changed 0 physical volumes changed / 1 physical volume not changed $ pvremove /dev/sda PV '/dev/sda' is marked as belonging to a VG but its metadata is missing. (If you are certain you need pvremove, then confirm by using --force twice.) $ vgcreate vg /dev/sda Physical volume '/dev/sda' is marked as belonging to a VG but its metadata is missing. Unable to add physical volume '/dev/sda' to volume group 'vg'.
2015-03-10 13:25:14 +03:00
int used;
o vgcreate
2001-10-12 18:25:53 +04:00
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
log_verbose("Adding physical volume '%s' to volume group '%s'",
o Added lvextend o Full signed arguments to lvreduce/lvextend o Consistent lv_number/pe map use o Populate pv->pe_allocated o Fixes for allocation/writing of multiple LVs
2001-11-10 01:01:04 +03:00
pv_name, vg->name);
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
Use hash, bitset, malloc, pool from libdevmapper.
2005-10-17 03:03:59 +04:00
if (!(pvl = dm_pool_zalloc(mem, sizeof(*pvl)))) {
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
log_error("pv_list allocation for '%s' failed", pv_name);
o vgcreate
2001-10-12 18:25:53 +04:00
return 0;
}
Add is_orphan_vg() and change all hardcoded checks to use it.
2007-11-02 16:06:42 +03:00
if (!is_orphan_vg(pv->vg_name)) {
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
log_error("Physical volume '%s' is already in volume group "
"'%s'", pv_name, pv->vg_name);
o lvm readline error-case tidy-up o more vgcreate error cases
2001-10-15 16:49:58 +04:00
return 0;
tidy: prefer not using else after return clang-tidy: avoid using 'else' after return - give more readable code, and also saves indention level.
2017-07-19 17:16:12 +03:00
}
if (!new_pv) {
pv: check for the PV_EXT_USED flag and deny pvcreate/pvchange/pvremove/vgcreate on such PV (unless forced) Make sure we won't use a PV that is already marked as used. Normally, VG metadata would stop us from doing that, but we can run into a situation where such metadata is missing because PVs with MDAs are missing and the PVs left are the ones with 0 MDAs. (/dev/sda in this example has 0 MDAs and it belongs to a VG, but other PVs with MDA are missing) $ pvs -o pv_name,pv_mda_count /dev/sda PV #PMda /dev/sda 0 $ pvcreate /dev/sda PV '/dev/sda' is marked as belonging to a VG but its metadata is missing. Can't initialize PV '/dev/sda' without -ff. $ pvchange -u /dev/sda PV '/dev/sda' is marked as belonging to a VG but its metadata is missing. Can't change PV '/dev/sda' without -ff. Physical volume /dev/sda not changed 0 physical volumes changed / 1 physical volume not changed $ pvremove /dev/sda PV '/dev/sda' is marked as belonging to a VG but its metadata is missing. (If you are certain you need pvremove, then confirm by using --force twice.) $ vgcreate vg /dev/sda Physical volume '/dev/sda' is marked as belonging to a VG but its metadata is missing. Unable to add physical volume '/dev/sda' to volume group 'vg'.
2015-03-10 13:25:14 +03:00
if ((used = is_used_pv(pv)) < 0)
return_0;
if (used) {
Use a common message for a used PV Change some inconsistent messages and adopt the new wording "PV %s is used by" in place of "PV %s is marked as belonging to" or "PV %s belongs to".
2016-02-25 23:12:08 +03:00
log_error("PV %s is used by a VG but its metadata is missing.", pv_name);
pv: check for the PV_EXT_USED flag and deny pvcreate/pvchange/pvremove/vgcreate on such PV (unless forced) Make sure we won't use a PV that is already marked as used. Normally, VG metadata would stop us from doing that, but we can run into a situation where such metadata is missing because PVs with MDAs are missing and the PVs left are the ones with 0 MDAs. (/dev/sda in this example has 0 MDAs and it belongs to a VG, but other PVs with MDA are missing) $ pvs -o pv_name,pv_mda_count /dev/sda PV #PMda /dev/sda 0 $ pvcreate /dev/sda PV '/dev/sda' is marked as belonging to a VG but its metadata is missing. Can't initialize PV '/dev/sda' without -ff. $ pvchange -u /dev/sda PV '/dev/sda' is marked as belonging to a VG but its metadata is missing. Can't change PV '/dev/sda' without -ff. Physical volume /dev/sda not changed 0 physical volumes changed / 1 physical volume not changed $ pvremove /dev/sda PV '/dev/sda' is marked as belonging to a VG but its metadata is missing. (If you are certain you need pvremove, then confirm by using --force twice.) $ vgcreate vg /dev/sda Physical volume '/dev/sda' is marked as belonging to a VG but its metadata is missing. Unable to add physical volume '/dev/sda' to volume group 'vg'.
2015-03-10 13:25:14 +03:00
return 0;
}
o lvm readline error-case tidy-up o more vgcreate error cases
2001-10-15 16:49:58 +04:00
}
o vgcreate
2001-10-12 18:25:53 +04:00
Some new features.
2002-11-18 17:04:08 +03:00
if (pv->fmt != fid->fmt) {
log_error("Physical volume %s is of different format type (%s)",
pv_name, pv->fmt->name);
return 0;
}
Use dm_is_dm_major instead of local copy. Allow mapped devices to be used as PVs safely.
2005-10-25 23:08:21 +04:00
/* Ensure PV doesn't depend on another PV already in the VG */
Add mirror_library description to example.conf. More compile-time cleanup.
2006-05-11 21:58:58 +04:00
if (pv_uses_vg(pv, vg)) {
Use dm_is_dm_major instead of local copy. Allow mapped devices to be used as PVs safely.
2005-10-25 23:08:21 +04:00
log_error("Physical volume %s might be constructed from same "
"volume group %s", pv_name, vg->name);
return 0;
}
Use hash, bitset, malloc, pool from libdevmapper.
2005-10-17 03:03:59 +04:00
if (!(pv->vg_name = dm_pool_strdup(mem, vg->name))) {
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
log_error("vg->name allocation failed for '%s'", pv_name);
o vgcreate
2001-10-12 18:25:53 +04:00
return 0;
}
Fix vgexport/vgimport to set/reset PV exported flag so pv_attr is correct. Add vgid to struct physical_volume and pass with vg_name to some functions.
2006-04-13 01:23:04 +04:00
memcpy(&pv->vgid, &vg->id, sizeof(vg->id));
vgcreate basic extent size validation
2001-10-16 00:29:15 +04:00
/* Units of 512-byte sectors */
o vgcreate
2001-10-12 18:25:53 +04:00
pv->pe_size = vg->extent_size;
/*
rely on pv_setup functions to fix up pe_count
2006-10-08 03:06:18 +04:00
* pe_count must always be calculated by pv_setup
o vgcreate
2001-10-12 18:25:53 +04:00
*/
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
pv->pe_alloc_count = 0;
o vgcreate
2001-10-12 18:25:53 +04:00
vgcfgrestore: Retain allocatable PV attribute. pvchange -xn was getting lost. All PVs were set to allocatable again after restore. Moved setting ALLOCATABLE_PV outside pv_setup().
2016-01-14 03:46:45 +03:00
/* LVM1 stores this outside a VG; LVM2 only stores it inside */
/* FIXME Default from config file? vgextend cmdline flag? */
pv->status |= ALLOCATABLE_PV;
Change the code throughout to use new pv_initialise and modified pv_setup fn. Change pv_create code to work with these changes together with using new pv_add_metadata_area fn to add metadata areas for a PV being created.
2011-02-21 15:24:15 +03:00
if (!fid->fmt->ops->pv_setup(fid->fmt, pv, vg)) {
Tweak some error message levels.
2002-01-28 00:30:47 +03:00
log_error("Format-specific setup of physical volume '%s' "
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
"failed.", pv_name);
return 0;
}
cleanup: remove superfluous wrappers _find_pv_by_name -> find_pv_by_name _find_pv_in_vg -> find_pv_in_vg _find_pv_in_vg_by_uuid -> find_pv_in_vg_by_uuid The only callers of the underscored variants were their wrappers without the underscore. No other part of the code referenced the underscored variants.
2013-03-19 16:58:02 +04:00
if (find_pv_in_vg(vg, pv_name) ||
find_pv_in_vg_by_uuid(vg, &pv->id)) {
Check for duplicate paths (pvids) on the commandline of vgcreate. A user specifying duplicate paths on the cmdline of vgcreate will get a message similar to the following: vgcreate vgtest2 /dev/loop3 /dev/loop5 Found duplicate PV jk1lXsKzwyOKlXq6bhaFFKMQQ06oPgu8: using /dev/loop5 not /dev/loop3 Found duplicate PV jk1lXsKzwyOKlXq6bhaFFKMQQ06oPgu8: using /dev/loop3 not /dev/loop5 Internal error: Duplicate PV id jk1lXs-Kzwy-OKlX-q6bh-aFFK-MQQ0-6oPgu8 detected for /dev/loop3 in vgtest2. This is caught by vg_validate(), but it would be good to find this condition earlier in the vgcreate code. add_pv_to_vg() currently checks by pvname, but does not look for duplcate pvids. This patch adds the check for duplicate pvids and results in new error output as follows: vgcreate vgtest2 /dev/loop3 /dev/loop5 Found duplicate PV jk1lXsKzwyOKlXq6bhaFFKMQQ06oPgu8: using /dev/loop5 not /dev/loop3 Found duplicate PV jk1lXsKzwyOKlXq6bhaFFKMQQ06oPgu8: using /dev/loop3 not /dev/loop5 Physical volume '/dev/loop5 (jk1lXs-Kzwy-OKlX-q6bh-aFFK-MQQ0-6oPgu8)' listed more than once. Unable to add physical volume '/dev/loop5' to volume group 'vgtest2'. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-08 19:18:35 +04:00
if (!id_write_format(&pv->id, uuid, sizeof(uuid))) {
stack;
uuid[0] = '\0';
}
The lvmetad client-side integration. Only active when use_lvmetad = 1 is set in lvm.conf *and* lvmetad is running.
2012-02-23 17:11:07 +04:00
log_error("Physical volume '%s (%s)' already in the VG.",
Check for duplicate paths (pvids) on the commandline of vgcreate. A user specifying duplicate paths on the cmdline of vgcreate will get a message similar to the following: vgcreate vgtest2 /dev/loop3 /dev/loop5 Found duplicate PV jk1lXsKzwyOKlXq6bhaFFKMQQ06oPgu8: using /dev/loop5 not /dev/loop3 Found duplicate PV jk1lXsKzwyOKlXq6bhaFFKMQQ06oPgu8: using /dev/loop3 not /dev/loop5 Internal error: Duplicate PV id jk1lXs-Kzwy-OKlX-q6bh-aFFK-MQQ0-6oPgu8 detected for /dev/loop3 in vgtest2. This is caught by vg_validate(), but it would be good to find this condition earlier in the vgcreate code. add_pv_to_vg() currently checks by pvname, but does not look for duplcate pvids. This patch adds the check for duplicate pvids and results in new error output as follows: vgcreate vgtest2 /dev/loop3 /dev/loop5 Found duplicate PV jk1lXsKzwyOKlXq6bhaFFKMQQ06oPgu8: using /dev/loop5 not /dev/loop3 Found duplicate PV jk1lXsKzwyOKlXq6bhaFFKMQQ06oPgu8: using /dev/loop3 not /dev/loop5 Physical volume '/dev/loop5 (jk1lXs-Kzwy-OKlX-q6bh-aFFK-MQQ0-6oPgu8)' listed more than once. Unable to add physical volume '/dev/loop5' to volume group 'vgtest2'. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-08 19:18:35 +04:00
pv_name, uuid);
o vgcreate
2001-10-12 18:25:53 +04:00
return 0;
}
Default to unlimited number of LVs/PVs in lvm2 format.
2003-11-06 23:33:34 +03:00
if (vg->pv_count && (vg->pv_count == vg->max_pv)) {
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
log_error("No space for '%s' - volume group '%s' "
"holds max %d physical volume(s).", pv_name,
vg->name, vg->max_pv);
return 0;
}
Use stack return macros throughout.
2008-01-30 16:19:47 +03:00
if (!alloc_pv_segment_whole_pv(mem, pv))
return_0;
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
Avoid lvextend to overflow Add extra check to extent_count overflow. Use internal define MAX_EXTENT_COUNT instead UINT32_MAX.
2011-11-05 02:49:53 +04:00
if ((uint64_t) vg->extent_count + pv->pe_count > MAX_EXTENT_COUNT) {
Add some missing bounds checks on 32 bit extent counters. Add Petabyte and Exabyte support. Fix lvcreate error message when 0 extents requested.
2006-11-10 21:24:11 +03:00
log_error("Unable to add %s to %s: new extent count (%"
PRIu64 ") exceeds limit (%" PRIu32 ").",
pv_name, vg->name,
(uint64_t) vg->extent_count + pv->pe_count,
Avoid lvextend to overflow Add extra check to extent_count overflow. Use internal define MAX_EXTENT_COUNT instead UINT32_MAX.
2011-11-05 02:49:53 +04:00
MAX_EXTENT_COUNT);
Add some missing bounds checks on 32 bit extent counters. Add Petabyte and Exabyte support. Fix lvcreate error message when 0 extents requested.
2006-11-10 21:24:11 +03:00
return 0;
}
Add pv to vg->pvs after check for maximum value of vg->extent_count. In add_pv_to_vg(), we should only add the pv to vg->pvs after all internal checks have passed. The check for vg->extent_count exeeding maximum was after we added the pv to the list, so this function could return a state of vg->pvs that did not reflect other parameters such as vg->pv_count. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-06 18:03:43 +04:00
pvl->pv = pv;
Add add_pvl_to_vgs() - helper function to add a pv to a vg list. Small refactor of main places in the code where a pv is added to a vg into a small function which adds the pv to the list and updates the vg counts. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-06 18:04:54 +04:00
add_pvl_to_vgs(vg, pvl);
lvcreate
2001-11-06 22:02:26 +03:00
vg->extent_count += pv->pe_count;
vg->free_count += pv->pe_count;
o vgcreate
2001-10-12 18:25:53 +04:00
metadata: Fix handling of orphan PV linking & re-linking.
2013-02-19 06:13:59 +04:00
dm_list_iterate_items(pvl, &fid->fmt->orphan_vg->pvs)
if (pv == pvl->pv) { /* unlink from orphan */
dm_list_del(&pvl->list);
break;
}
o vgcreate
2001-10-12 18:25:53 +04:00
return 1;
}
cleanup: _move_pv is static metadata/metadata.c:363:5: warning: no previous prototype for '_move_pv' [-Wmissing-prototypes]
2014-04-28 14:11:44 +04:00
static int _move_pv(struct volume_group *vg_from, struct volume_group *vg_to,
const char *pv_name, int enforce_pv_from_source)
Refactor vgsplit - move move_pvs_used_by_lv and move_pv inside library. In the future we may export these functions or something like them in liblvm For now this helps in cleaning up the checks for RESIZEABLE since we can use the internal library function vg_bad_status_bits. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Acked-by: Alasdair G Kergon <agk@redhat.com>
2009-07-14 06:15:21 +04:00
{
struct physical_volume *pv;
struct pv_list *pvl;
/* FIXME: handle tags */
if (!(pvl = find_pv_in_vg(vg_from, pv_name))) {
vgsplit: Make vgsplit work on mirrors with leg and log on same PV Given a named mirror LV, vgsplit will look for the PVs that compose it and move them to a new VG. It does this by first looking at the log and then the legs. If the log is on the same device as one of the mirror images, a problem occurs. This is because the PV is moved to the new VG as the log is processed and thus cannot be found in the current VG when the image is processed. The solution is to check and see if the PV we are looking for has already been moved to the new VG. If so, it is not an error.
2014-04-25 23:53:34 +04:00
if (!enforce_pv_from_source &&
coverity: fix issues reported by coverity
2014-06-24 16:58:53 +04:00
find_pv_in_vg(vg_to, pv_name))
vgsplit: Make vgsplit work on mirrors with leg and log on same PV Given a named mirror LV, vgsplit will look for the PVs that compose it and move them to a new VG. It does this by first looking at the log and then the legs. If the log is on the same device as one of the mirror images, a problem occurs. This is because the PV is moved to the new VG as the log is processed and thus cannot be found in the current VG when the image is processed. The solution is to check and see if the PV we are looking for has already been moved to the new VG. If so, it is not an error.
2014-04-25 23:53:34 +04:00
/*
* PV has already been moved. This can happen if an
* LV is being moved that has multiple sub-LVs on the
* same PV.
*/
return 1;
Refactor vgsplit - move move_pvs_used_by_lv and move_pv inside library. In the future we may export these functions or something like them in liblvm For now this helps in cleaning up the checks for RESIZEABLE since we can use the internal library function vg_bad_status_bits. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Acked-by: Alasdair G Kergon <agk@redhat.com>
2009-07-14 06:15:21 +04:00
log_error("Physical volume %s not in volume group %s",
pv_name, vg_from->name);
return 0;
}
liblvm: Move lib code used exclusively into metadata-liblvm.c Also remove some redundant function definitions from metadata.h.
2017-10-18 21:29:32 +03:00
if (vg_bad_status_bits(vg_from, RESIZEABLE_VG) ||
vg_bad_status_bits(vg_to, RESIZEABLE_VG))
Remove READ_REQUIRE_RESIZEABLE flag from vgsplit. Remove READ_REQUIRE_RESIZEABLE flag from vgsplit similar to the removal from vgextend. Move the check inside the functions that actually move pvs from one vg structure to another. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Acked-by: Alasdair G Kergon <agk@redhat.com>
2009-07-14 06:16:05 +04:00
return 0;
Call add_pvl_to_vgs() and del_pvl_from_vgs() from more places. Now that we have library functions to add/delete a pv from the vg->pvs list, call them from everywhere. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-13 21:26:03 +04:00
del_pvl_from_vgs(vg_from, pvl);
add_pvl_to_vgs(vg_to, pvl);
Refactor vgsplit - move move_pvs_used_by_lv and move_pv inside library. In the future we may export these functions or something like them in liblvm For now this helps in cleaning up the checks for RESIZEABLE since we can use the internal library function vg_bad_status_bits. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Acked-by: Alasdair G Kergon <agk@redhat.com>
2009-07-14 06:15:21 +04:00
pv = pvl->pv;
vg_from->extent_count -= pv_pe_count(pv);
vg_to->extent_count += pv_pe_count(pv);
vg_from->free_count -= pv_pe_count(pv) - pv_pe_alloc_count(pv);
vg_to->free_count += pv_pe_count(pv) - pv_pe_alloc_count(pv);
return 1;
}
vgsplit: Make vgsplit work on mirrors with leg and log on same PV Given a named mirror LV, vgsplit will look for the PVs that compose it and move them to a new VG. It does this by first looking at the log and then the legs. If the log is on the same device as one of the mirror images, a problem occurs. This is because the PV is moved to the new VG as the log is processed and thus cannot be found in the current VG when the image is processed. The solution is to check and see if the PV we are looking for has already been moved to the new VG. If so, it is not an error.
2014-04-25 23:53:34 +04:00
int move_pv(struct volume_group *vg_from, struct volume_group *vg_to,
const char *pv_name)
{
return _move_pv(vg_from, vg_to, pv_name, 1);
}
Refactor vgsplit - move move_pvs_used_by_lv and move_pv inside library. In the future we may export these functions or something like them in liblvm For now this helps in cleaning up the checks for RESIZEABLE since we can use the internal library function vg_bad_status_bits. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Acked-by: Alasdair G Kergon <agk@redhat.com>
2009-07-14 06:15:21 +04:00
int move_pvs_used_by_lv(struct volume_group *vg_from,
struct volume_group *vg_to,
const char *lv_name)
{
struct lv_segment *lvseg;
unsigned s;
struct lv_list *lvl;
struct logical_volume *lv;
/* FIXME: handle tags */
if (!(lvl = find_lv_in_vg(vg_from, lv_name))) {
log_error("Logical volume %s not in volume group %s",
lv_name, vg_from->name);
return 0;
}
liblvm: Move lib code used exclusively into metadata-liblvm.c Also remove some redundant function definitions from metadata.h.
2017-10-18 21:29:32 +03:00
if (vg_bad_status_bits(vg_from, RESIZEABLE_VG) ||
vg_bad_status_bits(vg_to, RESIZEABLE_VG))
Remove READ_REQUIRE_RESIZEABLE flag from vgsplit. Remove READ_REQUIRE_RESIZEABLE flag from vgsplit similar to the removal from vgextend. Move the check inside the functions that actually move pvs from one vg structure to another. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Acked-by: Alasdair G Kergon <agk@redhat.com>
2009-07-14 06:16:05 +04:00
return 0;
Refactor vgsplit - move move_pvs_used_by_lv and move_pv inside library. In the future we may export these functions or something like them in liblvm For now this helps in cleaning up the checks for RESIZEABLE since we can use the internal library function vg_bad_status_bits. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Acked-by: Alasdair G Kergon <agk@redhat.com>
2009-07-14 06:15:21 +04:00
dm_list_iterate_items(lvseg, &lvl->lv->segments) {
if (lvseg->log_lv)
if (!move_pvs_used_by_lv(vg_from, vg_to,
lvseg->log_lv->name))
return_0;
for (s = 0; s < lvseg->area_count; s++) {
if (seg_type(lvseg, s) == AREA_PV) {
vgsplit: Make vgsplit work on mirrors with leg and log on same PV Given a named mirror LV, vgsplit will look for the PVs that compose it and move them to a new VG. It does this by first looking at the log and then the legs. If the log is on the same device as one of the mirror images, a problem occurs. This is because the PV is moved to the new VG as the log is processed and thus cannot be found in the current VG when the image is processed. The solution is to check and see if the PV we are looking for has already been moved to the new VG. If so, it is not an error.
2014-04-25 23:53:34 +04:00
if (!_move_pv(vg_from, vg_to,
pv_dev_name(seg_pv(lvseg, s)), 0))
Refactor vgsplit - move move_pvs_used_by_lv and move_pv inside library. In the future we may export these functions or something like them in liblvm For now this helps in cleaning up the checks for RESIZEABLE since we can use the internal library function vg_bad_status_bits. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Acked-by: Alasdair G Kergon <agk@redhat.com>
2009-07-14 06:15:21 +04:00
return_0;
} else if (seg_type(lvseg, s) == AREA_LV) {
lv = seg_lv(lvseg, s);
if (!move_pvs_used_by_lv(vg_from, vg_to,
lv->name))
return_0;
}
}
}
return 1;
}
lvm2app: Add VG/LV name validation C library portion for https://bugzilla.redhat.com/show_bug.cgi?id=883689 Signed-off-by: Tony Asleson <tasleson@redhat.com>
2013-09-26 20:37:40 +04:00
int validate_new_vg_name(struct cmd_context *cmd, const char *vg_name)
Move more parameter validation into the library. Update vgrename to call validate_vg_rename_params(). Fix vgcreate and vgsplit default arguments by adding defaults parameter to fill_vg_create_params(). Add t-vgrename-usage.sh test. Bugzilla: bz251992 --- tools/toollib.c | 32 ++++++++------------------------ tools/toollib.h | 5 ++--- tools/vgcreate.c | 35 +++++++++++++++++++++-------------- tools/vgrename.c | 35 ++++++----------------------------- tools/vgsplit.c | 21 ++++++++++++++------- 5 files changed, 51 insertions(+), 77 deletions(-)
2008-01-16 01:56:30 +03:00
{
Replace dynamic buffer allocations for PATH_MAX Use static buffer instead of stack allocated buffer. This reduces stack size usage of lvm tool and the change is very simple. Since the whole library is not thread safe - it should not add any new problems - and if there will be some conversion it's easy to convert this to use some preallocated buffer.
2011-11-18 23:31:09 +04:00
static char vg_path[PATH_MAX];
lvm2app: Add VG/LV name validation C library portion for https://bugzilla.redhat.com/show_bug.cgi?id=883689 Signed-off-by: Tony Asleson <tasleson@redhat.com>
2013-09-26 20:37:40 +04:00
name_error_t name_error;
Move more parameter validation into the library. Update vgrename to call validate_vg_rename_params(). Fix vgcreate and vgsplit default arguments by adding defaults parameter to fill_vg_create_params(). Add t-vgrename-usage.sh test. Bugzilla: bz251992 --- tools/toollib.c | 32 ++++++++------------------------ tools/toollib.h | 5 ++--- tools/vgcreate.c | 35 +++++++++++++++++++++-------------- tools/vgrename.c | 35 ++++++----------------------------- tools/vgsplit.c | 21 ++++++++++++++------- 5 files changed, 51 insertions(+), 77 deletions(-)
2008-01-16 01:56:30 +03:00
lvm2app: Add VG/LV name validation C library portion for https://bugzilla.redhat.com/show_bug.cgi?id=883689 Signed-off-by: Tony Asleson <tasleson@redhat.com>
2013-09-26 20:37:40 +04:00
name_error = validate_name_detailed(vg_name);
if (NAME_VALID != name_error) {
display_name_error(name_error);
log_error("New volume group name \"%s\" is invalid.", vg_name);
cleanup: move log_error upward in code stack Report log_error earlier.
2012-10-16 12:20:02 +04:00
return 0;
}
Move more parameter validation into the library. Update vgrename to call validate_vg_rename_params(). Fix vgcreate and vgsplit default arguments by adding defaults parameter to fill_vg_create_params(). Add t-vgrename-usage.sh test. Bugzilla: bz251992 --- tools/toollib.c | 32 ++++++++------------------------ tools/toollib.h | 5 ++--- tools/vgcreate.c | 35 +++++++++++++++++++++-------------- tools/vgrename.c | 35 ++++++----------------------------- tools/vgsplit.c | 21 ++++++++++++++------- 5 files changed, 51 insertions(+), 77 deletions(-)
2008-01-16 01:56:30 +03:00
Replace dynamic buffer allocations for PATH_MAX Use static buffer instead of stack allocated buffer. This reduces stack size usage of lvm tool and the change is very simple. Since the whole library is not thread safe - it should not add any new problems - and if there will be some conversion it's easy to convert this to use some preallocated buffer.
2011-11-18 23:31:09 +04:00
snprintf(vg_path, sizeof(vg_path), "%s%s", cmd->dev_dir, vg_name);
Move more parameter validation into the library. Update vgrename to call validate_vg_rename_params(). Fix vgcreate and vgsplit default arguments by adding defaults parameter to fill_vg_create_params(). Add t-vgrename-usage.sh test. Bugzilla: bz251992 --- tools/toollib.c | 32 ++++++++------------------------ tools/toollib.h | 5 ++--- tools/vgcreate.c | 35 +++++++++++++++++++++-------------- tools/vgrename.c | 35 ++++++----------------------------- tools/vgsplit.c | 21 ++++++++++++++------- 5 files changed, 51 insertions(+), 77 deletions(-)
2008-01-16 01:56:30 +03:00
if (path_exists(vg_path)) {
log_error("%s: already exists in filesystem", vg_path);
return 0;
}
return 1;
}
int validate_vg_rename_params(struct cmd_context *cmd,
const char *vg_name_old,
const char *vg_name_new)
{
unsigned length;
char *dev_dir;
dev_dir = cmd->dev_dir;
length = strlen(dev_dir);
/* Check sanity of new name */
if (strlen(vg_name_new) > NAME_LEN - length - 2) {
log_error("New volume group path exceeds maximum length "
"of %d!", NAME_LEN - length - 2);
return 0;
}
cleanup: move log_error upward in code stack Report log_error earlier.
2012-10-16 12:20:02 +04:00
if (!validate_new_vg_name(cmd, vg_name_new))
return_0;
Move more parameter validation into the library. Update vgrename to call validate_vg_rename_params(). Fix vgcreate and vgsplit default arguments by adding defaults parameter to fill_vg_create_params(). Add t-vgrename-usage.sh test. Bugzilla: bz251992 --- tools/toollib.c | 32 ++++++++------------------------ tools/toollib.h | 5 ++--- tools/vgcreate.c | 35 +++++++++++++++++++++-------------- tools/vgrename.c | 35 ++++++----------------------------- tools/vgsplit.c | 21 ++++++++++++++------- 5 files changed, 51 insertions(+), 77 deletions(-)
2008-01-16 01:56:30 +03:00
if (!strcmp(vg_name_old, vg_name_new)) {
log_error("Old and new volume group names must differ");
return 0;
}
return 1;
}
Default stripesize 64k & config file setting for it; Clear many compiler warnings (i386) & associated bugs - hopefully without introducing too many new bugs:-) (Same exercise required for other archs.) Default compilation has optimisation - or else use ./configure --enable-debug
2002-12-20 02:25:55 +03:00
int vg_rename(struct cmd_context *cmd, struct volume_group *vg,
const char *new_name)
{
Properly release VG memory pool in metadata manipulation code.
2009-04-10 14:01:08 +04:00
struct dm_pool *mem = vg->vgmem;
Basic support for mirrors.
2005-06-01 20:51:55 +04:00
struct pv_list *pvl;
Default stripesize 64k & config file setting for it; Clear many compiler warnings (i386) & associated bugs - hopefully without introducing too many new bugs:-) (Same exercise required for other archs.) Default compilation has optimisation - or else use ./configure --enable-debug
2002-12-20 02:25:55 +03:00
Do not reset position in metadata ring buffer on vgrename and vgcfgrestore. We should write metadata into next position in the ring buffer while calling vgrename and vgcfgrestore. At this code level (_vg_write_raw), we were not able to determine if this is a rename or not. If yes, then accompanying VG structure passed here has a new name set, not the old one. When looking for a location where to put metadata next, we were given a NULL value because of failed VG name comparison (in _find_vg_rlocn) between the name in existing metadata and metadata we're just about to write. This resets the position in the ring buffer, overwriting any existing metadata (and also incorrectly updates the cache to "orphan" afterwards). This patch just adds old_name item in struct volume_group that we can check and use if necessary and detect renames at lower layers as well. The same applies for vgcfgrestore, but here we're using a special value of old_name, an empty string, to disable the check with existing metadata totally.
2010-04-14 17:09:16 +04:00
vg->old_name = vg->name;
Use hash, bitset, malloc, pool from libdevmapper.
2005-10-17 03:03:59 +04:00
if (!(vg->name = dm_pool_strdup(mem, new_name))) {
Default stripesize 64k & config file setting for it; Clear many compiler warnings (i386) & associated bugs - hopefully without introducing too many new bugs:-) (Same exercise required for other archs.) Default compilation has optimisation - or else use ./configure --enable-debug
2002-12-20 02:25:55 +03:00
log_error("vg->name allocation failed for '%s'", new_name);
return 0;
}
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(pvl, &vg->pvs) {
vgsplit: Fix detection of moved PVs. vgsplit shares the vg_rename code so that must only set the PV_MOVED_VG flag introduced in commit 486ed108481cfbe4336f0fa590cbae251188ecc6 ("vgmerge: Fix intermediate metadata corruption") on PVs that moved.
2017-10-27 23:38:16 +03:00
/* Skip if VG didn't change e.g. with vgsplit */
if (pvl->pv->vg_name && !strcmp(new_name, pvl->pv->vg_name))
continue;
Use hash, bitset, malloc, pool from libdevmapper.
2005-10-17 03:03:59 +04:00
if (!(pvl->pv->vg_name = dm_pool_strdup(mem, new_name))) {
Default stripesize 64k & config file setting for it; Clear many compiler warnings (i386) & associated bugs - hopefully without introducing too many new bugs:-) (Same exercise required for other archs.) Default compilation has optimisation - or else use ./configure --enable-debug
2002-12-20 02:25:55 +03:00
log_error("pv->vg_name allocation failed for '%s'",
Add pv_dev_name() to access PV device name. Patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 18:29:32 +04:00
pv_dev_name(pvl->pv));
Default stripesize 64k & config file setting for it; Clear many compiler warnings (i386) & associated bugs - hopefully without introducing too many new bugs:-) (Same exercise required for other archs.) Default compilation has optimisation - or else use ./configure --enable-debug
2002-12-20 02:25:55 +03:00
return 0;
}
vgsplit: Fix detection of moved PVs. vgsplit shares the vg_rename code so that must only set the PV_MOVED_VG flag introduced in commit 486ed108481cfbe4336f0fa590cbae251188ecc6 ("vgmerge: Fix intermediate metadata corruption") on PVs that moved.
2017-10-27 23:38:16 +03:00
vgmerge: Fix intermediate metadata corruption vgmerge suffers from a similar problem to the one fixed in commit 8146548d25e9104f0d530d943290d448c1994c0a ("vgsplit: Fix intermediate metadata corruption.") When merging, splitting or renaming VGs, use a new PV status flag PV_MOVED_VG to mark the PVs that hold metadata with the old VG name and use this to provide PV-level granularity instead of incorrectly assuming all PVs in the VG are the same.
2017-10-06 04:12:42 +03:00
/* Mark the PVs that still hold metadata with the old VG name */
vgsplit: Fix detection of moved PVs. vgsplit shares the vg_rename code so that must only set the PV_MOVED_VG flag introduced in commit 486ed108481cfbe4336f0fa590cbae251188ecc6 ("vgmerge: Fix intermediate metadata corruption") on PVs that moved.
2017-10-27 23:38:16 +03:00
log_debug_metadata("Marking PV %s as moved to VG %s", dev_name(pvl->pv->dev), new_name);
pvl->pv->status |= PV_MOVED_VG;
Default stripesize 64k & config file setting for it; Clear many compiler warnings (i386) & associated bugs - hopefully without introducing too many new bugs:-) (Same exercise required for other archs.) Default compilation has optimisation - or else use ./configure --enable-debug
2002-12-20 02:25:55 +03:00
}
return 1;
}
Split vg_remove_single into 2 functions - the second part commits to disk. Split vg_remove_single into vg_remove_check (mandatory checks before vgremove) and vg_remove (do actual remove by committing to disk). In liblvm, we'd like to provide an consistent API that allows multiple changes in memory, then let lvm_vg_write() control the commit to disk. In some cases (for example, lvresize calls fsadm) this may not be possible. However, since we are using an object model and dividing things into small operations, the most logical model seems to be the lvm_vg_write model, and handling the special cases as they arrive. So as best as possible we move towards this end. A possible optimization would be to consolidate vg_remove (committing) code with vgreduce code. A second possible optimization is making vgreduce of the last device equivalent to vgremove. Today, lvm_vg_reduce fails if vgreduce is called with the last device, but from an object model perspective we could view this as equivalent to vgremove and allow it. My gut feel is we do not want to do this though. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-09-03 01:39:29 +04:00
int vg_remove_check(struct volume_group *vg)
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
{
Added displayable_lvs_in_vg and lv_is_displayable functions to deal with the counts of visible LVs from user's perspective consistently throughout the code.
2008-12-04 18:54:26 +03:00
unsigned lv_count;
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
if (vg_missing_pv_count(vg)) {
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
log_error("Volume group \"%s\" not found, is inconsistent "
Remove unnecessary parameters from vg_remove_single(). Use vg_t instead of struct volume_group. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2009-07-11 00:05:29 +04:00
"or has PVs missing.", vg ? vg->name : "");
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
log_error("Consider vgreduce --removemissing if metadata "
"is inconsistent.");
return 0;
}
Merge lv_is_displayable and lv_is_visible. Displayable and visible is the same thing. volumes_count(vg) is now vg_visible_lvs() and always returns number of LVs from user perspective.
2009-05-14 01:27:43 +04:00
lv_count = vg_visible_lvs(vg);
Added displayable_lvs_in_vg and lv_is_displayable functions to deal with the counts of visible LVs from user's perspective consistently throughout the code.
2008-12-04 18:54:26 +03:00
if (lv_count) {
log_error("Volume group \"%s\" still contains %u "
Remove unnecessary parameters from vg_remove_single(). Use vg_t instead of struct volume_group. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2009-07-11 00:05:29 +04:00
"logical volume(s)", vg->name, lv_count);
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
return 0;
}
if (!archive(vg))
return 0;
Refactor vg_remove_check to place pv removal into separate function.
2010-06-30 22:03:52 +04:00
return 1;
}
void vg_remove_pvs(struct volume_group *vg)
{
struct pv_list *pvl, *tpvl;
Update vg_remove_single_* functions to use the removed_pvs list. Now that we've split vg_remove_single into two routines, in the first routine that only manipulates memory, we move the PVs from the vg->pvs list to the vg->removed_pvs list. Then later, we iterate through this list to write the removed PVs to disk, which removes them from the volume group and places them into the internal ORPHAN VG. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Author: Dave Wysochanski <dwysocha@redhat.com>
2009-09-03 01:39:49 +04:00
dm_list_iterate_items_safe(pvl, tpvl, &vg->pvs) {
Call add_pvl_to_vgs() and del_pvl_from_vgs() from more places. Now that we have library functions to add/delete a pv from the vg->pvs list, call them from everywhere. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-13 21:26:03 +04:00
del_pvl_from_vgs(vg, pvl);
Update vg_remove_single_* functions to use the removed_pvs list. Now that we've split vg_remove_single into two routines, in the first routine that only manipulates memory, we move the PVs from the vg->pvs list to the vg->removed_pvs list. Then later, we iterate through this list to write the removed PVs to disk, which removes them from the volume group and places them into the internal ORPHAN VG. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Author: Dave Wysochanski <dwysocha@redhat.com>
2009-09-03 01:39:49 +04:00
dm_list_add(&vg->removed_pvs, &pvl->list);
}
Split vg_remove_single into 2 functions - the second part commits to disk. Split vg_remove_single into vg_remove_check (mandatory checks before vgremove) and vg_remove (do actual remove by committing to disk). In liblvm, we'd like to provide an consistent API that allows multiple changes in memory, then let lvm_vg_write() control the commit to disk. In some cases (for example, lvresize calls fsadm) this may not be possible. However, since we are using an object model and dividing things into small operations, the most logical model seems to be the lvm_vg_write model, and handling the special cases as they arrive. So as best as possible we move towards this end. A possible optimization would be to consolidate vg_remove (committing) code with vgreduce code. A second possible optimization is making vgreduce of the last device equivalent to vgremove. Today, lvm_vg_reduce fails if vgreduce is called with the last device, but from an object model perspective we could view this as equivalent to vgremove and allow it. My gut feel is we do not want to do this though. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-09-03 01:39:29 +04:00
}
Add lvmlockd
2015-03-05 23:00:44 +03:00
int vg_remove_direct(struct volume_group *vg)
Split vg_remove_single into 2 functions - the second part commits to disk. Split vg_remove_single into vg_remove_check (mandatory checks before vgremove) and vg_remove (do actual remove by committing to disk). In liblvm, we'd like to provide an consistent API that allows multiple changes in memory, then let lvm_vg_write() control the commit to disk. In some cases (for example, lvresize calls fsadm) this may not be possible. However, since we are using an object model and dividing things into small operations, the most logical model seems to be the lvm_vg_write model, and handling the special cases as they arrive. So as best as possible we move towards this end. A possible optimization would be to consolidate vg_remove (committing) code with vgreduce code. A second possible optimization is making vgreduce of the last device equivalent to vgremove. Today, lvm_vg_reduce fails if vgreduce is called with the last device, but from an object model perspective we could view this as equivalent to vgremove and allow it. My gut feel is we do not want to do this though. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-09-03 01:39:29 +04:00
{
struct physical_volume *pv;
struct pv_list *pvl;
int ret = 1;
Rename internal library function vg_remove to vg_remove_mdas. Later patches should consolidate the vgremove / vgreduce functions but for now let's clarify what vg_remove actually does by changing the name. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-09-03 01:39:07 +04:00
if (!vg_remove_mdas(vg)) {
log_error("vg_remove_mdas %s failed", vg->name);
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
return 0;
}
/* init physical volumes */
Update vg_remove_single_* functions to use the removed_pvs list. Now that we've split vg_remove_single into two routines, in the first routine that only manipulates memory, we move the PVs from the vg->pvs list to the vg->removed_pvs list. Then later, we iterate through this list to write the removed PVs to disk, which removes them from the volume group and places them into the internal ORPHAN VG. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Author: Dave Wysochanski <dwysocha@redhat.com>
2009-09-03 01:39:49 +04:00
dm_list_iterate_items(pvl, &vg->removed_pvs) {
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
pv = pvl->pv;
Check for missing_pv in vg_remove loop. If a pv is missing, we should just skip it rather than checking the device size and failing the vgremove.
2010-06-30 23:55:43 +04:00
if (is_missing_pv(pv))
continue;
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
log_verbose("Removing physical volume \"%s\" from "
Remove unnecessary parameters from vg_remove_single(). Use vg_t instead of struct volume_group. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2009-07-11 00:05:29 +04:00
"volume group \"%s\"", pv_dev_name(pv), vg->name);
split orphan VG by format type
2008-02-06 18:47:28 +03:00
pv->vg_name = vg->fid->fmt->orphan_vg_name;
metadata: fix ALLOCATABLE_PV for lvm1 format This is addendum for commit 6dc7b783c80c6834aa724df49b5d47b9b5601506. LVM1 format stores the ALLOCATABLE flag directly in PV header, not in VG metadata. So the code needs to be fixed further to work properly for lvm1 format so that the correct PV header is written (the flag is set only if the PV is in some VG, unset otherwise).
2014-07-11 14:24:15 +04:00
pv->status &= ~ALLOCATABLE_PV;
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
if (!dev_get_size(pv_dev(pv), &pv->size)) {
Add pv_dev_name() to access PV device name. Patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 18:29:32 +04:00
log_error("%s: Couldn't get size.", pv_dev_name(pv));
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
ret = 0;
continue;
}
/* FIXME Write to same sector label was read from */
Allow non-orphan PVs with two metadata areas to be resized. We allow writing non-orphan PVs only for resize now. The "orphan PV" assert in pv_write fn uses the "allow_non_orphan" parameter to control this assert. However, we should find a more elaborate solution so we can remove this restriction altogether (pv_write together with vg_write is not atomic, we need to find a safe mechanism so there's an easy revert possible in case of an error).
2011-02-28 16:19:02 +03:00
if (!pv_write(vg->cmd, pv, 0)) {
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
log_error("Failed to remove physical volume \"%s\""
" from volume group \"%s\"",
Remove unnecessary parameters from vg_remove_single(). Use vg_t instead of struct volume_group. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2009-07-11 00:05:29 +04:00
pv_dev_name(pv), vg->name);
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
ret = 0;
}
}
Add lvmlockd
2015-03-05 23:00:44 +03:00
lockd_vg_update(vg);
dbus: add notification from commands When a command modifies a PV or VG, or changes the activation state of an LV, it will send a dbus notification when the command is finished. This can be enabled/disabled with a config setting.
2016-02-22 18:42:03 +03:00
set_vg_notify(vg->cmd);
Add backtraces for backup and backup_remove fail paths
2010-12-22 18:36:41 +03:00
if (!backup_remove(vg->cmd, vg->name))
stack;
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
if (ret)
config: add silent mode Accept -q as the short form of --quiet. Suppress non-essential standard output if -q is given twice. Treat log/silent in lvm.conf as equivalent to -qq. Review all log_print messages and change some to log_print_unless_silent. When silent, the following commands still produce output: dumpconfig, lvdisplay, lvmdiskscan, lvs, pvck, pvdisplay, pvs, version, vgcfgrestore -l, vgdisplay, vgs. [Needs checking.] Non-essential messages are shifted from log level 4 to log level 5 for syslog and lvm2_log_fn purposes.
2012-08-25 23:35:48 +04:00
log_print_unless_silent("Volume group \"%s\" successfully removed", vg->name);
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
else
Remove unnecessary parameters from vg_remove_single(). Use vg_t instead of struct volume_group. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2009-07-11 00:05:29 +04:00
log_error("Volume group \"%s\" not properly removed", vg->name);
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
Add lvmlockd
2015-03-05 23:00:44 +03:00
return ret;
}
int vg_remove(struct volume_group *vg)
{
int ret;
ret = vg_remove_direct(vg);
Move guts of vgremove into lvm library. Include archiver.h in metadata.c as a result of prior move.
2007-08-21 21:38:20 +04:00
return ret;
}
device: add physical block size info and make sure VG extent size >= PV's phys. block size
2013-12-12 14:26:35 +04:00
int check_dev_block_size_for_vg(struct device *dev, const struct volume_group *vg,
devices: clean up block size functions Replace calls to the old dev_get_block_size function with calls to the new dev_get_direct_block_size function, and remove the old function.
2019-08-07 19:38:06 +03:00
unsigned int *max_logical_block_size_found)
device: add physical block size info and make sure VG extent size >= PV's phys. block size
2013-12-12 14:26:35 +04:00
{
devices: clean up block size functions Replace calls to the old dev_get_block_size function with calls to the new dev_get_direct_block_size function, and remove the old function.
2019-08-07 19:38:06 +03:00
unsigned int physical_block_size, logical_block_size;
device: add physical block size info and make sure VG extent size >= PV's phys. block size
2013-12-12 14:26:35 +04:00
devices: clean up block size functions Replace calls to the old dev_get_block_size function with calls to the new dev_get_direct_block_size function, and remove the old function.
2019-08-07 19:38:06 +03:00
if (!(dev_get_direct_block_sizes(dev, &physical_block_size, &logical_block_size)))
device: add physical block size info and make sure VG extent size >= PV's phys. block size
2013-12-12 14:26:35 +04:00
return_0;
devices: clean up block size functions Replace calls to the old dev_get_block_size function with calls to the new dev_get_direct_block_size function, and remove the old function.
2019-08-07 19:38:06 +03:00
/* FIXME: max_logical_block_size_found does not seem to be used anywhere */
if (logical_block_size > *max_logical_block_size_found)
*max_logical_block_size_found = logical_block_size;
device: add physical block size info and make sure VG extent size >= PV's phys. block size
2013-12-12 14:26:35 +04:00
devices: clean up block size functions Replace calls to the old dev_get_block_size function with calls to the new dev_get_direct_block_size function, and remove the old function.
2019-08-07 19:38:06 +03:00
if (logical_block_size >> SECTOR_SHIFT > vg->extent_size) {
device: add physical block size info and make sure VG extent size >= PV's phys. block size
2013-12-12 14:26:35 +04:00
log_error("Physical extent size used for volume group %s "
devices: clean up block size functions Replace calls to the old dev_get_block_size function with calls to the new dev_get_direct_block_size function, and remove the old function.
2019-08-07 19:38:06 +03:00
"is less than logical block size (%u bytes) that %s uses.",
vg->name, logical_block_size, dev_name(dev));
device: add physical block size info and make sure VG extent size >= PV's phys. block size
2013-12-12 14:26:35 +04:00
return 0;
}
return 1;
}
int vg_check_pv_dev_block_sizes(const struct volume_group *vg)
{
struct pv_list *pvl;
devices: clean up block size functions Replace calls to the old dev_get_block_size function with calls to the new dev_get_direct_block_size function, and remove the old function.
2019-08-07 19:38:06 +03:00
unsigned int max_logical_block_size_found = 0;
device: add physical block size info and make sure VG extent size >= PV's phys. block size
2013-12-12 14:26:35 +04:00
dm_list_iterate_items(pvl, &vg->pvs) {
devices: clean up block size functions Replace calls to the old dev_get_block_size function with calls to the new dev_get_direct_block_size function, and remove the old function.
2019-08-07 19:38:06 +03:00
if (!check_dev_block_size_for_vg(pvl->pv->dev, vg, &max_logical_block_size_found))
device: add physical block size info and make sure VG extent size >= PV's phys. block size
2013-12-12 14:26:35 +04:00
return 0;
}
return 1;
}
liblvm: Move lib code used exclusively into metadata-liblvm.c Also remove some redundant function definitions from metadata.h.
2017-10-18 21:29:32 +03:00
int check_pv_dev_sizes(struct volume_group *vg)
metadata: check PV dev size is not less than PV size
2016-01-22 13:37:09 +03:00
{
struct pv_list *pvl;
uint64_t dev_size, size;
int r = 1;
conf: add metadata/check_pv_device_sizes
2016-01-22 15:20:21 +03:00
if (!vg->cmd->check_pv_dev_sizes ||
is_orphan_vg(vg->name))
metadata: check PV dev size is not less than PV size
2016-01-22 13:37:09 +03:00
return 1;
dm_list_iterate_items(pvl, &vg->pvs) {
if (is_missing_pv(pvl->pv))
continue;
metadata: do not issue warning message about PV dev size being 0 when the device has gone just after VG read There's a window between doing VG read and checking PV device size against real device size. If the device is removed in this window, the dev cache still holds struct device and pv->dev still references that and that PV is not marked as missing. However, if we're trying to get size for such device, the open fails because that device doesn't exists anymore. We called existing pv_dev_size in _check_pv_dev_sizes fn. But pv_dev_size assigned a size of 0 if the dev_get_size it called failed (because the device is gone). So call the dev_get_size directly and check for the return code in _check_pv_dev_sizes and go further only if we really know the device size. This is to avoid confusing warning messages like: Device /dev/sdd1 has size of 0 sectors which is smaller than corresponding PV size of 31455207 sectors. Was device resized? One or more devices used as PVs in VG helter_skelter have changed sizes.
2016-03-10 15:02:38 +03:00
/*
* Don't compare the sizes if we're not able
* to determine the real dev_size. This may
* happen if the device has gone since we did
* VG read.
*/
if (!dev_get_size(pvl->pv->dev, &dev_size))
continue;
metadata: check PV dev size is not less than PV size
2016-01-22 13:37:09 +03:00
size = pv_size(pvl->pv);
if (dev_size < size) {
cleanup: enhance warning message Add WARNING: for log_warn. Show device name which is marked missing.
2016-05-05 22:13:41 +03:00
log_warn("WARNING: Device %s has size of %" PRIu64 " sectors which "
metadata: check PV dev size is not less than PV size
2016-01-22 13:37:09 +03:00
"is smaller than corresponding PV size of %" PRIu64
" sectors. Was device resized?",
pv_dev_name(pvl->pv), dev_size, size);
r = 0;
}
}
return r;
}
replace pvcreate_params with pvcreate_each_params "pvcreate_each_params" was a temporary name used to transition from the old "pvcreate_params". Remove the old pvcreate_params struct and rename the new pvcreate_each_params struct to pvcreate_params. Rename various pvcreate_each_params terms to simply pvcreate_params.
2016-02-19 00:38:23 +03:00
int vg_extend_each_pv(struct volume_group *vg, struct pvcreate_params *pp)
toollib: add two phase pv processing code This is common code for handling PV create/remove that can be shared by pvcreate/vgcreate/vgextend/pvremove. This does not change any commands to use the new code. - Pull out the hidden equivalent of process_each_pv into an actual top level process_each_pv. - Pull the prompts to the top level, and do not run any prompts while locks are held. The orphan lock is reacquired after any prompts are done, and the devices being created are checked for any change made while the lock was not held. Previously, pvcreate_vol() was the shared function for creating a PV for pvcreate, vgcreate, vgextend. Now, it will be toollib function pvcreate_each_device(). pvcreate_vol() was called effectively as a helper, from within vgcreate and vgextend code paths. pvcreate_each_device() will be called at the same level as other process_each functions. One of the main problems with pvcreate_vol() is that it included a hidden equivalent of process_each_pv for each device being created: pvcreate_vol() -> _pvcreate_check() -> find_pv_by_name() -> get_pvs() -> get_pvs_internal() -> _get_pvs() -> get_vgids() -> /* equivalent to process_each_pv */ dm_list_iterate_items(vgids) vg = vg_read_internal() dm_list_iterate_items(&vg->pvs) pvcreate_each_device() reorganizes the code so that each-VG-each-PV loop is done once, and uses the standard process_each_pv function at the top level of the function.
2016-02-16 23:15:24 +03:00
{
struct pv_list *pvl;
devices: clean up block size functions Replace calls to the old dev_get_block_size function with calls to the new dev_get_direct_block_size function, and remove the old function.
2019-08-07 19:38:06 +03:00
unsigned int max_logical_block_size = 0;
vgcreate/vgextend: restrict PVs with mixed block sizes Avoid having PVs with different logical block sizes in the same VG. This prevents LVs from having mixed block sizes, which can produce file system errors. The new config setting devices/allow_mixed_block_sizes (default 0) can be changed to 1 to return to the unrestricted mode.
2019-08-01 18:06:47 +03:00
unsigned int physical_block_size, logical_block_size;
unsigned int prev_lbs = 0;
int inconsistent_existing_lbs = 0;
toollib: add two phase pv processing code This is common code for handling PV create/remove that can be shared by pvcreate/vgcreate/vgextend/pvremove. This does not change any commands to use the new code. - Pull out the hidden equivalent of process_each_pv into an actual top level process_each_pv. - Pull the prompts to the top level, and do not run any prompts while locks are held. The orphan lock is reacquired after any prompts are done, and the devices being created are checked for any change made while the lock was not held. Previously, pvcreate_vol() was the shared function for creating a PV for pvcreate, vgcreate, vgextend. Now, it will be toollib function pvcreate_each_device(). pvcreate_vol() was called effectively as a helper, from within vgcreate and vgextend code paths. pvcreate_each_device() will be called at the same level as other process_each functions. One of the main problems with pvcreate_vol() is that it included a hidden equivalent of process_each_pv for each device being created: pvcreate_vol() -> _pvcreate_check() -> find_pv_by_name() -> get_pvs() -> get_pvs_internal() -> _get_pvs() -> get_vgids() -> /* equivalent to process_each_pv */ dm_list_iterate_items(vgids) vg = vg_read_internal() dm_list_iterate_items(&vg->pvs) pvcreate_each_device() reorganizes the code so that each-VG-each-PV loop is done once, and uses the standard process_each_pv function at the top level of the function.
2016-02-16 23:15:24 +03:00
cleanup: add some dots and use display_lvname Just some more VG/LV printing.
2016-11-25 16:08:39 +03:00
log_debug_metadata("Adding PVs to VG %s.", vg->name);
toollib: add two phase pv processing code This is common code for handling PV create/remove that can be shared by pvcreate/vgcreate/vgextend/pvremove. This does not change any commands to use the new code. - Pull out the hidden equivalent of process_each_pv into an actual top level process_each_pv. - Pull the prompts to the top level, and do not run any prompts while locks are held. The orphan lock is reacquired after any prompts are done, and the devices being created are checked for any change made while the lock was not held. Previously, pvcreate_vol() was the shared function for creating a PV for pvcreate, vgcreate, vgextend. Now, it will be toollib function pvcreate_each_device(). pvcreate_vol() was called effectively as a helper, from within vgcreate and vgextend code paths. pvcreate_each_device() will be called at the same level as other process_each functions. One of the main problems with pvcreate_vol() is that it included a hidden equivalent of process_each_pv for each device being created: pvcreate_vol() -> _pvcreate_check() -> find_pv_by_name() -> get_pvs() -> get_pvs_internal() -> _get_pvs() -> get_vgids() -> /* equivalent to process_each_pv */ dm_list_iterate_items(vgids) vg = vg_read_internal() dm_list_iterate_items(&vg->pvs) pvcreate_each_device() reorganizes the code so that each-VG-each-PV loop is done once, and uses the standard process_each_pv function at the top level of the function.
2016-02-16 23:15:24 +03:00
liblvm: Move lib code used exclusively into metadata-liblvm.c Also remove some redundant function definitions from metadata.h.
2017-10-18 21:29:32 +03:00
if (vg_bad_status_bits(vg, RESIZEABLE_VG))
toollib: add two phase pv processing code This is common code for handling PV create/remove that can be shared by pvcreate/vgcreate/vgextend/pvremove. This does not change any commands to use the new code. - Pull out the hidden equivalent of process_each_pv into an actual top level process_each_pv. - Pull the prompts to the top level, and do not run any prompts while locks are held. The orphan lock is reacquired after any prompts are done, and the devices being created are checked for any change made while the lock was not held. Previously, pvcreate_vol() was the shared function for creating a PV for pvcreate, vgcreate, vgextend. Now, it will be toollib function pvcreate_each_device(). pvcreate_vol() was called effectively as a helper, from within vgcreate and vgextend code paths. pvcreate_each_device() will be called at the same level as other process_each functions. One of the main problems with pvcreate_vol() is that it included a hidden equivalent of process_each_pv for each device being created: pvcreate_vol() -> _pvcreate_check() -> find_pv_by_name() -> get_pvs() -> get_pvs_internal() -> _get_pvs() -> get_vgids() -> /* equivalent to process_each_pv */ dm_list_iterate_items(vgids) vg = vg_read_internal() dm_list_iterate_items(&vg->pvs) pvcreate_each_device() reorganizes the code so that each-VG-each-PV loop is done once, and uses the standard process_each_pv function at the top level of the function.
2016-02-16 23:15:24 +03:00
return_0;
vgcreate/vgextend: restrict PVs with mixed block sizes Avoid having PVs with different logical block sizes in the same VG. This prevents LVs from having mixed block sizes, which can produce file system errors. The new config setting devices/allow_mixed_block_sizes (default 0) can be changed to 1 to return to the unrestricted mode.
2019-08-01 18:06:47 +03:00
/*
* Check if existing PVs have inconsistent block sizes.
* If so, do not enforce new devices to be consistent.
*/
dm_list_iterate_items(pvl, &vg->pvs) {
logical_block_size = 0;
physical_block_size = 0;
vgextend: check missing device during block size check Checking the block size when a device is missing could trigger a segfault.
2019-09-03 18:07:56 +03:00
if (!pvl->pv->dev)
continue;
vgcreate/vgextend: restrict PVs with mixed block sizes Avoid having PVs with different logical block sizes in the same VG. This prevents LVs from having mixed block sizes, which can produce file system errors. The new config setting devices/allow_mixed_block_sizes (default 0) can be changed to 1 to return to the unrestricted mode.
2019-08-01 18:06:47 +03:00
if (!dev_get_direct_block_sizes(pvl->pv->dev, &physical_block_size, &logical_block_size))
continue;
if (!logical_block_size)
continue;
if (!prev_lbs) {
prev_lbs = logical_block_size;
continue;
}
if (prev_lbs != logical_block_size) {
inconsistent_existing_lbs = 1;
break;
}
}
toollib: add two phase pv processing code This is common code for handling PV create/remove that can be shared by pvcreate/vgcreate/vgextend/pvremove. This does not change any commands to use the new code. - Pull out the hidden equivalent of process_each_pv into an actual top level process_each_pv. - Pull the prompts to the top level, and do not run any prompts while locks are held. The orphan lock is reacquired after any prompts are done, and the devices being created are checked for any change made while the lock was not held. Previously, pvcreate_vol() was the shared function for creating a PV for pvcreate, vgcreate, vgextend. Now, it will be toollib function pvcreate_each_device(). pvcreate_vol() was called effectively as a helper, from within vgcreate and vgextend code paths. pvcreate_each_device() will be called at the same level as other process_each functions. One of the main problems with pvcreate_vol() is that it included a hidden equivalent of process_each_pv for each device being created: pvcreate_vol() -> _pvcreate_check() -> find_pv_by_name() -> get_pvs() -> get_pvs_internal() -> _get_pvs() -> get_vgids() -> /* equivalent to process_each_pv */ dm_list_iterate_items(vgids) vg = vg_read_internal() dm_list_iterate_items(&vg->pvs) pvcreate_each_device() reorganizes the code so that each-VG-each-PV loop is done once, and uses the standard process_each_pv function at the top level of the function.
2016-02-16 23:15:24 +03:00
dm_list_iterate_items(pvl, &pp->pvs) {
cleanup: add some dots and use display_lvname Just some more VG/LV printing.
2016-11-25 16:08:39 +03:00
log_debug_metadata("Adding PV %s to VG %s.", pv_dev_name(pvl->pv), vg->name);
toollib: add two phase pv processing code This is common code for handling PV create/remove that can be shared by pvcreate/vgcreate/vgextend/pvremove. This does not change any commands to use the new code. - Pull out the hidden equivalent of process_each_pv into an actual top level process_each_pv. - Pull the prompts to the top level, and do not run any prompts while locks are held. The orphan lock is reacquired after any prompts are done, and the devices being created are checked for any change made while the lock was not held. Previously, pvcreate_vol() was the shared function for creating a PV for pvcreate, vgcreate, vgextend. Now, it will be toollib function pvcreate_each_device(). pvcreate_vol() was called effectively as a helper, from within vgcreate and vgextend code paths. pvcreate_each_device() will be called at the same level as other process_each functions. One of the main problems with pvcreate_vol() is that it included a hidden equivalent of process_each_pv for each device being created: pvcreate_vol() -> _pvcreate_check() -> find_pv_by_name() -> get_pvs() -> get_pvs_internal() -> _get_pvs() -> get_vgids() -> /* equivalent to process_each_pv */ dm_list_iterate_items(vgids) vg = vg_read_internal() dm_list_iterate_items(&vg->pvs) pvcreate_each_device() reorganizes the code so that each-VG-each-PV loop is done once, and uses the standard process_each_pv function at the top level of the function.
2016-02-16 23:15:24 +03:00
if (!(check_dev_block_size_for_vg(pvl->pv->dev,
(const struct volume_group *) vg,
devices: clean up block size functions Replace calls to the old dev_get_block_size function with calls to the new dev_get_direct_block_size function, and remove the old function.
2019-08-07 19:38:06 +03:00
&max_logical_block_size))) {
cleanup: add some dots and use display_lvname Just some more VG/LV printing.
2016-11-25 16:08:39 +03:00
log_error("PV %s has wrong block size.", pv_dev_name(pvl->pv));
debug: more stacktrace corrections Continue previous patch dropping some unneeded stack traces after printed log_error/warn messages.
2016-11-25 15:46:06 +03:00
return 0;
toollib: add two phase pv processing code This is common code for handling PV create/remove that can be shared by pvcreate/vgcreate/vgextend/pvremove. This does not change any commands to use the new code. - Pull out the hidden equivalent of process_each_pv into an actual top level process_each_pv. - Pull the prompts to the top level, and do not run any prompts while locks are held. The orphan lock is reacquired after any prompts are done, and the devices being created are checked for any change made while the lock was not held. Previously, pvcreate_vol() was the shared function for creating a PV for pvcreate, vgcreate, vgextend. Now, it will be toollib function pvcreate_each_device(). pvcreate_vol() was called effectively as a helper, from within vgcreate and vgextend code paths. pvcreate_each_device() will be called at the same level as other process_each functions. One of the main problems with pvcreate_vol() is that it included a hidden equivalent of process_each_pv for each device being created: pvcreate_vol() -> _pvcreate_check() -> find_pv_by_name() -> get_pvs() -> get_pvs_internal() -> _get_pvs() -> get_vgids() -> /* equivalent to process_each_pv */ dm_list_iterate_items(vgids) vg = vg_read_internal() dm_list_iterate_items(&vg->pvs) pvcreate_each_device() reorganizes the code so that each-VG-each-PV loop is done once, and uses the standard process_each_pv function at the top level of the function.
2016-02-16 23:15:24 +03:00
}
vgcreate/vgextend: restrict PVs with mixed block sizes Avoid having PVs with different logical block sizes in the same VG. This prevents LVs from having mixed block sizes, which can produce file system errors. The new config setting devices/allow_mixed_block_sizes (default 0) can be changed to 1 to return to the unrestricted mode.
2019-08-01 18:06:47 +03:00
logical_block_size = 0;
physical_block_size = 0;
if (!dev_get_direct_block_sizes(pvl->pv->dev, &physical_block_size, &logical_block_size))
log_warn("WARNING: PV %s has unknown block size.", pv_dev_name(pvl->pv));
else if (prev_lbs && logical_block_size && (logical_block_size != prev_lbs)) {
if (vg->cmd->allow_mixed_block_sizes || inconsistent_existing_lbs)
log_debug("Devices have inconsistent block sizes (%u and %u)", prev_lbs, logical_block_size);
else {
log_error("Devices have inconsistent logical block sizes (%u and %u).",
prev_lbs, logical_block_size);
return 0;
}
}
liblvm: Move lib code used exclusively into metadata-liblvm.c Also remove some redundant function definitions from metadata.h.
2017-10-18 21:29:32 +03:00
if (!add_pv_to_vg(vg, pv_dev_name(pvl->pv), pvl->pv, 0)) {
toollib: add two phase pv processing code This is common code for handling PV create/remove that can be shared by pvcreate/vgcreate/vgextend/pvremove. This does not change any commands to use the new code. - Pull out the hidden equivalent of process_each_pv into an actual top level process_each_pv. - Pull the prompts to the top level, and do not run any prompts while locks are held. The orphan lock is reacquired after any prompts are done, and the devices being created are checked for any change made while the lock was not held. Previously, pvcreate_vol() was the shared function for creating a PV for pvcreate, vgcreate, vgextend. Now, it will be toollib function pvcreate_each_device(). pvcreate_vol() was called effectively as a helper, from within vgcreate and vgextend code paths. pvcreate_each_device() will be called at the same level as other process_each functions. One of the main problems with pvcreate_vol() is that it included a hidden equivalent of process_each_pv for each device being created: pvcreate_vol() -> _pvcreate_check() -> find_pv_by_name() -> get_pvs() -> get_pvs_internal() -> _get_pvs() -> get_vgids() -> /* equivalent to process_each_pv */ dm_list_iterate_items(vgids) vg = vg_read_internal() dm_list_iterate_items(&vg->pvs) pvcreate_each_device() reorganizes the code so that each-VG-each-PV loop is done once, and uses the standard process_each_pv function at the top level of the function.
2016-02-16 23:15:24 +03:00
log_error("PV %s cannot be added to VG %s.",
pv_dev_name(pvl->pv), vg->name);
debug: more stacktrace corrections Continue previous patch dropping some unneeded stack traces after printed log_error/warn messages.
2016-11-25 15:46:06 +03:00
return 0;
toollib: add two phase pv processing code This is common code for handling PV create/remove that can be shared by pvcreate/vgcreate/vgextend/pvremove. This does not change any commands to use the new code. - Pull out the hidden equivalent of process_each_pv into an actual top level process_each_pv. - Pull the prompts to the top level, and do not run any prompts while locks are held. The orphan lock is reacquired after any prompts are done, and the devices being created are checked for any change made while the lock was not held. Previously, pvcreate_vol() was the shared function for creating a PV for pvcreate, vgcreate, vgextend. Now, it will be toollib function pvcreate_each_device(). pvcreate_vol() was called effectively as a helper, from within vgcreate and vgextend code paths. pvcreate_each_device() will be called at the same level as other process_each functions. One of the main problems with pvcreate_vol() is that it included a hidden equivalent of process_each_pv for each device being created: pvcreate_vol() -> _pvcreate_check() -> find_pv_by_name() -> get_pvs() -> get_pvs_internal() -> _get_pvs() -> get_vgids() -> /* equivalent to process_each_pv */ dm_list_iterate_items(vgids) vg = vg_read_internal() dm_list_iterate_items(&vg->pvs) pvcreate_each_device() reorganizes the code so that each-VG-each-PV loop is done once, and uses the standard process_each_pv function at the top level of the function.
2016-02-16 23:15:24 +03:00
}
}
liblvm: Move lib code used exclusively into metadata-liblvm.c Also remove some redundant function definitions from metadata.h.
2017-10-18 21:29:32 +03:00
(void) check_pv_dev_sizes(vg);
toollib: add two phase pv processing code This is common code for handling PV create/remove that can be shared by pvcreate/vgcreate/vgextend/pvremove. This does not change any commands to use the new code. - Pull out the hidden equivalent of process_each_pv into an actual top level process_each_pv. - Pull the prompts to the top level, and do not run any prompts while locks are held. The orphan lock is reacquired after any prompts are done, and the devices being created are checked for any change made while the lock was not held. Previously, pvcreate_vol() was the shared function for creating a PV for pvcreate, vgcreate, vgextend. Now, it will be toollib function pvcreate_each_device(). pvcreate_vol() was called effectively as a helper, from within vgcreate and vgextend code paths. pvcreate_each_device() will be called at the same level as other process_each functions. One of the main problems with pvcreate_vol() is that it included a hidden equivalent of process_each_pv for each device being created: pvcreate_vol() -> _pvcreate_check() -> find_pv_by_name() -> get_pvs() -> get_pvs_internal() -> _get_pvs() -> get_vgids() -> /* equivalent to process_each_pv */ dm_list_iterate_items(vgids) vg = vg_read_internal() dm_list_iterate_items(&vg->pvs) pvcreate_each_device() reorganizes the code so that each-VG-each-PV loop is done once, and uses the standard process_each_pv function at the top level of the function.
2016-02-16 23:15:24 +03:00
dm_list_splice(&vg->pv_write_list, &pp->pvs);
return 1;
}
Refactor lvchange_tag() to call lv_change_tag() library function. Similar refactoring to vgchange - pull out common parts and put into library function for reuse. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:49 +03:00
int lv_change_tag(struct logical_volume *lv, const char *tag, int add_tag)
{
Add dm_pool_strdup to allocate memory and copy a tag in {lv|vg}_change_tag() We need to allocate memory for the tag and copy the tag value before we add it to the list of tags. We could put this inside lvm2app since the tools keep their memory around until vg_write/vg_commit is called, but we put it inside the internal library to minimize code in lvm2app. We need to copy the tag passed in by the caller to ensure the lifetime of the memory until the {vg|lv} handle is released. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:57 +03:00
char *tag_new;
Refactor lvchange_tag() to call lv_change_tag() library function. Similar refactoring to vgchange - pull out common parts and put into library function for reuse. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:49 +03:00
if (!(lv->vg->fid->fmt->features & FMT_TAGS)) {
log_error("Logical volume %s/%s does not support tags",
lv->vg->name, lv->name);
return 0;
}
if (add_tag) {
Add dm_pool_strdup to allocate memory and copy a tag in {lv|vg}_change_tag() We need to allocate memory for the tag and copy the tag value before we add it to the list of tags. We could put this inside lvm2app since the tools keep their memory around until vg_write/vg_commit is called, but we put it inside the internal library to minimize code in lvm2app. We need to copy the tag passed in by the caller to ensure the lifetime of the memory until the {vg|lv} handle is released. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:57 +03:00
if (!(tag_new = dm_pool_strdup(lv->vg->vgmem, tag))) {
Add log_error when strdup fails in {vg|lv}_change_tag(). Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-07-09 20:57:44 +04:00
log_error("Failed to duplicate tag %s from %s/%s",
tag, lv->vg->name, lv->name);
return 0;
Add dm_pool_strdup to allocate memory and copy a tag in {lv|vg}_change_tag() We need to allocate memory for the tag and copy the tag value before we add it to the list of tags. We could put this inside lvm2app since the tools keep their memory around until vg_write/vg_commit is called, but we put it inside the internal library to minimize code in lvm2app. We need to copy the tag passed in by the caller to ensure the lifetime of the memory until the {vg|lv} handle is released. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:57 +03:00
}
if (!str_list_add(lv->vg->vgmem, &lv->tags, tag_new)) {
Refactor lvchange_tag() to call lv_change_tag() library function. Similar refactoring to vgchange - pull out common parts and put into library function for reuse. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:49 +03:00
log_error("Failed to add tag %s to %s/%s",
tag, lv->vg->name, lv->name);
return 0;
}
Switch to return void List delete cannot fail, so there is no reason to test for error.
2012-02-08 16:52:58 +04:00
} else
str_list_del(&lv->tags, tag);
Refactor lvchange_tag() to call lv_change_tag() library function. Similar refactoring to vgchange - pull out common parts and put into library function for reuse. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:49 +03:00
return 1;
}
Refactor _vgchange_tag() to vg_change_tag() library function. Pull out common code to be called from tools as well as lvm2app. Leave archive() at tool level so we can use from vgcreate as well as vgchange. Should be no functional change. - add stack macro in vgchange Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:05 +03:00
int vg_change_tag(struct volume_group *vg, const char *tag, int add_tag)
{
Add dm_pool_strdup to allocate memory and copy a tag in {lv|vg}_change_tag() We need to allocate memory for the tag and copy the tag value before we add it to the list of tags. We could put this inside lvm2app since the tools keep their memory around until vg_write/vg_commit is called, but we put it inside the internal library to minimize code in lvm2app. We need to copy the tag passed in by the caller to ensure the lifetime of the memory until the {vg|lv} handle is released. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:57 +03:00
char *tag_new;
Refactor _vgchange_tag() to vg_change_tag() library function. Pull out common code to be called from tools as well as lvm2app. Leave archive() at tool level so we can use from vgcreate as well as vgchange. Should be no functional change. - add stack macro in vgchange Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:05 +03:00
if (!(vg->fid->fmt->features & FMT_TAGS)) {
log_error("Volume group %s does not support tags", vg->name);
return 0;
}
if (add_tag) {
Add dm_pool_strdup to allocate memory and copy a tag in {lv|vg}_change_tag() We need to allocate memory for the tag and copy the tag value before we add it to the list of tags. We could put this inside lvm2app since the tools keep their memory around until vg_write/vg_commit is called, but we put it inside the internal library to minimize code in lvm2app. We need to copy the tag passed in by the caller to ensure the lifetime of the memory until the {vg|lv} handle is released. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:57 +03:00
if (!(tag_new = dm_pool_strdup(vg->vgmem, tag))) {
Add log_error when strdup fails in {vg|lv}_change_tag(). Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-07-09 20:57:44 +04:00
log_error("Failed to duplicate tag %s from %s",
tag, vg->name);
return 0;
Add dm_pool_strdup to allocate memory and copy a tag in {lv|vg}_change_tag() We need to allocate memory for the tag and copy the tag value before we add it to the list of tags. We could put this inside lvm2app since the tools keep their memory around until vg_write/vg_commit is called, but we put it inside the internal library to minimize code in lvm2app. We need to copy the tag passed in by the caller to ensure the lifetime of the memory until the {vg|lv} handle is released. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:57 +03:00
}
if (!str_list_add(vg->vgmem, &vg->tags, tag_new)) {
Refactor _vgchange_tag() to vg_change_tag() library function. Pull out common code to be called from tools as well as lvm2app. Leave archive() at tool level so we can use from vgcreate as well as vgchange. Should be no functional change. - add stack macro in vgchange Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:05 +03:00
log_error("Failed to add tag %s to volume group %s",
tag, vg->name);
return 0;
}
Switch to return void List delete cannot fail, so there is no reason to test for error.
2012-02-08 16:52:58 +04:00
} else
str_list_del(&vg->tags, tag);
Refactor _vgchange_tag() to vg_change_tag() library function. Pull out common code to be called from tools as well as lvm2app. Leave archive() at tool level so we can use from vgcreate as well as vgchange. Should be no functional change. - add stack macro in vgchange Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-02-24 21:15:05 +03:00
return 1;
}
iospace restructured
2001-11-12 18:10:01 +03:00
const char *strip_dir(const char *vg_name, const char *dev_dir)
o Split struct io_space into: struct format_handler - format methods struct format_instance - links instance data, methods, and cmd struct cmd_context - dev_dir, memory allocator, device filter
2001-11-12 15:16:57 +03:00
{
Default stripesize 64k & config file setting for it; Clear many compiler warnings (i386) & associated bugs - hopefully without introducing too many new bugs:-) (Same exercise required for other archs.) Default compilation has optimisation - or else use ./configure --enable-debug
2002-12-20 02:25:55 +03:00
size_t len = strlen(dev_dir);
o Split struct io_space into: struct format_handler - format methods struct format_instance - links instance data, methods, and cmd struct cmd_context - dev_dir, memory allocator, device filter
2001-11-12 15:16:57 +03:00
if (!strncmp(vg_name, dev_dir, len))
vg_name += len;
return vg_name;
}
metadata: validate major, minor numbers Validate major, minor numbers after reading them from metadata.
2014-09-19 16:51:41 +04:00
/*
* Validates major and minor numbers.
* On >2.4 kernel we only support dynamic major number.
*/
int validate_major_minor(const struct cmd_context *cmd,
const struct format_type *fmt,
int32_t major, int32_t minor)
{
int r = 1;
if (!strncmp(cmd->kernel_vsn, "2.4.", 4) ||
(fmt->features & FMT_RESTRICTED_LVIDS)) {
if (major < 0 || major > 255) {
log_error("Major number %d outside range 0-255.", major);
r = 0;
}
if (minor < 0 || minor > 255) {
log_error("Minor number %d outside range 0-255.", minor);
r = 0;
}
} else {
/* 12 bits for major number */
if ((major != -1) &&
(major != cmd->dev_types->device_mapper_major)) {
/* User supplied some major number */
if (major < 0 || major > 4095) {
log_error("Major number %d outside range 0-4095.", major);
r = 0;
} else
log_print_unless_silent("Ignoring supplied major %d number - "
"kernel assigns major numbers dynamically.",
major);
}
/* 20 bits for minor number */
if (minor < 0 || minor > 1048575) {
log_error("Minor number %d outside range 0-1048575.", minor);
r = 0;
}
}
return r;
}
Allow vgcreate options as input to vgsplit when new vg is split destination.
2008-01-15 00:07:58 +03:00
/*
* Validate parameters to vg_create() before calling.
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
* FIXME: Move inside vg_create library function.
* FIXME: Change vgcreate_params struct to individual gets/sets
Allow vgcreate options as input to vgsplit when new vg is split destination.
2008-01-15 00:07:58 +03:00
*/
Rename validate_vg_create_params to vgcreate_params_validate.
2009-11-01 23:05:17 +03:00
int vgcreate_params_validate(struct cmd_context *cmd,
struct vgcreate_params *vp)
Allow vgcreate options as input to vgsplit when new vg is split destination.
2008-01-15 00:07:58 +03:00
{
cleanup: move log_error upward in code stack Report log_error earlier.
2012-10-16 12:20:02 +04:00
if (!validate_new_vg_name(cmd, vp->vg_name))
return_0;
Allow vgcreate options as input to vgsplit when new vg is split destination.
2008-01-15 00:07:58 +03:00
if (vp->alloc == ALLOC_INHERIT) {
log_error("Volume Group allocation policy cannot inherit "
"from anything");
cleanup: swap return values Use lvm standard return code for success/fail 1/0.
2012-10-16 12:07:27 +04:00
return 0;
Allow vgcreate options as input to vgsplit when new vg is split destination.
2008-01-15 00:07:58 +03:00
}
if (!vp->extent_size) {
log_error("Physical extent size may not be zero");
cleanup: swap return values Use lvm standard return code for success/fail 1/0.
2012-10-16 12:07:27 +04:00
return 0;
Allow vgcreate options as input to vgsplit when new vg is split destination.
2008-01-15 00:07:58 +03:00
}
if (!(cmd->fmt->features & FMT_UNLIMITED_VOLS)) {
if (!vp->max_lv)
vp->max_lv = 255;
if (!vp->max_pv)
vp->max_pv = 255;
if (vp->max_lv > 255 || vp->max_pv > 255) {
log_error("Number of volumes may not exceed 255");
cleanup: swap return values Use lvm standard return code for success/fail 1/0.
2012-10-16 12:07:27 +04:00
return 0;
Allow vgcreate options as input to vgsplit when new vg is split destination.
2008-01-15 00:07:58 +03:00
}
}
cleanup: swap return values Use lvm standard return code for success/fail 1/0.
2012-10-16 12:07:27 +04:00
return 1;
Allow vgcreate options as input to vgsplit when new vg is split destination.
2008-01-15 00:07:58 +03:00
}
metadata: Eliminate redundant nested VG metadata Only lv_committed() now uses vg->vg_committed and it appears redundant if its contents match the enclosing VG so don't waste cycles creating it when that's known to be true when no write lock is held so the struct won't get modified.
2017-11-14 18:38:55 +03:00
static void _vg_wipe_cached_precommitted(struct volume_group *vg)
{
release_vg(vg->vg_precommitted);
vg->vg_precommitted = NULL;
}
static void _vg_move_cached_precommitted_to_committed(struct volume_group *vg)
{
release_vg(vg->vg_committed);
vg->vg_committed = vg->vg_precommitted;
vg->vg_precommitted = NULL;
}
metadata: move vg parsing to vg_write Parsing vg structure during supend/commit/resume may require a lot of memory - so move this into vg_write. FIXME: there are now multiple cache layers which our doing some thing multiple times at different levels. Moreover there is now different caching path with and without lvmetad - this should be unified and both path should use same mechanism.
2014-02-22 04:44:21 +04:00
/*
* Update content of precommitted VG
*
* TODO: Optimize in the future, since lvmetad needs similar
* config tree processing in lvmetad_vg_update().
*/
metadata: Eliminate redundant nested VG metadata Only lv_committed() now uses vg->vg_committed and it appears redundant if its contents match the enclosing VG so don't waste cycles creating it when that's known to be true when no write lock is held so the struct won't get modified.
2017-11-14 18:38:55 +03:00
static int _vg_update_embedded_copy(struct volume_group *vg, struct volume_group **vg_embedded)
Add and track an "ondisk" pointer to struct volume_group. This allows us to get the current on-disk version of the metadata whenever we have the current in-flight version, without a recourse to scanning or lvmcache.
2013-03-17 19:27:44 +04:00
{
metadata: Eliminate redundant nested VG metadata Only lv_committed() now uses vg->vg_committed and it appears redundant if its contents match the enclosing VG so don't waste cycles creating it when that's known to be true when no write lock is held so the struct won't get modified.
2017-11-14 18:38:55 +03:00
struct dm_config_tree *cft;
metadata: Remove unused vg.cft_precommitted The precommitted metadata config_tree is now only referenced from a single function so just use a local variable instead.
2017-11-14 04:09:58 +03:00
metadata: Eliminate redundant nested VG metadata Only lv_committed() now uses vg->vg_committed and it appears redundant if its contents match the enclosing VG so don't waste cycles creating it when that's known to be true when no write lock is held so the struct won't get modified.
2017-11-14 18:38:55 +03:00
_vg_wipe_cached_precommitted(vg);
Add and track an "ondisk" pointer to struct volume_group. This allows us to get the current on-disk version of the metadata whenever we have the current in-flight version, without a recourse to scanning or lvmcache.
2013-03-17 19:27:44 +04:00
metadata: Remove unused vg.cft_precommitted The precommitted metadata config_tree is now only referenced from a single function so just use a local variable instead.
2017-11-14 04:09:58 +03:00
/* Copy the VG using an export followed by import */
metadata: Eliminate redundant nested VG metadata Only lv_committed() now uses vg->vg_committed and it appears redundant if its contents match the enclosing VG so don't waste cycles creating it when that's known to be true when no write lock is held so the struct won't get modified.
2017-11-14 18:38:55 +03:00
if (!(cft = export_vg_to_config_tree(vg)))
lvmetad: more reuse precommit buffer This patch moves more allocation to vg_write (as started in 8c878438f5a75310d09a1ee035d9caf8e2b9e50b) TODO: relocate also communication. (in-release update)
2014-03-01 01:40:00 +04:00
return_0;
Add and track an "ondisk" pointer to struct volume_group. This allows us to get the current on-disk version of the metadata whenever we have the current in-flight version, without a recourse to scanning or lvmcache.
2013-03-17 19:27:44 +04:00
metadata: add vg_from_config_tree Add cmd/fmt args to import functions so that they can be used without the fid arg which.
2019-10-29 22:31:37 +03:00
if (!(*vg_embedded = import_vg_from_config_tree(vg->cmd, vg->fid, cft))) {
metadata: Eliminate redundant nested VG metadata Only lv_committed() now uses vg->vg_committed and it appears redundant if its contents match the enclosing VG so don't waste cycles creating it when that's known to be true when no write lock is held so the struct won't get modified.
2017-11-14 18:38:55 +03:00
dm_config_destroy(cft);
lvmetad: more reuse precommit buffer This patch moves more allocation to vg_write (as started in 8c878438f5a75310d09a1ee035d9caf8e2b9e50b) TODO: relocate also communication. (in-release update)
2014-03-01 01:40:00 +04:00
return_0;
}
metadata: Fix a pool CRC failure due to "late" ondisk copy creation.
2013-03-18 18:46:00 +04:00
metadata: Eliminate redundant nested VG metadata Only lv_committed() now uses vg->vg_committed and it appears redundant if its contents match the enclosing VG so don't waste cycles creating it when that's known to be true when no write lock is held so the struct won't get modified.
2017-11-14 18:38:55 +03:00
dm_config_destroy(cft);
metadata: move vg parsing to vg_write Parsing vg structure during supend/commit/resume may require a lot of memory - so move this into vg_write. FIXME: there are now multiple cache layers which our doing some thing multiple times at different levels. Moreover there is now different caching path with and without lvmetad - this should be unified and both path should use same mechanism.
2014-02-22 04:44:21 +04:00
return 1;
Add and track an "ondisk" pointer to struct volume_group. This allows us to get the current on-disk version of the metadata whenever we have the current in-flight version, without a recourse to scanning or lvmcache.
2013-03-17 19:27:44 +04:00
}
Handle metadata with unknown segment types more gracefully.
2009-10-16 21:41:49 +04:00
int lv_has_unknown_segments(const struct logical_volume *lv)
{
struct lv_segment *seg;
/* foreach segment */
dm_list_iterate_items(seg, &lv->segments)
if (seg_unknown(seg))
return 1;
return 0;
}
int vg_has_unknown_segments(const struct volume_group *vg)
{
struct lv_list *lvl;
/* foreach LV */
dm_list_iterate_items(lvl, &vg->lvs)
if (lv_has_unknown_segments(lvl->lv))
return 1;
return 0;
}
Change vg_create() to take only minimal parameters and obtain a lock. vg_t *vg_create(struct cmd_context *cmd, const char *vg_name); This is the first step towards the API called to create a VG. Call vg_lock_newname() inside this function. Use _vg_make_handle() where possible. Now we have 2 ways to construct a volume group: 1) vg_read: Used when constructing an existing VG from disks 2) vg_create: Used when constructing a new VG Both of these interfaces obtain a lock, and return a vg_t *. The usage of _vg_make_handle() inside vg_create() doesn't fit perfectly but it's ok for now. Needs some cleanup though and I've noted "FIXME" in the code. Add the new vg_create() plus vg 'set' functions for non-default VG parameters in the following tools: - vgcreate: Fairly straightforward refactoring. We just moved vg_lock_newname inside vg_create so we check the return via vg_read_error. - vgsplit: The refactoring here is a bit more tricky. Originally we called vg_lock_newname and depending on the error code, we either read the existing vg or created the new one. Now vg_create() calls vg_lock_newname, so we first try to create the VG. If this fails with FAILED_EXIST, we can then do the vg_read. If the create succeeds, we check the input parameters and set any new values on the VG. TODO in future patches: 1. The VG_ORPHAN lock needs some thought. We may want to treat this as any other VG, and require the application to obtain a handle and pass it to other API calls (for example, vg_extend). Or, we may find that hiding the VG_ORPHAN lock inside other APIs is the way to go. I thought of placing the VG_ORPHAN lock inside vg_create() and tying it to the vg handle, but was not certain this was the right approach. 2. Cleanup error paths. Integrate vg_read_error() with vg_create and vg_read* error codes and/or the new error APIs. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2009-07-09 14:09:33 +04:00
/*
* Create a VG with default parameters.
*/
Remove pv_t, vg_t & lv_t handles from lib. Only liblvm uses them. Rename lvm.h to lvm2app.h for now.
2009-07-29 17:26:01 +04:00
struct volume_group *vg_create(struct cmd_context *cmd, const char *vg_name)
o vgcreate
2001-10-12 18:25:53 +04:00
{
Remove pv_t, vg_t & lv_t handles from lib. Only liblvm uses them. Rename lvm.h to lvm2app.h for now.
2009-07-29 17:26:01 +04:00
struct volume_group *vg;
cleanup: use struct assign Simplier code with struct assign. Drop unneeded zeroing of zallocated memory.
2013-06-16 23:07:39 +04:00
struct format_instance_ctx fic = {
.type = FMT_INSTANCE_MDAS | FMT_INSTANCE_AUX_MDAS,
.context.vg_ref.vg_name = vg_name
};
Use only vg_set_fid and new pv_set_fid fn to assign the format instance. This is essential for proper format instance ref_count support. We must use these functions to set the fid everywhere from now on, even the NULL value!
2011-03-11 17:50:13 +03:00
struct format_instance *fid;
Introduce memory pool per volume group. Since now, all code reading volume group is responsible for releasing the memory allocated by calling vg_release(vg). (For simplicity of use, vg_releae can be called for vg == NULL, the same logic like free(NULL)). Also providing simple macro for unlocking & releasing in one step, tools usualy uses this approach. The global memory pool (cmd->mem) should be used only for global physical volume operations. This patch have to be applied with all subsequent patches to complete memory pool per vg logic. Using separate memory pool has quite bit memory saving impact when using large VGs, this is mainly needed when we have to use preallocated and locked memory (and should not overflow from that memory space).
2009-04-10 13:59:18 +04:00
Refactor vg allocation code Create new function alloc_vg() to allocate VG structure. It takes pool_name (for easier debugging). and also take vg_name to futher simplify code. Move remainder of _build_vg_from_pds to _pool_vg_read and use vg memory pool for import functions. (it's been using smem -> fid mempool -> cmd mempool) (FIXME: remove mempool parameter for import functions and use vg). Move remainder of the _build_vg to _format1_vg_read
2011-03-10 15:43:29 +03:00
if (!(vg = alloc_vg("vg_create", cmd, vg_name)))
Introduce memory pool per volume group. Since now, all code reading volume group is responsible for releasing the memory allocated by calling vg_release(vg). (For simplicity of use, vg_releae can be called for vg == NULL, the same logic like free(NULL)). Also providing simple macro for unlocking & releasing in one step, tools usualy uses this approach. The global memory pool (cmd->mem) should be used only for global physical volume operations. This patch have to be applied with all subsequent patches to complete memory pool per vg logic. Using separate memory pool has quite bit memory saving impact when using large VGs, this is mainly needed when we have to use preallocated and locked memory (and should not overflow from that memory space).
2009-04-10 13:59:18 +04:00
goto_bad;
o vgcreate
2001-10-12 18:25:53 +04:00
if (!id_create(&vg->id)) {
Use log_error macro consistently throughout in place of log_err.
2009-07-16 00:02:46 +04:00
log_error("Couldn't create uuid for volume group '%s'.",
vg_name);
o vgcreate
2001-10-12 18:25:53 +04:00
goto bad;
}
o ACTIVE is no longer a status flag - lv_active() used to check if an LV is active in the device-mapper. o Many operations can be carried out regardless of whether the VG is active or not. o vgscan does not activate anything - use vgchange. o Change lvrename to support renaming of active LVs. o Remove '//' appearing in some pathnames. o Dummy lv_check_segments() for compilation.
2002-01-11 02:21:07 +03:00
vg->status = (RESIZEABLE_VG | LVM_READ | LVM_WRITE);
lvm1: Reenable sys ID. Move the lvm1 sys ID into vg->lvm1_system_id and reenable the #if 0 LVM1 code. Still display the new-style system ID in the same reporting field, though, as only one can be set. Add a format feature flag FMT_SYSTEM_ON_PVS for LVM1 and disallow access to LVM1 VGs if a new-style system ID has been set. Treat the new vg->system_id as const.
2015-02-24 02:03:52 +03:00
vg->system_id = NULL;
Add devices/preferred_names config regex list for displayed device names. Free a temporary dir string in fcntl_lock_file() after use. Fix a dm_pool_destroy() in matcher_create(). Introduce goto_bad macro.
2007-04-26 20:44:59 +04:00
Change vg_create() to take only minimal parameters and obtain a lock. vg_t *vg_create(struct cmd_context *cmd, const char *vg_name); This is the first step towards the API called to create a VG. Call vg_lock_newname() inside this function. Use _vg_make_handle() where possible. Now we have 2 ways to construct a volume group: 1) vg_read: Used when constructing an existing VG from disks 2) vg_create: Used when constructing a new VG Both of these interfaces obtain a lock, and return a vg_t *. The usage of _vg_make_handle() inside vg_create() doesn't fit perfectly but it's ok for now. Needs some cleanup though and I've noted "FIXME" in the code. Add the new vg_create() plus vg 'set' functions for non-default VG parameters in the following tools: - vgcreate: Fairly straightforward refactoring. We just moved vg_lock_newname inside vg_create so we check the return via vg_read_error. - vgsplit: The refactoring here is a bit more tricky. Originally we called vg_lock_newname and depending on the error code, we either read the existing vg or created the new one. Now vg_create() calls vg_lock_newname, so we first try to create the VG. If this fails with FAILED_EXIST, we can then do the vg_read. If the create succeeds, we check the input parameters and set any new values on the VG. TODO in future patches: 1. The VG_ORPHAN lock needs some thought. We may want to treat this as any other VG, and require the application to obtain a handle and pass it to other API calls (for example, vg_extend). Or, we may find that hiding the VG_ORPHAN lock inside other APIs is the way to go. I thought of placing the VG_ORPHAN lock inside vg_create() and tying it to the vg handle, but was not certain this was the right approach. 2. Cleanup error paths. Integrate vg_read_error() with vg_create and vg_read* error codes and/or the new error APIs. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2009-07-09 14:09:33 +04:00
vg->extent_size = DEFAULT_EXTENT_SIZE * 2;
vg->max_lv = DEFAULT_MAX_LV;
vg->max_pv = DEFAULT_MAX_PV;
vg->alloc = DEFAULT_ALLOC_POLICY;
Add mda_copies to VG structures and initialization. Add a field to struct volume_group to later implement metadata balancing: - mda_copies: target # of non-ignored mdas in the VG; default 0 (do not control pv 'ignore mdas' bit. This patch just adds the parameter to the structures with the default values but does not modify any commands. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:36:37 +04:00
vg->mda_copies = DEFAULT_VGMETADATACOPIES;
Rename allocation policies; add --alloc to cmdline; LV inherits from VG.
2004-05-19 02:12:53 +04:00
Use only vg_set_fid and new pv_set_fid fn to assign the format instance. This is essential for proper format instance ref_count support. We must use these functions to set the fid everywhere from now on, even the NULL value!
2011-03-11 17:50:13 +03:00
if (!(fid = cmd->fmt->ops->create_instance(cmd->fmt, &fic))) {
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
log_error("Failed to create format instance");
goto bad;
}
Use only vg_set_fid and new pv_set_fid fn to assign the format instance. This is essential for proper format instance ref_count support. We must use these functions to set the fid everywhere from now on, even the NULL value!
2011-03-11 17:50:13 +03:00
vg_set_fid(vg, fid);
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
Cope better when format functions are missing.
2003-08-27 01:12:06 +04:00
if (vg->fid->fmt->ops->vg_setup &&
!vg->fid->fmt->ops->vg_setup(vg->fid, vg)) {
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
log_error("Format specific setup of volume group '%s' failed.",
vg_name);
o vgcreate
2001-10-12 18:25:53 +04:00
goto bad;
}
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
return vg;
o vgcreate
2001-10-12 18:25:53 +04:00
Change vg_create() to take only minimal parameters and obtain a lock. vg_t *vg_create(struct cmd_context *cmd, const char *vg_name); This is the first step towards the API called to create a VG. Call vg_lock_newname() inside this function. Use _vg_make_handle() where possible. Now we have 2 ways to construct a volume group: 1) vg_read: Used when constructing an existing VG from disks 2) vg_create: Used when constructing a new VG Both of these interfaces obtain a lock, and return a vg_t *. The usage of _vg_make_handle() inside vg_create() doesn't fit perfectly but it's ok for now. Needs some cleanup though and I've noted "FIXME" in the code. Add the new vg_create() plus vg 'set' functions for non-default VG parameters in the following tools: - vgcreate: Fairly straightforward refactoring. We just moved vg_lock_newname inside vg_create so we check the return via vg_read_error. - vgsplit: The refactoring here is a bit more tricky. Originally we called vg_lock_newname and depending on the error code, we either read the existing vg or created the new one. Now vg_create() calls vg_lock_newname, so we first try to create the VG. If this fails with FAILED_EXIST, we can then do the vg_read. If the create succeeds, we check the input parameters and set any new values on the VG. TODO in future patches: 1. The VG_ORPHAN lock needs some thought. We may want to treat this as any other VG, and require the application to obtain a handle and pass it to other API calls (for example, vg_extend). Or, we may find that hiding the VG_ORPHAN lock inside other APIs is the way to go. I thought of placing the VG_ORPHAN lock inside vg_create() and tying it to the vg handle, but was not certain this was the right approach. 2. Cleanup error paths. Integrate vg_read_error() with vg_create and vg_read* error codes and/or the new error APIs. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2009-07-09 14:09:33 +04:00
bad:
Replace free_vg with release_vg Move the free_vg() to vg.c and replace free_vg with release_vg and make the _free_vg internal. Patch is needed for sharing VG in vginfo cache so the release_vg function name is a better fit here.
2011-08-11 00:25:29 +04:00
unlock_and_release_vg(cmd, vg, vg_name);
o vgcreate
2001-10-12 18:25:53 +04:00
return NULL;
}
cleanup: extents_from_size is 32bit value Current lvm2 does support only 32bit extents count.
2014-10-30 13:38:49 +03:00
/* Rounds up by default */
uint32_t extents_from_size(struct cmd_context *cmd, uint64_t size,
Move extents_from_size from lvcreate into internal library so we can reuse. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-07-26 06:34:09 +04:00
uint32_t extent_size)
{
if (size % extent_size) {
size += extent_size - size % extent_size;
config: add silent mode Accept -q as the short form of --quiet. Suppress non-essential standard output if -q is given twice. Treat log/silent in lvm.conf as equivalent to -qq. Review all log_print messages and change some to log_print_unless_silent. When silent, the following commands still produce output: dumpconfig, lvdisplay, lvmdiskscan, lvs, pvck, pvdisplay, pvs, version, vgcfgrestore -l, vgdisplay, vgs. [Needs checking.] Non-essential messages are shifted from log level 4 to log level 5 for syslog and lvm2_log_fn purposes.
2012-08-25 23:35:48 +04:00
log_print_unless_silent("Rounding up size to full physical extent %s",
display_size(cmd, size));
Move extents_from_size from lvcreate into internal library so we can reuse. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-07-26 06:34:09 +04:00
}
Avoid lvextend to overflow Add extra check to extent_count overflow. Use internal define MAX_EXTENT_COUNT instead UINT32_MAX.
2011-11-05 02:49:53 +04:00
if (size > (uint64_t) MAX_EXTENT_COUNT * extent_size) {
Move extents_from_size from lvcreate into internal library so we can reuse. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-07-26 06:34:09 +04:00
log_error("Volume too large (%s) for extent size %s. "
cleanup: fix grammar in output - less then -> less than This minor patch fixes grammar in a few messages which get printed to users. It also fixes the same grammar mistake in several comments. Signed-off-by: Rick Elrod <relrod@redhat.com> --
2018-05-17 00:53:38 +03:00
"Upper limit is less than %s.",
Move extents_from_size from lvcreate into internal library so we can reuse. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-07-26 06:34:09 +04:00
display_size(cmd, size),
display_size(cmd, (uint64_t) extent_size),
Avoid lvextend to overflow Add extra check to extent_count overflow. Use internal define MAX_EXTENT_COUNT instead UINT32_MAX.
2011-11-05 02:49:53 +04:00
display_size(cmd, (uint64_t) MAX_EXTENT_COUNT *
Move extents_from_size from lvcreate into internal library so we can reuse. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-07-26 06:34:09 +04:00
extent_size));
return 0;
}
cleanup: extents_from_size is 32bit value Current lvm2 does support only 32bit extents count.
2014-10-30 13:38:49 +03:00
return (uint32_t) (size / extent_size);
Move extents_from_size from lvcreate into internal library so we can reuse. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-07-26 06:34:09 +04:00
}
metadata: add extents_from_percent_size Add API call to calculate extents from percentage value. Size is based in DM_PERCENT_1 units. (Supporting decimal point number). This commit is preparing functionality for more global usage of % with i.e. --size option.
2014-10-31 01:43:12 +03:00
/*
* Converts size according to percentage with specified rounding to extents
*
* For PERCENT_NONE size is in standard sector units.
* For all other percent type is in DM_PERCENT_1 base unit (supports decimal point)
*
* Return value of 0 extents is an error.
*/
uint32_t extents_from_percent_size(struct volume_group *vg, const struct dm_list *pvh,
uint32_t extents, int roundup,
percent_type_t percent, uint64_t size)
{
uint32_t count;
switch (percent) {
case PERCENT_NONE:
if (!roundup && (size % vg->extent_size)) {
if (!(size -= size % vg->extent_size)) {
log_error("Specified size is smaller then physical extent boundary.");
return 0;
}
log_print_unless_silent("Rounding size to boundary between physical extents: %s.",
display_size(vg->cmd, size));
}
return extents_from_size(vg->cmd, size, vg->extent_size);
case PERCENT_LV:
break; /* Base extents already passed in. */
case PERCENT_VG:
extents = vg->extent_count;
break;
case PERCENT_PVS:
if (pvh != &vg->pvs) {
/* Physical volumes are specified on cmdline */
if (!(extents = pv_list_extents_free(pvh))) {
log_error("No free extents in the list of physical volumes.");
return 0;
}
break;
}
cleanup: use fall through gcc gets 'selective' on having commented fall through case.
2017-02-12 20:17:07 +03:00
/* fall through to use all PVs in VG like %FREE */
metadata: add extents_from_percent_size Add API call to calculate extents from percentage value. Size is based in DM_PERCENT_1 units. (Supporting decimal point number). This commit is preparing functionality for more global usage of % with i.e. --size option.
2014-10-31 01:43:12 +03:00
case PERCENT_FREE:
if (!(extents = vg->free_count)) {
log_error("No free extents in Volume group %s.", vg->name);
return 0;
}
break;
default:
log_error(INTERNAL_ERROR "Unsupported percent type %u.", percent);
return 0;
}
if (!(count = percent_of_extents(size, extents, roundup)))
cleanup: use display_percent Replace occurence of %.2f with call of display_percent function.
2017-06-24 17:22:36 +03:00
log_error("Converted %s%%%s into 0 extents.",
display_percent(vg->cmd, size), get_percent_string(percent));
metadata: add extents_from_percent_size Add API call to calculate extents from percentage value. Size is based in DM_PERCENT_1 units. (Supporting decimal point number). This commit is preparing functionality for more global usage of % with i.e. --size option.
2014-10-31 01:43:12 +03:00
else
cleanup: use display_percent Replace occurence of %.2f with call of display_percent function.
2017-06-24 17:22:36 +03:00
log_verbose("Converted %s%%%s into %" PRIu32 " extents.",
display_percent(vg->cmd, size), get_percent_string(percent), count);
metadata: add extents_from_percent_size Add API call to calculate extents from percentage value. Size is based in DM_PERCENT_1 units. (Supporting decimal point number). This commit is preparing functionality for more global usage of % with i.e. --size option.
2014-10-31 01:43:12 +03:00
return count;
}
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
static dm_bitset_t _bitset_with_random_bits(struct dm_pool *mem, uint32_t num_bits,
uint32_t num_set_bits, unsigned *seed)
{
dm_bitset_t bs;
unsigned bit_selected;
char buf[32];
uint32_t i = num_bits - num_set_bits;
tidy: drop unneeded cast Avoid casting to the same type.
2017-07-20 10:57:09 +03:00
if (!(bs = dm_bitset_create(mem, num_bits))) {
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
log_error("Failed to allocate bitset for setting random bits.");
return NULL;
}
if (!dm_pool_begin_object(mem, 512)) {
log_error("dm_pool_begin_object failed for random list of bits.");
dm_pool_free(mem, bs);
return NULL;
}
/* Perform loop num_set_bits times, selecting one bit each time */
while (i++ < num_bits) {
/* Select a random bit between 0 and (i-1) inclusive. */
lib: Share lvm_even_rand for random numbers.
2014-04-04 04:26:19 +04:00
bit_selected = lvm_even_rand(seed, i);
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
/*
* If the bit was already set, set the new bit that became
* choosable for the first time during this pass.
* This maintains a uniform probability distribution by compensating
* for being unable to select it until this pass.
*/
if (dm_bit(bs, bit_selected))
bit_selected = i - 1;
dm_bit_set(bs, bit_selected);
if (dm_snprintf(buf, sizeof(buf), "%u ", bit_selected) < 0) {
log_error("snprintf random bit failed.");
dm_pool_free(mem, bs);
return NULL;
}
if (!dm_pool_grow_object(mem, buf, strlen(buf))) {
log_error("Failed to generate list of random bits.");
dm_pool_free(mem, bs);
return NULL;
}
}
Add missing \0 for grown debug object Attach \0 for proper char* display - otherwise somewhat random message could be displayed in debug more and read of unpredictable read of uninitilized memory values could happen.
2011-03-14 20:00:57 +03:00
if (!dm_pool_grow_object(mem, "\0", 1)) {
log_error("Failed to finish list of random bits.");
dm_pool_free(mem, bs);
return NULL;
}
logging: classify log_debug messages Place most log_debug() messages into a class.
2013-01-08 02:30:29 +04:00
log_debug_metadata("Selected %" PRIu32 " random bits from %" PRIu32 ": %s", num_set_bits, num_bits, (char *) dm_pool_end_object(mem));
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
return bs;
}
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
static int _vg_ignore_mdas(struct volume_group *vg, uint32_t num_to_ignore)
{
struct metadata_area *mda;
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
uint32_t mda_used_count = vg_mda_used_count(vg);
dm_bitset_t mda_to_ignore_bs;
int r = 1;
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
logging: classify log_debug messages Place most log_debug() messages into a class.
2013-01-08 02:30:29 +04:00
log_debug_metadata("Adjusting ignored mdas for %s: %" PRIu32 " of %" PRIu32 " mdas in use "
"but %" PRIu32 " required. Changing %" PRIu32 " mda.",
vg->name, mda_used_count, vg_mda_count(vg), vg_mda_copies(vg), num_to_ignore);
Improve various log messages.
2010-06-30 17:51:11 +04:00
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
if (!num_to_ignore)
return 1;
Improve various log messages.
2010-06-30 17:51:11 +04:00
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
if (!(mda_to_ignore_bs = _bitset_with_random_bits(vg->vgmem, mda_used_count,
num_to_ignore, &vg->cmd->rand_seed)))
return_0;
more mda ignore cleanups
2010-06-30 23:28:35 +04:00
dm_list_iterate_items(mda, &vg->fid->metadata_areas_in_use)
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
if (!mda_is_ignored(mda) && (--mda_used_count,
dm_bit(mda_to_ignore_bs, mda_used_count))) {
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
mda_set_ignored(mda, 1);
more mda ignore cleanups
2010-06-30 23:28:35 +04:00
if (!--num_to_ignore)
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
goto out;
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
}
Improve various log messages.
2010-06-30 17:51:11 +04:00
log_error(INTERNAL_ERROR "Unable to find %"PRIu32" metadata areas to ignore "
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
"on volume group %s", num_to_ignore, vg->name);
Improve various log messages.
2010-06-30 17:51:11 +04:00
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
r = 0;
out:
dm_pool_free(vg->vgmem, mda_to_ignore_bs);
return r;
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
}
static int _vg_unignore_mdas(struct volume_group *vg, uint32_t num_to_unignore)
{
struct metadata_area *mda, *tmda;
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
uint32_t mda_used_count = vg_mda_used_count(vg);
uint32_t mda_count = vg_mda_count(vg);
uint32_t mda_free_count = mda_count - mda_used_count;
dm_bitset_t mda_to_unignore_bs;
int r = 1;
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
if (!num_to_unignore)
return 1;
Improve various log messages.
2010-06-30 17:51:11 +04:00
logging: classify log_debug messages Place most log_debug() messages into a class.
2013-01-08 02:30:29 +04:00
log_debug_metadata("Adjusting ignored mdas for %s: %" PRIu32 " of %" PRIu32 " mdas in use "
"but %" PRIu32 " required. Changing %" PRIu32 " mda.",
vg->name, mda_used_count, mda_count, vg_mda_copies(vg), num_to_unignore);
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
if (!(mda_to_unignore_bs = _bitset_with_random_bits(vg->vgmem, mda_free_count,
num_to_unignore, &vg->cmd->rand_seed)))
return_0;
Improve various log messages.
2010-06-30 17:51:11 +04:00
more mda ignore cleanups
2010-06-30 23:28:35 +04:00
dm_list_iterate_items_safe(mda, tmda, &vg->fid->metadata_areas_ignored)
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
if (mda_is_ignored(mda) && (--mda_free_count,
dm_bit(mda_to_unignore_bs, mda_free_count))) {
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
mda_set_ignored(mda, 0);
dm_list_move(&vg->fid->metadata_areas_in_use,
&mda->list);
more mda ignore cleanups
2010-06-30 23:28:35 +04:00
if (!--num_to_unignore)
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
goto out;
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
}
Improve various log messages.
2010-06-30 17:51:11 +04:00
more mda ignore cleanups
2010-06-30 23:28:35 +04:00
dm_list_iterate_items(mda, &vg->fid->metadata_areas_in_use)
fix code in 2nd mda unignore loop to match 1st loop
2010-07-07 00:09:38 +04:00
if (mda_is_ignored(mda) && (--mda_free_count,
dm_bit(mda_to_unignore_bs, mda_free_count))) {
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
mda_set_ignored(mda, 0);
more mda ignore cleanups
2010-06-30 23:28:35 +04:00
if (!--num_to_unignore)
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
goto out;
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
}
Improve various log messages.
2010-06-30 17:51:11 +04:00
log_error(INTERNAL_ERROR "Unable to find %"PRIu32" metadata areas to unignore "
"on volume group %s", num_to_unignore, vg->name);
Randomly select which mdas to use or ignore. Add some missing standard configure.in checks.
2010-07-06 02:23:15 +04:00
r = 0;
out:
dm_pool_free(vg->vgmem, mda_to_unignore_bs);
return r;
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
}
static int _vg_adjust_ignored_mdas(struct volume_group *vg)
{
more mda ignore cleanups
2010-06-30 23:28:35 +04:00
uint32_t mda_copies_used = vg_mda_used_count(vg);
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
more mda ignore cleanups
2010-06-30 23:28:35 +04:00
if (vg->mda_copies == VGMETADATACOPIES_UNMANAGED) {
/* Ensure at least one mda is in use. */
if (!mda_copies_used && vg_mda_count(vg) && !_vg_unignore_mdas(vg, 1))
return_0;
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
else
more mda ignore cleanups
2010-06-30 23:28:35 +04:00
return 1;
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
}
Improve various log messages.
2010-06-30 17:51:11 +04:00
more mda ignore cleanups
2010-06-30 23:28:35 +04:00
/* Not an error to have vg_mda_count larger than total mdas. */
if (vg->mda_copies == VGMETADATACOPIES_ALL ||
vg->mda_copies >= vg_mda_count(vg)) {
/* Use all */
if (!_vg_unignore_mdas(vg, vg_mda_count(vg) - mda_copies_used))
return_0;
} else if (mda_copies_used < vg->mda_copies) {
if (!_vg_unignore_mdas(vg, vg->mda_copies - mda_copies_used))
return_0;
} else if (mda_copies_used > vg->mda_copies)
if (!_vg_ignore_mdas(vg, mda_copies_used - vg->mda_copies))
return_0;
Allow 'all' and 'unmanaged' values for --vgmetadatacopies. Allowing an 'all' and 'unmanaged' value is more intuitive, and provides a simple way for users to get back to original LVM behavior of metadata written to all PVs in the volume group. If the user requests "--vgmetadatacopies unmanaged", this instructs LVM not to manage the ignore bits to achieve a specific number of metadata copies in the volume group. The user is free to use "pvchange --metadataignore" to control the mdas on a per-PV basis. If the user requests "--vgmetadatacopies all", this instructs LVM to do 2 things: 1) clear all ignore bits, and 2) set the "unmanaged" policy going forward. Internally, we use the special MAX_UINT32 value to indicate 'all'. This 'just' works since it's the largest value possible for the field and so all 'ignore' bits on all mdas in the VG will get cleared inside _vg_metadata_balance(). However, after we've called the _vg_metadata_balance function, we check for the special 'all' value, and if set, we write the "unmanaged" value into the metadata. As such, the 'all' value is never written to disk. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:40:01 +04:00
/*
* The VGMETADATACOPIES_ALL value will never be written disk.
* It is a special cmdline value that means 2 things:
* 1. clear all ignore bits in all mdas in this vg
* 2. set the "unmanaged" policy going forward for metadata balancing
*/
if (vg->mda_copies == VGMETADATACOPIES_ALL)
vg->mda_copies = VGMETADATACOPIES_UNMANAGED;
Improve various log messages.
2010-06-30 17:51:11 +04:00
return 1;
Implement _vg_adjust_ignored_mdas and call from vg_write() path. Compare the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of in-use mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:37:54 +04:00
}
Add vg_mda_size and vg_mda_free functions. Add supporting functions to get vg_mda_size and vg_mda_free fields. Should be no functional change.
2010-08-20 16:43:49 +04:00
uint64_t find_min_mda_size(struct dm_list *mdas)
{
uint64_t min_mda_size = UINT64_MAX, mda_size;
struct metadata_area *mda;
dm_list_iterate_items(mda, mdas) {
if (!mda->ops->mda_total_sectors)
continue;
mda_size = mda->ops->mda_total_sectors(mda);
if (mda_size < min_mda_size)
min_mda_size = mda_size;
}
if (min_mda_size == UINT64_MAX)
min_mda_size = UINT64_C(0);
return min_mda_size;
}
Update check in vg_split_mdas to account for ignored mdas list. The check in vg_split_mdas will trigger an error if the 'from' vg list is empty. However, this might be ok in some instances now that we have ignored mdas. Relax this check so an error is triggered only in the case where there's truly no more mdas in the 'from' vg. One example of where this makes a difference is with vgreduce. If we try to vgreduce a PV with un-ignored mdas, this should trigger the balancing function to un-ignore mdas on another PV in the VG. However, we don't get to vg_write() before we fail because this list size check fails, and we see an error message indicating: "Cannot remove final metadata area ..." Another example is with vgsplit into a new VG, where the PVs being moved contain all ignored mdas. We must move the mdas on fid->metadata_areas_ignored from 'vg_from' to 'vg_to'. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:38:56 +04:00
static int _move_mdas(struct volume_group *vg_from, struct volume_group *vg_to,
struct dm_list *mdas_from, struct dm_list *mdas_to)
Fix vgsplit for lvm1 format (set and validate VG name in PVs metadata). Split metadata areas in vgsplit properly.
2007-03-23 15:43:17 +03:00
{
struct metadata_area *mda, *mda2;
int common_mda = 0;
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items_safe(mda, mda2, mdas_from) {
Fix vgsplit for lvm1 format (set and validate VG name in PVs metadata). Split metadata areas in vgsplit properly.
2007-03-23 15:43:17 +03:00
if (!mda->ops->mda_in_vg) {
common_mda = 1;
continue;
}
Fix vgreduce to use vg_split_mdas to check sufficient mdas remain. Add (empty) orphan VGs to lvmcache during initialisation. Fix orphan VG name used for format_pool.
2008-04-08 16:49:21 +04:00
if (!mda->ops->mda_in_vg(vg_from->fid, vg_from, mda)) {
if (is_orphan_vg(vg_to->name))
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_del(&mda->list);
Fix vgreduce to use vg_split_mdas to check sufficient mdas remain. Add (empty) orphan VGs to lvmcache during initialisation. Fix orphan VG name used for format_pool.
2008-04-08 16:49:21 +04:00
else
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_move(mdas_to, &mda->list);
Fix vgreduce to use vg_split_mdas to check sufficient mdas remain. Add (empty) orphan VGs to lvmcache during initialisation. Fix orphan VG name used for format_pool.
2008-04-08 16:49:21 +04:00
}
Fix vgsplit for lvm1 format (set and validate VG name in PVs metadata). Split metadata areas in vgsplit properly.
2007-03-23 15:43:17 +03:00
}
Update check in vg_split_mdas to account for ignored mdas list. The check in vg_split_mdas will trigger an error if the 'from' vg list is empty. However, this might be ok in some instances now that we have ignored mdas. Relax this check so an error is triggered only in the case where there's truly no more mdas in the 'from' vg. One example of where this makes a difference is with vgreduce. If we try to vgreduce a PV with un-ignored mdas, this should trigger the balancing function to un-ignore mdas on another PV in the VG. However, we don't get to vg_write() before we fail because this list size check fails, and we see an error message indicating: "Cannot remove final metadata area ..." Another example is with vgsplit into a new VG, where the PVs being moved contain all ignored mdas. We must move the mdas on fid->metadata_areas_ignored from 'vg_from' to 'vg_to'. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:38:56 +04:00
return common_mda;
}
/*
* Separate metadata areas after splitting a VG.
* Also accepts orphan VG as destination (for vgreduce).
*/
Use __attribute__ consistently throughout.
2010-07-09 19:34:40 +04:00
int vg_split_mdas(struct cmd_context *cmd __attribute__((unused)),
Update check in vg_split_mdas to account for ignored mdas list. The check in vg_split_mdas will trigger an error if the 'from' vg list is empty. However, this might be ok in some instances now that we have ignored mdas. Relax this check so an error is triggered only in the case where there's truly no more mdas in the 'from' vg. One example of where this makes a difference is with vgreduce. If we try to vgreduce a PV with un-ignored mdas, this should trigger the balancing function to un-ignore mdas on another PV in the VG. However, we don't get to vg_write() before we fail because this list size check fails, and we see an error message indicating: "Cannot remove final metadata area ..." Another example is with vgsplit into a new VG, where the PVs being moved contain all ignored mdas. We must move the mdas on fid->metadata_areas_ignored from 'vg_from' to 'vg_to'. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:38:56 +04:00
struct volume_group *vg_from, struct volume_group *vg_to)
{
struct dm_list *mdas_from_in_use, *mdas_to_in_use;
struct dm_list *mdas_from_ignored, *mdas_to_ignored;
int common_mda = 0;
Fix vgsplit for lvm1 format (set and validate VG name in PVs metadata). Split metadata areas in vgsplit properly.
2007-03-23 15:43:17 +03:00
Update check in vg_split_mdas to account for ignored mdas list. The check in vg_split_mdas will trigger an error if the 'from' vg list is empty. However, this might be ok in some instances now that we have ignored mdas. Relax this check so an error is triggered only in the case where there's truly no more mdas in the 'from' vg. One example of where this makes a difference is with vgreduce. If we try to vgreduce a PV with un-ignored mdas, this should trigger the balancing function to un-ignore mdas on another PV in the VG. However, we don't get to vg_write() before we fail because this list size check fails, and we see an error message indicating: "Cannot remove final metadata area ..." Another example is with vgsplit into a new VG, where the PVs being moved contain all ignored mdas. We must move the mdas on fid->metadata_areas_ignored from 'vg_from' to 'vg_to'. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:38:56 +04:00
mdas_from_in_use = &vg_from->fid->metadata_areas_in_use;
mdas_from_ignored = &vg_from->fid->metadata_areas_ignored;
mdas_to_in_use = &vg_to->fid->metadata_areas_in_use;
mdas_to_ignored = &vg_to->fid->metadata_areas_ignored;
common_mda = _move_mdas(vg_from, vg_to,
mdas_from_in_use, mdas_to_in_use);
common_mda = _move_mdas(vg_from, vg_to,
mdas_from_ignored, mdas_to_ignored);
if ((dm_list_empty(mdas_from_in_use) &&
dm_list_empty(mdas_from_ignored)) ||
((!is_orphan_vg(vg_to->name) &&
dm_list_empty(mdas_to_in_use) &&
dm_list_empty(mdas_to_ignored))))
Fix vgsplit for lvm1 format (set and validate VG name in PVs metadata). Split metadata areas in vgsplit properly.
2007-03-23 15:43:17 +03:00
return common_mda;
return 1;
}
replace pvcreate_params with pvcreate_each_params "pvcreate_each_params" was a temporary name used to transition from the old "pvcreate_params". Remove the old pvcreate_params struct and rename the new pvcreate_each_params struct to pvcreate_params. Rename various pvcreate_each_params terms to simply pvcreate_params.
2016-02-19 00:38:23 +03:00
void pvcreate_params_set_defaults(struct pvcreate_params *pp)
{
memset(pp, 0, sizeof(*pp));
pp->zero = 1;
pp->force = PROMPT;
pp->yes = 0;
pp->restorefile = NULL;
pp->uuid_str = NULL;
pp->pva.size = 0;
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
pp->pva.data_alignment = 0;
pp->pva.data_alignment_offset = 0;
replace pvcreate_params with pvcreate_each_params "pvcreate_each_params" was a temporary name used to transition from the old "pvcreate_params". Remove the old pvcreate_params struct and rename the new pvcreate_each_params struct to pvcreate_params. Rename various pvcreate_each_params terms to simply pvcreate_params.
2016-02-19 00:38:23 +03:00
pp->pva.pvmetadatacopies = DEFAULT_PVMETADATACOPIES;
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
pp->pva.pvmetadatasize = get_default_pvmetadatasize_sectors();
replace pvcreate_params with pvcreate_each_params "pvcreate_each_params" was a temporary name used to transition from the old "pvcreate_params". Remove the old pvcreate_params struct and rename the new pvcreate_each_params struct to pvcreate_params. Rename various pvcreate_each_params terms to simply pvcreate_params.
2016-02-19 00:38:23 +03:00
pp->pva.label_sector = DEFAULT_LABELSECTOR;
pp->pva.metadataignore = DEFAULT_PVMETADATAIGNORE;
pp->pva.ba_start = 0;
pp->pva.ba_size = 0;
pp->pva.pe_start = PV_PE_START_CALC;
pp->pva.extent_count = 0;
pp->pva.extent_size = 0;
dm_list_init(&pp->prompts);
dm_list_init(&pp->arg_devices);
dm_list_init(&pp->arg_process);
dm_list_init(&pp->arg_confirm);
dm_list_init(&pp->arg_create);
dm_list_init(&pp->arg_remove);
dm_list_init(&pp->arg_fail);
dm_list_init(&pp->pvs);
}
Pass struct physical_volume to pe_align.
2008-09-19 08:28:58 +04:00
static struct physical_volume *_alloc_pv(struct dm_pool *mem, struct device *dev)
Add _alloc_pv() and _free_pv() from _pv_create() code and fix error paths. Modified original patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 22:37:19 +04:00
{
Log error reporting for failing _alloc_pv Drop unneeded zeroing of zalloced memory region.
2012-02-13 14:51:52 +04:00
struct physical_volume *pv;
Add _alloc_pv() and _free_pv() from _pv_create() code and fix error paths. Modified original patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 22:37:19 +04:00
Log error reporting for failing _alloc_pv Drop unneeded zeroing of zalloced memory region.
2012-02-13 14:51:52 +04:00
if (!(pv = dm_pool_zalloc(mem, sizeof(*pv)))) {
log_error("Failed to allocate pv structure.");
return NULL;
}
Add _alloc_pv() and _free_pv() from _pv_create() code and fix error paths. Modified original patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 22:37:19 +04:00
Pass struct physical_volume to pe_align.
2008-09-19 08:28:58 +04:00
pv->dev = dev;
Add _alloc_pv() and _free_pv() from _pv_create() code and fix error paths. Modified original patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 22:37:19 +04:00
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_init(&pv->tags);
dm_list_init(&pv->segments);
Add _alloc_pv() and _free_pv() from _pv_create() code and fix error paths. Modified original patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 22:37:19 +04:00
return pv;
}
Remove unneeded pv_create wrapper function. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-07-26 05:52:19 +04:00
/**
* pv_create - initialize a physical volume for use with a volume group
Add free_orphan_vg Move commod code to destroy orphan VG into free_orphan_vg() function. Use orphan vgmem for creation of PV lists. Remove some free_pv_fid() calls (FIXME: check all of them) FIXME: Check whether we could merge release_vg back again for all VGs.
2012-02-13 15:03:59 +04:00
* created PV belongs to Orphan VG.
Remove unneeded pv_create wrapper function. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-07-26 05:52:19 +04:00
*
* Returns:
* PV handle - physical volume initialized successfully
* NULL - invalid parameter or problem initializing the physical volume
*/
redefine pvcreate structs New pv_create_args struct contains all the specific parameters for creating a PV, independent of the command.
2016-02-19 00:31:27 +03:00
Remove unneeded pv_create wrapper function. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-07-26 05:52:19 +04:00
struct physical_volume *pv_create(const struct cmd_context *cmd,
Add wrappers to functions related to pv commands in preparation for exported LVM lib
2007-06-11 22:29:30 +04:00
struct device *dev,
redefine pvcreate structs New pv_create_args struct contains all the specific parameters for creating a PV, independent of the command.
2016-02-19 00:31:27 +03:00
struct pv_create_args *pva)
lvactivate checkpoint commit
2001-09-25 16:49:28 +04:00
{
Remove unneeded pv_create wrapper function. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-07-26 05:52:19 +04:00
const struct format_type *fmt = cmd->fmt;
Add free_orphan_vg Move commod code to destroy orphan VG into free_orphan_vg() function. Use orphan vgmem for creation of PV lists. Remove some free_pv_fid() calls (FIXME: check all of them) FIXME: Check whether we could merge release_vg back again for all VGs.
2012-02-13 15:03:59 +04:00
struct dm_pool *mem = fmt->orphan_vg->vgmem;
Pass struct physical_volume to pe_align.
2008-09-19 08:28:58 +04:00
struct physical_volume *pv = _alloc_pv(mem, dev);
Change the code throughout to use new pv_initialise and modified pv_setup fn. Change pv_create code to work with these changes together with using new pv_add_metadata_area fn to add metadata areas for a PV being created.
2011-02-21 15:24:15 +03:00
unsigned mda_index;
Add free_orphan_vg Move commod code to destroy orphan VG into free_orphan_vg() function. Use orphan vgmem for creation of PV lists. Remove some free_pv_fid() calls (FIXME: check all of them) FIXME: Check whether we could merge release_vg back again for all VGs.
2012-02-13 15:03:59 +04:00
struct pv_list *pvl;
redefine pvcreate structs New pv_create_args struct contains all the specific parameters for creating a PV, independent of the command.
2016-02-19 00:31:27 +03:00
uint64_t size = pva->size;
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
uint64_t data_alignment = pva->data_alignment;
uint64_t data_alignment_offset = pva->data_alignment_offset;
redefine pvcreate structs New pv_create_args struct contains all the specific parameters for creating a PV, independent of the command.
2016-02-19 00:31:27 +03:00
unsigned pvmetadatacopies = pva->pvmetadatacopies;
uint64_t pvmetadatasize = pva->pvmetadatasize;
unsigned metadataignore = pva->metadataignore;
o pvcreate o added uuid unit o stubbed partition stuff
2001-10-12 14:32:06 +04:00
Add _alloc_pv() and _free_pv() from _pv_create() code and fix error paths. Modified original patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 22:37:19 +04:00
if (!pv)
Log error reporting for failing _alloc_pv Drop unneeded zeroing of zalloced memory region.
2012-02-13 14:51:52 +04:00
return_NULL;
o pvcreate o added uuid unit o stubbed partition stuff
2001-10-12 14:32:06 +04:00
redefine pvcreate structs New pv_create_args struct contains all the specific parameters for creating a PV, independent of the command.
2016-02-19 00:31:27 +03:00
if (pva->idp)
memcpy(&pv->id, pva->idp, sizeof(*pva->idp));
Always fail if random id generation fails.
2005-01-20 21:11:53 +03:00
else if (!id_create(&pv->id)) {
log_error("Failed to create random uuid for %s.",
dev_name(dev));
Add _alloc_pv() and _free_pv() from _pv_create() code and fix error paths. Modified original patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 22:37:19 +04:00
goto bad;
Always fail if random id generation fails.
2005-01-20 21:11:53 +03:00
}
o pvcreate --uuid to specify the uuid (required before using vgcfgrestore onto a new device). uuid specified must not already exist on the system. o More message tidying. o When checking for label, only read PV metadata. o Add ataraid. [Needs moving into config/defaults files.]
2002-01-16 21:10:08 +03:00
o Use 'pvcreate --setphysicalvolumesize' with no short form (instead of -s) and add severe warning if it's used to make a device seem bigger than it really is. This is not an option people should be using as it breaks metadata integrity. o Use uint64_t throughout (rather than unsigned long long) o Convert a few messages that contain pathnames into the more common form: pathname: message
2002-02-20 21:29:30 +03:00
if (!dev_get_size(pv->dev, &pv->size)) {
Add pv_dev_name() to access PV device name. Patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 18:29:32 +04:00
log_error("%s: Couldn't get size.", pv_dev_name(pv));
o Use 'pvcreate --setphysicalvolumesize' with no short form (instead of -s) and add severe warning if it's used to make a device seem bigger than it really is. This is not an option people should be using as it breaks metadata integrity. o Use uint64_t throughout (rather than unsigned long long) o Convert a few messages that contain pathnames into the more common form: pathname: message
2002-02-20 21:29:30 +03:00
goto bad;
}
if (size) {
if (size > pv->size)
Make warnings go to stderr. Change log_warn to that effect, log_print continues to send messages to stdout.
2007-06-28 21:33:44 +04:00
log_warn("WARNING: %s: Overriding real size. "
Add pv_dev_name() to access PV device name. Patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 18:29:32 +04:00
"You could lose data.", pv_dev_name(pv));
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
log_verbose("%s: Pretending size is %" PRIu64 " sectors.",
Add pv_dev_name() to access PV device name. Patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 18:29:32 +04:00
pv_dev_name(pv), size);
o Use 'pvcreate --setphysicalvolumesize' with no short form (instead of -s) and add severe warning if it's used to make a device seem bigger than it really is. This is not an option people should be using as it breaks metadata integrity. o Use uint64_t throughout (rather than unsigned long long) o Convert a few messages that contain pathnames into the more common form: pathname: message
2002-02-20 21:29:30 +03:00
pv->size = size;
}
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
Replace PV_MIN_SIZE with function pv_min_size() Add configurable option to define minimal size of of block device usable as a PV. pv_min_size() is added to lvm-globals and it's being initialized through _process_config. Macro PV_MIN_SIZE is unused and removed. New define DEFAULT_PV_MIN_SIZE_KB is added to lvm-global and unlike PV_MIN_SIZE it uses KB units. Should help users with various slow devices attached to the system, which cannot be easily filtered out (like FDD on /dev/sdX): https://bugzilla.redhat.com/show_bug.cgi?id=644578
2011-02-18 17:11:22 +03:00
if (pv->size < pv_min_size()) {
log_error("%s: Size must exceed minimum of %" PRIu64 " sectors.",
pv_dev_name(pv), pv_min_size());
o pvcreate o added uuid unit o stubbed partition stuff
2001-10-12 14:32:06 +04:00
goto bad;
}
pvcreate: fix alignment to incorporate alignment offset if PV has 0 MDAs If zero metadata copies are used, there's no further recalculation of PV alignment that happens when adding metadata areas to the PV and which actually calculates the alignment correctly as a matter of fact. So fix this for "PV without MDA" case as well. Before this patch: [1] raw/~ # pvcreate --dataalignment 8m --dataalignmentoffset 4m --metadatacopies 1 /dev/sda Physical volume "/dev/sda" successfully created [1] raw/~ # pvs -o pv_name,pe_start PV 1st PE /dev/sda 12.00m [1] raw/~ # pvcreate --dataalignment 8m --dataalignmentoffset 4m --metadatacopies 0 /dev/sda Physical volume "/dev/sda" successfully created [1] raw/~ # pvs -o pv_name,pe_start PV 1st PE /dev/sda 8.00m After this patch: [1] raw/~ # pvcreate --dataalignment 8m --dataalignmentoffset 4m --metadatacopies 1 /dev/sda Physical volume "/dev/sda" successfully created [1] raw/~ # pvs -o pv_name,pe_start PV 1st PE /dev/sda 12.00m [1] raw/~ # pvcreate --dataalignment 8m --dataalignmentoffset 4m --metadatacopies 0 /dev/sda Physical volume "/dev/sda" successfully created [1] raw/~ # pvs -o pv_name,pe_start PV 1st PE /dev/sda 12.00m Also, remove a superfluous condition "pv->pe_start < pv->pe_align" in: if (pe_start == PV_PE_START_CALC && pv->pe_start < pv->pe_align) pv->pe_start = pv->pe_align ... This part of the condition is not reachable as with the PV_PE_START_CALC, we always have pv->pe_start set to 0 from the PV struct initialisation (...the pv->pe_start value is just being calculated).
2013-02-21 17:47:49 +04:00
if (pv->size < data_alignment + data_alignment_offset) {
Fix validation of dataalignment value introduced in previous commit.
2009-02-23 19:53:42 +03:00
log_error("%s: Data alignment must not exceed device size.",
pv_dev_name(pv));
goto bad;
}
Keep a global (per-format) orphan_vg and keep any and all orphan PVs linked to it. Avoids the need for FMT_INSTANCE_PV and enables further simplifications. No functional change, internal refactor only.
2012-02-10 06:53:03 +04:00
if (!(pvl = dm_pool_zalloc(mem, sizeof(*pvl)))) {
log_error("pv_list allocation in pv_create failed");
Initialise a new PV-based format instance for a PV that is being created.
2011-02-21 15:12:32 +03:00
goto bad;
}
Keep a global (per-format) orphan_vg and keep any and all orphan PVs linked to it. Avoids the need for FMT_INSTANCE_PV and enables further simplifications. No functional change, internal refactor only.
2012-02-10 06:53:03 +04:00
pvl->pv = pv;
add_pvl_to_vgs(fmt->orphan_vg, pvl);
fmt->orphan_vg->extent_count += pv->pe_count;
fmt->orphan_vg->free_count += pv->pe_count;
Initialise a new PV-based format instance for a PV that is being created.
2011-02-21 15:12:32 +03:00
Some new features.
2002-11-18 17:04:08 +03:00
pv->fmt = fmt;
split orphan VG by format type
2008-02-06 18:47:28 +03:00
pv->vg_name = fmt->orphan_vg_name;
Second path on "pvcreate -s"
2002-02-15 17:33:59 +03:00
Place the first PE at 1 MiB for all defaults . When using default settings, this commit should change nothing. The first PE continues to be placed at 1 MiB resulting in a metadata area size of 1020 KiB (for 4K page sizes; slightly smaller for larger page sizes.) . When default_data_alignment is disabled in lvm.conf, align pe_start at 1 MiB, based on a default metadata area size that adapts to the page size. Previously, disabling this option would result in mda_size that was too small for common use, and produced a 64 KiB aligned pe_start. . Customized pe_start and mda_size values continue to be set as before in lvm.conf and command line. . Remove the configure option for setting default_data_alignment at build time. . Improve alignment related option descriptions. . Add section about alignment to pvcreate man page. Previously, DEFAULT_PVMETADATASIZE was 255 sectors. However, the fact that the config setting named "default_data_alignment" has a default value of 1 (MiB) meant that DEFAULT_PVMETADATASIZE was having no effect. The metadata area size is the space between the start of the metadata area (page size offset from the start of the device) and the first PE (1 MiB by default due to default_data_alignment 1.) The result is a 1020 KiB metadata area on machines with 4KiB page size (1024 KiB - 4 KiB), and smaller on machines with larger page size. If default_data_alignment was set to 0 (disabled), then DEFAULT_PVMETADATASIZE 255 would take effect, and produce a metadata area that was 188 KiB and pe_start of 192 KiB. This was too small for common use. This is fixed by making the default metadata area size a computed value that matches the value produced by default_data_alignment.
2018-11-14 00:00:11 +03:00
/*
* Sets pv: pe_align, pe_align_offset, pe_start, pe_size
* Does not write to device.
*/
redefine pvcreate structs New pv_create_args struct contains all the specific parameters for creating a PV, independent of the command.
2016-02-19 00:31:27 +03:00
if (!fmt->ops->pv_initialise(fmt, pva, pv)) {
Change the code throughout to use new pv_initialise and modified pv_setup fn. Change pv_create code to work with these changes together with using new pv_add_metadata_area fn to add metadata areas for a PV being created.
2011-02-21 15:24:15 +03:00
log_error("Format-specific initialisation of physical "
"volume %s failed.", pv_dev_name(pv));
Second path on "pvcreate -s"
2002-02-15 17:33:59 +03:00
goto bad;
}
Add --dataalignment to pvcreate to specify alignment of data area. (mbroz) This patch is not fully tested and leaves some related bugs unfixed. Intended behaviour of the code now: pe_start in the lvm2 format PV label header is set only by pvcreate (or vgconvert -M2) and then preserved in *all* operations thereafter. In some specialist cases, after the PV is added to a VG, the pe_start field in the VG metadata may hold a different value and if so, it overrides the other one for as long as the PV is in such a VG. Currently, the field storing the size of the data area in the PV label header always holds 0. As it only has meaning in the context of a volume group, it is calculated whenever the PV is added to a VG (and can be derived from extent_size and pe_count in the VG metadata).
2009-02-22 22:00:26 +03:00
Change the code throughout to use new pv_initialise and modified pv_setup fn. Change pv_create code to work with these changes together with using new pv_add_metadata_area fn to add metadata areas for a PV being created.
2011-02-21 15:24:15 +03:00
for (mda_index = 0; mda_index < pvmetadatacopies; mda_index++) {
if (pv->fmt->ops->pv_add_metadata_area &&
!pv->fmt->ops->pv_add_metadata_area(pv->fmt, pv,
redefine pvcreate structs New pv_create_args struct contains all the specific parameters for creating a PV, independent of the command.
2016-02-19 00:31:27 +03:00
pva->pe_start != PV_PE_START_CALC,
Change the code throughout to use new pv_initialise and modified pv_setup fn. Change pv_create code to work with these changes together with using new pv_add_metadata_area fn to add metadata areas for a PV being created.
2011-02-21 15:24:15 +03:00
mda_index, pvmetadatasize,
metadataignore)) {
log_error("Failed to add metadata area for "
"new physical volume %s", pv_dev_name(pv));
goto bad;
}
}
o pvcreate o added uuid unit o stubbed partition stuff
2001-10-12 14:32:06 +04:00
return pv;
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
bad:
Add free_orphan_vg Move commod code to destroy orphan VG into free_orphan_vg() function. Use orphan vgmem for creation of PV lists. Remove some free_pv_fid() calls (FIXME: check all of them) FIXME: Check whether we could merge release_vg back again for all VGs.
2012-02-13 15:03:59 +04:00
// FIXME: detach from orphan in error path
//free_pv_fid(pv);
//dm_pool_free(mem, pv);
o pvcreate o added uuid unit o stubbed partition stuff
2001-10-12 14:32:06 +04:00
return NULL;
lvactivate checkpoint commit
2001-09-25 16:49:28 +04:00
}
Add wrappers to functions related to pv commands in preparation for exported LVM lib
2007-06-11 22:29:30 +04:00
/* FIXME: liblvm todo - make into function that returns handle */
Const cleanups in find_* functions.
2008-03-14 01:51:24 +03:00
struct pv_list *find_pv_in_vg(const struct volume_group *vg,
cleanup: remove superfluous wrappers _find_pv_by_name -> find_pv_by_name _find_pv_in_vg -> find_pv_in_vg _find_pv_in_vg_by_uuid -> find_pv_in_vg_by_uuid The only callers of the underscored variants were their wrappers without the underscore. No other part of the code referenced the underscored variants.
2013-03-19 16:58:02 +04:00
const char *pv_name)
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
{
o Changed find_pv_in_vg, and find_lv_in_vg to return a struct pv_list * and struct lv_list * respectively.
2002-01-21 17:28:12 +03:00
struct pv_list *pvl;
Add cmd arg to more functions so that it can be used in the filter code
2018-06-15 19:03:55 +03:00
struct device *dev = dev_cache_get(vg->cmd, pv_name, vg->cmd->filter);
o use dev_name(dev) to get the name of a device, this operation is cheap since it just get's the first alias.
2001-10-25 18:04:18 +04:00
metadata: Fix find_pv_in_vg for missing PVs/filtered devices.
2014-10-07 18:06:21 +04:00
/*
* If the device does not exist or is filtered out, don't bother trying
* to find it in the list. This also prevents accidentally finding a
* non-NULL PV which happens to be missing (i.e. its pv->dev is NULL)
* for such devices.
*/
if (!dev)
return NULL;
dm_list_iterate_items(pvl, &vg->pvs)
if (pvl->pv->dev == dev)
o Changed find_pv_in_vg, and find_lv_in_vg to return a struct pv_list * and struct lv_list * respectively.
2002-01-21 17:28:12 +03:00
return pvl;
lvactivate checkpoint commit
2001-09-25 16:49:28 +04:00
More vgcreate error trapping
2001-10-15 22:39:40 +04:00
return NULL;
Some new features.
2002-11-18 17:04:08 +03:00
}
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
struct pv_list *find_pv_in_pv_list(const struct dm_list *pl,
Add find_lv_in_lv_list() and find_pv_in_pv_list(). Update _add_pvs() to call find_pv_in_pv_list().
2008-03-28 22:08:23 +03:00
const struct physical_volume *pv)
{
struct pv_list *pvl;
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(pvl, pl)
Add find_lv_in_lv_list() and find_pv_in_pv_list(). Update _add_pvs() to call find_pv_in_pv_list().
2008-03-28 22:08:23 +03:00
if (pvl->pv == pv)
return pvl;
more pre-release cleanup
2008-04-10 23:59:43 +04:00
Add find_lv_in_lv_list() and find_pv_in_pv_list(). Update _add_pvs() to call find_pv_in_pv_list().
2008-03-28 22:08:23 +03:00
return NULL;
}
vgreduce --removemissing to remove missing PVs & deps & make VG consistent
2003-01-18 00:04:26 +03:00
int pv_is_in_vg(struct volume_group *vg, struct physical_volume *pv)
{
Basic support for mirrors.
2005-06-01 20:51:55 +04:00
struct pv_list *pvl;
vgreduce --removemissing to remove missing PVs & deps & make VG consistent
2003-01-18 00:04:26 +03:00
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(pvl, &vg->pvs)
Basic support for mirrors.
2005-06-01 20:51:55 +04:00
if (pv == pvl->pv)
vgreduce --removemissing to remove missing PVs & deps & make VG consistent
2003-01-18 00:04:26 +03:00
return 1;
return 0;
}
Convert find_pv_in_vg_by_uuid and pv_create to use PV handles
2007-06-13 02:41:27 +04:00
/**
* find_pv_in_vg_by_uuid - Find PV in VG by PV UUID
* @vg: volume group to search
* @id: UUID of the PV to match
*
* Returns:
Look up missing PVs by uuid not dev_name in _pvs_single to avoid invalid stat. Make find_pv_in_vg_by_uuid() return same type as related functions.
2010-03-16 18:30:48 +03:00
* struct pv_list within owning struct volume_group - if UUID of PV found in VG
Convert find_pv_in_vg_by_uuid and pv_create to use PV handles
2007-06-13 02:41:27 +04:00
* NULL - invalid parameter or UUID of PV not found in VG
*
* Note
* FIXME - liblvm todo - make into function that takes VG handle
*/
Look up missing PVs by uuid not dev_name in _pvs_single to avoid invalid stat. Make find_pv_in_vg_by_uuid() return same type as related functions.
2010-03-16 18:30:48 +03:00
struct pv_list *find_pv_in_vg_by_uuid(const struct volume_group *vg,
const struct id *id)
Add wrappers to functions related to pv commands in preparation for exported LVM lib
2007-06-11 22:29:30 +04:00
{
cleanup: remove superfluous wrappers _find_pv_by_name -> find_pv_by_name _find_pv_in_vg -> find_pv_in_vg _find_pv_in_vg_by_uuid -> find_pv_in_vg_by_uuid The only callers of the underscored variants were their wrappers without the underscore. No other part of the code referenced the underscored variants.
2013-03-19 16:58:02 +04:00
struct pv_list *pvl;
dm_list_iterate_items(pvl, &vg->pvs)
if (id_equal(&pvl->pv->id, id))
return pvl;
return NULL;
Add wrappers to functions related to pv commands in preparation for exported LVM lib
2007-06-11 22:29:30 +04:00
}
Const cleanups in find_* functions.
2008-03-14 01:51:24 +03:00
struct lv_list *find_lv_in_vg(const struct volume_group *vg,
const char *lv_name)
lvremove
2001-10-29 16:52:23 +03:00
{
o Changed find_pv_in_vg, and find_lv_in_vg to return a struct pv_list * and struct lv_list * respectively.
2002-01-21 17:28:12 +03:00
struct lv_list *lvl;
lvremove
2001-10-29 16:52:23 +03:00
const char *ptr;
/* Use last component */
if ((ptr = strrchr(lv_name, '/')))
ptr++;
else
ptr = lv_name;
o LGPL list implementation
2001-10-31 15:47:01 +03:00
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(lvl, &vg->lvs)
o Similar changes for lv_list.
2002-01-21 19:49:32 +03:00
if (!strcmp(lvl->lv->name, ptr))
o Changed find_pv_in_vg, and find_lv_in_vg to return a struct pv_list * and struct lv_list * respectively.
2002-01-21 17:28:12 +03:00
return lvl;
lvremove
2001-10-29 16:52:23 +03:00
o Added lvextend o Full signed arguments to lvreduce/lvextend o Consistent lv_number/pe map use o Populate pv->pe_allocated o Fixes for allocation/writing of multiple LVs
2001-11-10 01:01:04 +03:00
return NULL;
lvremove
2001-10-29 16:52:23 +03:00
}
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
struct lv_list *find_lv_in_lv_list(const struct dm_list *ll,
Add find_lv_in_lv_list() and find_pv_in_pv_list(). Update _add_pvs() to call find_pv_in_pv_list().
2008-03-28 22:08:23 +03:00
const struct logical_volume *lv)
{
struct lv_list *lvl;
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(lvl, ll)
Add find_lv_in_lv_list() and find_pv_in_pv_list(). Update _add_pvs() to call find_pv_in_pv_list().
2008-03-28 22:08:23 +03:00
if (lvl->lv == lv)
return lvl;
more pre-release cleanup
2008-04-10 23:59:43 +04:00
Add find_lv_in_lv_list() and find_pv_in_pv_list(). Update _add_pvs() to call find_pv_in_pv_list().
2008-03-28 22:08:23 +03:00
return NULL;
}
cleanup: move towards using direct LV pointers We do not won't to 'expose' internals of VG struct. ATM we use lists to keep all LVs - we may want to switch to better struct for quicker 'search'. Since we do not need 'lists' but always actual LV, switch find_lv_in_vg_by_lvid() to return LV, and replaces some use case of find_lv_in_vg() with 'better' working find_lv() which already returns LV.
2015-11-22 01:31:44 +03:00
struct logical_volume *find_lv_in_vg_by_lvid(struct volume_group *vg,
const union lvid *lvid)
Begin conversion so LV id is passed to activation unit instead of struct logical_volume.
2002-02-25 15:56:16 +03:00
{
struct lv_list *lvl;
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(lvl, &vg->lvs)
o Use new LCK_HOLD flag to indicate whether lock should be held on return from lock_vol() - otherwise it now attempts to acquire the lock and then immediately releases it. o Extend the id field in struct logical_volume to hold VG uuid + LV uuid for format1. This unique lvid can be used directly when calling lock_vol(). o Add the VG uuid to vgcache to make VG uuid lookups possible. (Another step towards using them instead of VG names internally.)
2002-03-05 23:03:09 +03:00
if (!strncmp(lvl->lv->lvid.s, lvid->s, sizeof(*lvid)))
cleanup: move towards using direct LV pointers We do not won't to 'expose' internals of VG struct. ATM we use lists to keep all LVs - we may want to switch to better struct for quicker 'search'. Since we do not need 'lists' but always actual LV, switch find_lv_in_vg_by_lvid() to return LV, and replaces some use case of find_lv_in_vg() with 'better' working find_lv() which already returns LV.
2015-11-22 01:31:44 +03:00
return lvl->lv;
Begin conversion so LV id is passed to activation unit instead of struct logical_volume.
2002-02-25 15:56:16 +03:00
return NULL;
}
Const cleanups in find_* functions.
2008-03-14 01:51:24 +03:00
struct logical_volume *find_lv(const struct volume_group *vg,
const char *lv_name)
lvremove
2001-10-29 16:52:23 +03:00
{
o Changed find_pv_in_vg, and find_lv_in_vg to return a struct pv_list * and struct lv_list * respectively.
2002-01-21 17:28:12 +03:00
struct lv_list *lvl = find_lv_in_vg(vg, lv_name);
o Similar changes for lv_list.
2002-01-21 19:49:32 +03:00
return lvl ? lvl->lv : NULL;
lvremove
2001-10-29 16:52:23 +03:00
}
metadata: add find_historical_glv fn The find_historical_glv is helper function that looks up historical LV in struct volume_group's historical_lvs list and returns it if found.
2016-03-01 17:21:21 +03:00
struct generic_logical_volume *find_historical_glv(const struct volume_group *vg,
const char *historical_lv_name,
metadata: add historical_glv_remove
2016-03-01 17:26:57 +03:00
int check_removed_list,
metadata: add find_historical_glv fn The find_historical_glv is helper function that looks up historical LV in struct volume_group's historical_lvs list and returns it if found.
2016-03-01 17:21:21 +03:00
struct glv_list **glvl_found)
{
struct glv_list *glvl;
const char *ptr;
metadata: add historical_glv_remove
2016-03-01 17:26:57 +03:00
const struct dm_list *list = check_removed_list ? &vg->removed_historical_lvs
: &vg->historical_lvs;
metadata: add find_historical_glv fn The find_historical_glv is helper function that looks up historical LV in struct volume_group's historical_lvs list and returns it if found.
2016-03-01 17:21:21 +03:00
/* Use last component */
if ((ptr = strrchr(historical_lv_name, '/')))
ptr++;
else
ptr = historical_lv_name;
metadata: add historical_glv_remove
2016-03-01 17:26:57 +03:00
dm_list_iterate_items(glvl, list) {
metadata: add find_historical_glv fn The find_historical_glv is helper function that looks up historical LV in struct volume_group's historical_lvs list and returns it if found.
2016-03-01 17:21:21 +03:00
if (!strcmp(glvl->glv->historical->name, ptr)) {
if (glvl_found)
*glvl_found = glvl;
return glvl->glv;
}
}
if (glvl_found)
*glvl_found = NULL;
return NULL;
}
metadata: also look at historical LVs when checking LV name availability Live LVs and historical LVs are in one namespace and the name needs to be unique in whole VG.
2016-03-01 17:31:48 +03:00
int lv_name_is_used_in_vg(const struct volume_group *vg, const char *name, int *historical)
{
int found = 0;
cov: remove unused assigns
2018-11-03 18:48:20 +03:00
if (find_lv(vg, name)) {
metadata: also look at historical LVs when checking LV name availability Live LVs and historical LVs are in one namespace and the name needs to be unique in whole VG.
2016-03-01 17:31:48 +03:00
found = 1;
if (historical)
*historical = 0;
cov: remove unused assigns
2018-11-03 18:48:20 +03:00
} else if (find_historical_glv(vg, name, 0, NULL)) {
metadata: also look at historical LVs when checking LV name availability Live LVs and historical LVs are in one namespace and the name needs to be unique in whole VG.
2016-03-01 17:31:48 +03:00
found = 1;
if (historical)
*historical = 1;
}
return found;
}
o Tool support for segments. o vgmerge working.
2001-11-28 16:45:50 +03:00
struct physical_volume *find_pv(struct volume_group *vg, struct device *dev)
lvremove
2001-10-29 16:52:23 +03:00
{
Basic support for mirrors.
2005-06-01 20:51:55 +04:00
struct pv_list *pvl;
o Added lvextend o Full signed arguments to lvreduce/lvextend o Consistent lv_number/pe map use o Populate pv->pe_allocated o Fixes for allocation/writing of multiple LVs
2001-11-10 01:01:04 +03:00
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(pvl, &vg->pvs)
Basic support for mirrors.
2005-06-01 20:51:55 +04:00
if (dev == pvl->pv->dev)
return pvl->pv;
o Changed struct pv_list { struct list list; struct physical_volume pv; }; to struct pv_list { struct list list; struct physical_volume *pv; }; o New function in toollib 'create_pv_list', which creates a list of pv's from a given command line array of pv's. o Changed lvcreate/extend to use this (fixes lvextend [pv list] bug).
2002-01-21 19:05:23 +03:00
o Added lvextend o Full signed arguments to lvreduce/lvextend o Consistent lv_number/pe map use o Populate pv->pe_allocated o Fixes for allocation/writing of multiple LVs
2001-11-10 01:01:04 +03:00
return NULL;
lvremove
2001-10-29 16:52:23 +03:00
}
Begin conversion so LV id is passed to activation unit instead of struct logical_volume.
2002-02-25 15:56:16 +03:00
o Metadata area struct change. o Support physical extent restrictions on PV lists for allocations e.g. lvcreate -l 200 vg1 /dev/sda1:100-199:300-399
2003-04-25 02:23:24 +04:00
/* Find segment at a given logical extent in an LV */
stacked mirror support (incomplete)
2007-12-20 21:55:46 +03:00
struct lv_segment *find_seg_by_le(const struct logical_volume *lv, uint32_t le)
o Metadata area struct change. o Support physical extent restrictions on PV lists for allocations e.g. lvcreate -l 200 vg1 /dev/sda1:100-199:300-399
2003-04-25 02:23:24 +04:00
{
struct lv_segment *seg;
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(seg, &lv->segments)
o Metadata area struct change. o Support physical extent restrictions on PV lists for allocations e.g. lvcreate -l 200 vg1 /dev/sda1:100-199:300-399
2003-04-25 02:23:24 +04:00
if (le >= seg->le && le < seg->le + seg->len)
return seg;
return NULL;
}
stacked mirror support (incomplete)
2007-12-20 21:55:46 +03:00
struct lv_segment *first_seg(const struct logical_volume *lv)
more mirror library functions
2005-10-28 16:48:50 +04:00
{
Fix first_seg() call for empty segment list. The seg variable is temporary variable for list iterator, code cannot expect that after iteration it remains NULL (it contains non-NULL pointer here id list is empty). Patch fixes first_seg function so it now correctly returns NULL for empty segment list.
2009-05-12 23:09:21 +04:00
struct lv_segment *seg;
more mirror library functions
2005-10-28 16:48:50 +04:00
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(seg, &lv->segments)
Fix first_seg() call for empty segment list. The seg variable is temporary variable for list iterator, code cannot expect that after iteration it remains NULL (it contains non-NULL pointer here id list is empty). Patch fixes first_seg function so it now correctly returns NULL for empty segment list.
2009-05-12 23:09:21 +04:00
return seg;
more mirror library functions
2005-10-28 16:48:50 +04:00
Fix first_seg() call for empty segment list. The seg variable is temporary variable for list iterator, code cannot expect that after iteration it remains NULL (it contains non-NULL pointer here id list is empty). Patch fixes first_seg function so it now correctly returns NULL for empty segment list.
2009-05-12 23:09:21 +04:00
return NULL;
more mirror library functions
2005-10-28 16:48:50 +04:00
}
Add last_seg Implement a function to return the last segment in a LV. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2011-10-29 00:12:54 +04:00
struct lv_segment *last_seg(const struct logical_volume *lv)
{
Just reindent with tabs
2012-02-27 13:51:31 +04:00
struct lv_segment *seg;
Add last_seg Implement a function to return the last segment in a LV. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2011-10-29 00:12:54 +04:00
Just reindent with tabs
2012-02-27 13:51:31 +04:00
dm_list_iterate_back_items(seg, &lv->segments)
return seg;
Add last_seg Implement a function to return the last segment in a LV. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2011-10-29 00:12:54 +04:00
Just reindent with tabs
2012-02-27 13:51:31 +04:00
return NULL;
Add last_seg Implement a function to return the last segment in a LV. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2011-10-29 00:12:54 +04:00
}
Rename internal library function vg_remove to vg_remove_mdas. Later patches should consolidate the vgremove / vgreduce functions but for now let's clarify what vg_remove actually does by changing the name. Author: Dave Wysochanski <dwysocha@redhat.com>
2009-09-03 01:39:07 +04:00
int vg_remove_mdas(struct volume_group *vg)
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
{
Some new features.
2002-11-18 17:04:08 +03:00
struct metadata_area *mda;
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
/* FIXME Improve recovery situation? */
/* Remove each copy of the metadata */
Rename fid->metadata_areas to fid->metadata_areas_in_use. Rename the metadata_areas list to an 'in_use' list to prepare for future 'ignored' list.
2010-06-29 00:32:44 +04:00
dm_list_iterate_items(mda, &vg->fid->metadata_areas_in_use) {
Some new features.
2002-11-18 17:04:08 +03:00
if (mda->ops->vg_remove &&
Use stack return macros throughout.
2008-01-30 16:19:47 +03:00
!mda->ops->vg_remove(vg->fid, vg, mda))
return_0;
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
}
return 1;
}
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
/*
* Determine whether two vgs are compatible for merging.
*/
Use __attribute__ consistently throughout.
2010-07-09 19:34:40 +04:00
int vgs_are_compatible(struct cmd_context *cmd __attribute__((unused)),
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
struct volume_group *vg_from,
struct volume_group *vg_to)
{
struct lv_list *lvl1, *lvl2;
struct pv_list *pvl;
Const fixing Fixing some const warnings - with API change in: int vg_extend(struct volume_group *vg, int pv_count, const char *const *pv_names, Change is needed - as lvm2api expects const behaviour here. So vg_extend() is doing local strdup for unescaping. skip_dev_dir return const char* from const char* vg_name. Rest of the patch is cleanup of related warnings. Also using dm_report_filed_string() API change to simplify casting in _string_disp and _lvname_disp.
2011-02-18 17:47:28 +03:00
const char *name1, *name2;
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
if (lvs_in_vg_activated(vg_from)) {
log_error("Logical volumes in \"%s\" must be inactive",
vg_from->name);
pre-release review cleanups
2008-01-17 20:17:09 +03:00
return 0;
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
}
/* Check compatibility */
if (vg_to->extent_size != vg_from->extent_size) {
log_error("Extent sizes differ: %d (%s) and %d (%s)",
vg_to->extent_size, vg_to->name,
vg_from->extent_size, vg_from->name);
pre-release review cleanups
2008-01-17 20:17:09 +03:00
return 0;
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
}
if (vg_to->max_pv &&
(vg_to->max_pv < vg_to->pv_count + vg_from->pv_count)) {
log_error("Maximum number of physical volumes (%d) exceeded "
" for \"%s\" and \"%s\"", vg_to->max_pv, vg_to->name,
vg_from->name);
pre-release review cleanups
2008-01-17 20:17:09 +03:00
return 0;
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
}
if (vg_to->max_lv &&
Merge lv_is_displayable and lv_is_visible. Displayable and visible is the same thing. volumes_count(vg) is now vg_visible_lvs() and always returns number of LVs from user perspective.
2009-05-14 01:27:43 +04:00
(vg_to->max_lv < vg_visible_lvs(vg_to) + vg_visible_lvs(vg_from))) {
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
log_error("Maximum number of logical volumes (%d) exceeded "
" for \"%s\" and \"%s\"", vg_to->max_lv, vg_to->name,
vg_from->name);
pre-release review cleanups
2008-01-17 20:17:09 +03:00
return 0;
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
}
Fix vgsplit - print error if vgcreate option given w/existing vg destination Fix vgsplit - reject split if metadata types or clustered attributes differ Fix vgsplit - remove physicalextentsize option Add vgsplit test cases
2008-01-22 05:48:53 +03:00
/* Metadata types must be the same */
if (vg_to->fid->fmt != vg_from->fid->fmt) {
log_error("Metadata types differ for \"%s\" and \"%s\"",
vg_to->name, vg_from->name);
return 0;
}
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
/* Check no conflicts with LV names */
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(lvl1, &vg_to->lvs) {
pre-release review cleanups
2008-01-17 20:17:09 +03:00
name1 = lvl1->lv->name;
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(lvl2, &vg_from->lvs) {
pre-release review cleanups
2008-01-17 20:17:09 +03:00
name2 = lvl2->lv->name;
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
if (!strcmp(name1, name2)) {
log_error("Duplicate logical volume "
"name \"%s\" "
"in \"%s\" and \"%s\"",
name1, vg_to->name, vg_from->name);
pre-release review cleanups
2008-01-17 20:17:09 +03:00
return 0;
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
}
}
}
/* Check no PVs are constructed from either VG */
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(pvl, &vg_to->pvs) {
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
if (pv_uses_vg(pvl->pv, vg_from)) {
log_error("Physical volume %s might be constructed "
"from same volume group %s.",
pv_dev_name(pvl->pv), vg_from->name);
pre-release review cleanups
2008-01-17 20:17:09 +03:00
return 0;
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
}
}
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(pvl, &vg_from->pvs) {
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
if (pv_uses_vg(pvl->pv, vg_to)) {
log_error("Physical volume %s might be constructed "
"from same volume group %s.",
pv_dev_name(pvl->pv), vg_to->name);
pre-release review cleanups
2008-01-17 20:17:09 +03:00
return 0;
Create vgs_are_compatible() fn to check whether vgs are compatible for merging. Add new vgmerge and vgsplit tests to check rejection of incompatible vgs. Cleanup comments. Bugzilla: bz251992 --- lib/metadata/metadata-exported.h | 3 + lib/metadata/metadata.c | 89 +++++++++++++++++++++++++++++++++- test/t-vgmerge-usage.sh | 101 +++++++++++++++++++++++++++++++++++++++ test/t-vgsplit-operation.sh | 20 +++++++ tools/vgmerge.c | 69 -------------------------- tools/vgsplit.c | 5 - 6 files changed, 215 insertions(+), 72 deletions(-)
2008-01-16 22:54:39 +03:00
}
}
return 1;
}
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
struct _lv_postorder_baton {
int (*fn)(struct logical_volume *lv, void *data);
void *data;
};
tidy: name all parameters
2017-07-20 11:30:00 +03:00
static int _lv_postorder_visit(struct logical_volume *lv,
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
int (*fn)(struct logical_volume *lv, void *data),
void *data);
static int _lv_each_dependency(struct logical_volume *lv,
int (*fn)(struct logical_volume *lv, void *data),
void *data)
{
Fix some unmatching sign comparation gcc warnings Simple replacement for unsigned type - usually in for() loops.
2011-04-08 18:40:18 +04:00
unsigned i, s;
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
struct lv_segment *lvseg;
lv_dependency: scan also snapshots and extorigins When LV is scanned for its dependencies - scan also origin's snapshots, and thin external origins. So if any PV from snapshot or external origin device is missing - lvm2 will avoid trying to activate such device.
2013-12-17 16:53:15 +04:00
struct dm_list *snh;
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
struct logical_volume *deps[] = {
lv->snapshot ? lv->snapshot->origin : 0,
lv->snapshot ? lv->snapshot->cow : 0 };
cleanup: use DM_ARRAY_SIZE More use of libdevmapper macro
2014-04-04 23:10:30 +04:00
for (i = 0; i < DM_ARRAY_SIZE(deps); ++i) {
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
if (deps[i] && !fn(deps[i], data))
return_0;
}
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(lvseg, &lv->segments) {
lv_dependency: scan also snapshots and extorigins When LV is scanned for its dependencies - scan also origin's snapshots, and thin external origins. So if any PV from snapshot or external origin device is missing - lvm2 will avoid trying to activate such device.
2013-12-17 16:53:15 +04:00
if (lvseg->external_lv && !fn(lvseg->external_lv, data))
return_0;
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
if (lvseg->log_lv && !fn(lvseg->log_lv, data))
return_0;
Thin dependency scan support Go through pool_lv and metadata_lv LVs when doing recursive scan.
2012-01-25 12:50:10 +04:00
if (lvseg->pool_lv && !fn(lvseg->pool_lv, data))
return_0;
if (lvseg->metadata_lv && !fn(lvseg->metadata_lv, data))
return_0;
partial flag for writecache and integrity When a writecache sublv or an integrity metadata sublv are partial (missing a dev), set the partial flag on the upper level LV also, as is done for other sublvs.
2020-12-12 00:56:04 +03:00
if (lvseg->writecache && !fn(lvseg->writecache, data))
return_0;
if (lvseg->integrity_meta_dev && !fn(lvseg->integrity_meta_dev, data))
return_0;
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
for (s = 0; s < lvseg->area_count; ++s) {
if (seg_type(lvseg, s) == AREA_LV && !fn(seg_lv(lvseg,s), data))
return_0;
}
}
lv_dependency: scan also snapshots and extorigins When LV is scanned for its dependencies - scan also origin's snapshots, and thin external origins. So if any PV from snapshot or external origin device is missing - lvm2 will avoid trying to activate such device.
2013-12-17 16:53:15 +04:00
if (lv_is_origin(lv))
dm_list_iterate(snh, &lv->snapshot_segs)
if (!fn(dm_list_struct_base(snh, struct lv_segment, origin_list)->cow, data))
return_0;
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
return 1;
}
static int _lv_postorder_cleanup(struct logical_volume *lv, void *data)
{
if (!(lv->status & POSTORDER_FLAG))
return 1;
lv->status &= ~POSTORDER_FLAG;
if (!_lv_each_dependency(lv, _lv_postorder_cleanup, data))
return_0;
return 1;
}
Fix an lv_postorder bug where it failed to clear temporary flags, making it impossible to use twice with the same LV(s). Discovered by Milan.
2011-02-14 22:27:05 +03:00
static int _lv_postorder_level(struct logical_volume *lv, void *data)
{
struct _lv_postorder_baton *baton = data;
cleanup: add some test for NULL Coverity here is a bit 'blind' here and cannot resolve which code paths are actually able to hit this code path. (It's using 'statistic' to resolve all possible paths, and it's not scanning 'individual' code paths.) This just cleans warns and add 'cheap' tests.
2015-11-17 15:21:22 +03:00
return (data) ? _lv_postorder_visit(lv, baton->fn, baton->data) : 0;
gcc: drop bogus ;
2020-08-28 22:14:06 +03:00
}
Fix an lv_postorder bug where it failed to clear temporary flags, making it impossible to use twice with the same LV(s). Discovered by Milan.
2011-02-14 22:27:05 +03:00
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
static int _lv_postorder_visit(struct logical_volume *lv,
int (*fn)(struct logical_volume *lv, void *data),
void *data)
{
struct _lv_postorder_baton baton;
int r;
if (lv->status & POSTORDER_FLAG)
return 1;
Fix an lv_postorder bug where it failed to clear temporary flags, making it impossible to use twice with the same LV(s). Discovered by Milan.
2011-02-14 22:27:05 +03:00
if (lv->status & POSTORDER_OPEN_FLAG)
return 1; // a data structure loop has closed...
lv->status |= POSTORDER_OPEN_FLAG;
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
baton.fn = fn;
baton.data = data;
r = _lv_each_dependency(lv, _lv_postorder_level, &baton);
Fix an lv_postorder bug where it failed to clear temporary flags, making it impossible to use twice with the same LV(s). Discovered by Milan.
2011-02-14 22:27:05 +03:00
Remove verbose 'visited' messages.
2009-05-30 05:54:29 +04:00
if (r)
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
r = fn(lv, data);
Remove verbose 'visited' messages.
2009-05-30 05:54:29 +04:00
Fix an lv_postorder bug where it failed to clear temporary flags, making it impossible to use twice with the same LV(s). Discovered by Milan.
2011-02-14 22:27:05 +03:00
lv->status &= ~POSTORDER_OPEN_FLAG;
lv->status |= POSTORDER_FLAG;
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
return r;
}
/*
* This will walk the LV dependency graph in depth-first order and in the
* postorder, call a callback function "fn". The void *data is passed along all
* the calls. The callback may return zero to indicate an error and terminate
* the depth-first walk. The error is propagated to return value of
* _lv_postorder.
*/
static int _lv_postorder(struct logical_volume *lv,
int (*fn)(struct logical_volume *lv, void *data),
void *data)
{
int r;
Lock memory for shared VG Use debug pool locking functionality. So the command could check, whether the memory in the pool has not been modified. For lv_postoder() instead of unlocking and locking for every changed struct status member do it once when entering and leaving function. (mprotect would trap each such memory access). Currently lv_postoder() does not modify other part of vg structure then status flags of each LV with flags that are reverted back to its original state after function exit.
2011-08-11 21:34:30 +04:00
int pool_locked = dm_pool_locked(lv->vg->vgmem);
if (pool_locked && !dm_pool_unlock(lv->vg->vgmem, 0))
return_0;
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
r = _lv_postorder_visit(lv, fn, data);
_lv_postorder_cleanup(lv, 0);
Lock memory for shared VG Use debug pool locking functionality. So the command could check, whether the memory in the pool has not been modified. For lv_postoder() instead of unlocking and locking for every changed struct status member do it once when entering and leaving function. (mprotect would trap each such memory access). Currently lv_postoder() does not modify other part of vg structure then status flags of each LV with flags that are reverted back to its original state after function exit.
2011-08-11 21:34:30 +04:00
if (pool_locked && !dm_pool_lock(lv->vg->vgmem, 0))
return_0;
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
return r;
}
Refactor code for _lv_postoder Add _lv_postorder_vg() - for calling _lv_postorder() for every LV from VG. We use this in 2 places - vg_mark_partial_lvs() and vg_validate() so make it as a one function. Benefit here is - to use only one cleanup code and avoid potentially duplicate scans of same LVs.
2011-03-10 17:40:32 +03:00
/*
* Calls _lv_postorder() on each LV from VG. Avoids duplicate transitivity visits.
* Clears with _lv_postorder_cleanup() when all LVs were visited by postorder.
*/
static int _lv_postorder_vg(struct volume_group *vg,
int (*fn)(struct logical_volume *lv, void *data),
void *data)
{
struct lv_list *lvl;
int r = 1;
Lock memory for shared VG Use debug pool locking functionality. So the command could check, whether the memory in the pool has not been modified. For lv_postoder() instead of unlocking and locking for every changed struct status member do it once when entering and leaving function. (mprotect would trap each such memory access). Currently lv_postoder() does not modify other part of vg structure then status flags of each LV with flags that are reverted back to its original state after function exit.
2011-08-11 21:34:30 +04:00
int pool_locked = dm_pool_locked(vg->vgmem);
if (pool_locked && !dm_pool_unlock(vg->vgmem, 0))
return_0;
Refactor code for _lv_postoder Add _lv_postorder_vg() - for calling _lv_postorder() for every LV from VG. We use this in 2 places - vg_mark_partial_lvs() and vg_validate() so make it as a one function. Benefit here is - to use only one cleanup code and avoid potentially duplicate scans of same LVs.
2011-03-10 17:40:32 +03:00
dm_list_iterate_items(lvl, &vg->lvs)
if (!_lv_postorder_visit(lvl->lv, fn, data)) {
stack;
r = 0;
}
dm_list_iterate_items(lvl, &vg->lvs)
_lv_postorder_cleanup(lvl->lv, 0);
Lock memory for shared VG Use debug pool locking functionality. So the command could check, whether the memory in the pool has not been modified. For lv_postoder() instead of unlocking and locking for every changed struct status member do it once when entering and leaving function. (mprotect would trap each such memory access). Currently lv_postoder() does not modify other part of vg structure then status flags of each LV with flags that are reverted back to its original state after function exit.
2011-08-11 21:34:30 +04:00
if (pool_locked && !dm_pool_lock(vg->vgmem, 0))
return_0;
Refactor code for _lv_postoder Add _lv_postorder_vg() - for calling _lv_postorder() for every LV from VG. We use this in 2 places - vg_mark_partial_lvs() and vg_validate() so make it as a one function. Benefit here is - to use only one cleanup code and avoid potentially duplicate scans of same LVs.
2011-03-10 17:40:32 +03:00
return r;
}
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
struct _lv_mark_if_partial_baton {
int partial;
};
static int _lv_mark_if_partial_collect(struct logical_volume *lv, void *data)
{
struct _lv_mark_if_partial_baton *baton = data;
cleanup: use lv_is_partial Check for PARTIAL_LV flag in standard way.
2016-03-02 22:59:03 +03:00
if (baton && lv_is_partial(lv))
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
baton->partial = 1;
return 1;
}
static int _lv_mark_if_partial_single(struct logical_volume *lv, void *data)
{
Fix some unmatching sign comparation gcc warnings Simple replacement for unsigned type - usually in for() loops.
2011-04-08 18:40:18 +04:00
unsigned s;
cleanup: zero baton in struct initilizer
2016-12-09 17:08:04 +03:00
struct _lv_mark_if_partial_baton baton = { .partial = 0 };
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
struct lv_segment *lvseg;
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(lvseg, &lv->segments) {
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
for (s = 0; s < lvseg->area_count; ++s) {
if (seg_type(lvseg, s) == AREA_PV) {
Introduce is_missing_pv().
2010-03-16 17:37:38 +03:00
if (is_missing_pv(seg_pv(lvseg, s)))
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
lv->status |= PARTIAL_LV;
}
}
}
Check error from _lv_each_dependency _lv_mark_if_partial_collect cannot fail, however it's good to keep checking here as we do in all other cases.
2012-02-28 15:10:45 +04:00
if (!_lv_each_dependency(lv, _lv_mark_if_partial_collect, &baton))
return_0;
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
if (baton.partial)
lv->status |= PARTIAL_LV;
return 1;
}
/*
* Mark LVs with missing PVs using PARTIAL_LV status flag. The flag is
* propagated transitively, so LVs referencing other LVs are marked
* partial as well, if any of their referenced LVs are marked partial.
*/
Make vg_mark_partial_lvs also clear existing PARTIAL_LV flags, so it can be issued repeatedly on the same VG, keeping the PARTIAL_LV flags up to date.
2011-05-07 17:32:05 +04:00
int vg_mark_partial_lvs(struct volume_group *vg, int clear)
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
{
Make vg_mark_partial_lvs also clear existing PARTIAL_LV flags, so it can be issued repeatedly on the same VG, keeping the PARTIAL_LV flags up to date.
2011-05-07 17:32:05 +04:00
struct lv_list *lvl;
if (clear)
dm_list_iterate_items(lvl, &vg->lvs)
lvl->lv->status &= ~PARTIAL_LV;
Refactor code for _lv_postoder Add _lv_postorder_vg() - for calling _lv_postorder() for every LV from VG. We use this in 2 places - vg_mark_partial_lvs() and vg_validate() so make it as a one function. Benefit here is - to use only one cleanup code and avoid potentially duplicate scans of same LVs.
2011-03-10 17:40:32 +03:00
if (!_lv_postorder_vg(vg, _lv_mark_if_partial_single, NULL))
return_0;
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
return 1;
}
Use readahead of underlying device and not default (smaller) one. When we are stacking LV over device, which has for some reason increased read_ahead (e.g. MD RAID), the read_ahead hint for libdevmapper is wrong (it is zero). If the calculated read_ahead hint is zero, patch uses read_ahead of underlying device (if first segment is PV) when setting DM_READ_AHEAD_MINIMUM_FLAG. Because we are using dev-cache, it also store this value to cache for future use (if several LVs are over one PV, BLKRAGET is called only once for underlying device.) This should fix all the reamining problems with readahead mismatch reported for DM over MD configurations (and similar cases).
2009-05-20 15:09:49 +04:00
/*
* Be sure that all PV devices have cached read ahead in dev-cache
* Currently it takes read_ahead from first PV segment only
*/
static int _lv_read_ahead_single(struct logical_volume *lv, void *data)
{
struct lv_segment *seg = first_seg(lv);
uint32_t seg_read_ahead = 0, *read_ahead = data;
check: add internal errors for unexpected paths Adding couple INTERNAL_ERROR reports for unwanted parameters: Ensure the 'top' metadata node cannot be NULL for lvmetad. Make obvious vginfo2 cannot be NULL. Report internal error if handler and vg is undefined. Check for handle in poll_vg(). Ensure seg is not NULL in dev_manager_transient(). Report missing read_ahead for _lv_read_ahead_single(). Check for report handler in dm_report_object(). Check missing VG in _vgreduce_single().
2012-06-21 14:43:31 +04:00
if (!read_ahead) {
log_error(INTERNAL_ERROR "Read ahead data missing.");
return 0;
}
Fix readahead calculation problems. During vgreduce is failed mirror image replaced with error segment, this segmant type has always area_count == 0. Current code expects that there is at least one area with device, patch fixes it by additional check (fixes segfault during vgreduce). Also do not calculate readahead in every lv_info call, we only need to cache PV readahead before activation calls which locks memory.
2009-06-01 16:43:31 +04:00
if (seg && seg->area_count && seg_type(seg, 0) == AREA_PV)
Use readahead of underlying device and not default (smaller) one. When we are stacking LV over device, which has for some reason increased read_ahead (e.g. MD RAID), the read_ahead hint for libdevmapper is wrong (it is zero). If the calculated read_ahead hint is zero, patch uses read_ahead of underlying device (if first segment is PV) when setting DM_READ_AHEAD_MINIMUM_FLAG. Because we are using dev-cache, it also store this value to cache for future use (if several LVs are over one PV, BLKRAGET is called only once for underlying device.) This should fix all the reamining problems with readahead mismatch reported for DM over MD configurations (and similar cases).
2009-05-20 15:09:49 +04:00
dev_get_read_ahead(seg_pv(seg, 0)->dev, &seg_read_ahead);
if (seg_read_ahead > *read_ahead)
*read_ahead = seg_read_ahead;
return 1;
}
Fix readahead calculation problems. During vgreduce is failed mirror image replaced with error segment, this segmant type has always area_count == 0. Current code expects that there is at least one area with device, patch fixes it by additional check (fixes segfault during vgreduce). Also do not calculate readahead in every lv_info call, we only need to cache PV readahead before activation calls which locks memory.
2009-06-01 16:43:31 +04:00
/*
* Calculate readahead for logical volume from underlying PV devices.
* If read_ahead is NULL, only ensure that readahead of PVs are preloaded
* into PV struct device in dev cache.
*/
void lv_calculate_readahead(const struct logical_volume *lv, uint32_t *read_ahead)
Use readahead of underlying device and not default (smaller) one. When we are stacking LV over device, which has for some reason increased read_ahead (e.g. MD RAID), the read_ahead hint for libdevmapper is wrong (it is zero). If the calculated read_ahead hint is zero, patch uses read_ahead of underlying device (if first segment is PV) when setting DM_READ_AHEAD_MINIMUM_FLAG. Because we are using dev-cache, it also store this value to cache for future use (if several LVs are over one PV, BLKRAGET is called only once for underlying device.) This should fix all the reamining problems with readahead mismatch reported for DM over MD configurations (and similar cases).
2009-05-20 15:09:49 +04:00
{
Fix readahead calculation problems. During vgreduce is failed mirror image replaced with error segment, this segmant type has always area_count == 0. Current code expects that there is at least one area with device, patch fixes it by additional check (fixes segfault during vgreduce). Also do not calculate readahead in every lv_info call, we only need to cache PV readahead before activation calls which locks memory.
2009-06-01 16:43:31 +04:00
uint32_t _read_ahead = 0;
Use readahead of underlying device and not default (smaller) one. When we are stacking LV over device, which has for some reason increased read_ahead (e.g. MD RAID), the read_ahead hint for libdevmapper is wrong (it is zero). If the calculated read_ahead hint is zero, patch uses read_ahead of underlying device (if first segment is PV) when setting DM_READ_AHEAD_MINIMUM_FLAG. Because we are using dev-cache, it also store this value to cache for future use (if several LVs are over one PV, BLKRAGET is called only once for underlying device.) This should fix all the reamining problems with readahead mismatch reported for DM over MD configurations (and similar cases).
2009-05-20 15:09:49 +04:00
if (lv->read_ahead == DM_READ_AHEAD_AUTO)
Fix readahead calculation problems. During vgreduce is failed mirror image replaced with error segment, this segmant type has always area_count == 0. Current code expects that there is at least one area with device, patch fixes it by additional check (fixes segfault during vgreduce). Also do not calculate readahead in every lv_info call, we only need to cache PV readahead before activation calls which locks memory.
2009-06-01 16:43:31 +04:00
_lv_postorder((struct logical_volume *)lv, _lv_read_ahead_single, &_read_ahead);
Use readahead of underlying device and not default (smaller) one. When we are stacking LV over device, which has for some reason increased read_ahead (e.g. MD RAID), the read_ahead hint for libdevmapper is wrong (it is zero). If the calculated read_ahead hint is zero, patch uses read_ahead of underlying device (if first segment is PV) when setting DM_READ_AHEAD_MINIMUM_FLAG. Because we are using dev-cache, it also store this value to cache for future use (if several LVs are over one PV, BLKRAGET is called only once for underlying device.) This should fix all the reamining problems with readahead mismatch reported for DM over MD configurations (and similar cases).
2009-05-20 15:09:49 +04:00
Fix readahead calculation problems. During vgreduce is failed mirror image replaced with error segment, this segmant type has always area_count == 0. Current code expects that there is at least one area with device, patch fixes it by additional check (fixes segfault during vgreduce). Also do not calculate readahead in every lv_info call, we only need to cache PV readahead before activation calls which locks memory.
2009-06-01 16:43:31 +04:00
if (read_ahead) {
logging: classify log_debug messages Place most log_debug() messages into a class.
2013-01-08 02:30:29 +04:00
log_debug_metadata("Calculated readahead of LV %s is %u", lv->name, _read_ahead);
Fix readahead calculation problems. During vgreduce is failed mirror image replaced with error segment, this segmant type has always area_count == 0. Current code expects that there is at least one area with device, patch fixes it by additional check (fixes segfault during vgreduce). Also do not calculate readahead in every lv_info call, we only need to cache PV readahead before activation calls which locks memory.
2009-06-01 16:43:31 +04:00
*read_ahead = _read_ahead;
}
Use readahead of underlying device and not default (smaller) one. When we are stacking LV over device, which has for some reason increased read_ahead (e.g. MD RAID), the read_ahead hint for libdevmapper is wrong (it is zero). If the calculated read_ahead hint is zero, patch uses read_ahead of underlying device (if first segment is PV) when setting DM_READ_AHEAD_MINIMUM_FLAG. Because we are using dev-cache, it also store this value to cache for future use (if several LVs are over one PV, BLKRAGET is called only once for underlying device.) This should fix all the reamining problems with readahead mismatch reported for DM over MD configurations (and similar cases).
2009-05-20 15:09:49 +04:00
}
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
struct validate_hash {
struct dm_hash_table *lvname;
metadata: also validate historical LVs in VG in vg_validate and check_lv_segments
2016-03-01 17:32:01 +03:00
struct dm_hash_table *historical_lvname;
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
struct dm_hash_table *lvid;
metadata: also validate historical LVs in VG in vg_validate and check_lv_segments
2016-03-01 17:32:01 +03:00
struct dm_hash_table *historical_lvid;
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
struct dm_hash_table *pvid;
Add lvmlockd
2015-03-05 23:00:44 +03:00
struct dm_hash_table *lv_lock_args;
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
};
Add further consistency checking to vg_validate, ensuring that all segment areas point to LVs or PVs that are listed in the respective VG.
2010-12-14 20:51:09 +03:00
/*
* Check that an LV and all its PV references are correctly listed in vg->lvs
* and vg->pvs, respectively. This only looks at a single LV, but *not* at the
* LVs it is using. To do the latter, you should use _lv_postorder with this
* function. C.f. vg_validate.
*/
static int _lv_validate_references_single(struct logical_volume *lv, void *data)
{
struct volume_group *vg = lv->vg;
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
struct validate_hash *vhash = data;
Add further consistency checking to vg_validate, ensuring that all segment areas point to LVs or PVs that are listed in the respective VG.
2010-12-14 20:51:09 +03:00
struct lv_segment *lvseg;
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
struct physical_volume *pv;
Fix some unmatching sign comparation gcc warnings Simple replacement for unsigned type - usually in for() loops.
2011-04-08 18:40:18 +04:00
unsigned s;
Add further consistency checking to vg_validate, ensuring that all segment areas point to LVs or PVs that are listed in the respective VG.
2010-12-14 20:51:09 +03:00
int r = 1;
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (lv != dm_hash_lookup_binary(vhash->lvid, &lv->lvid.id[1],
sizeof(lv->lvid.id[1]))) {
Add further consistency checking to vg_validate, ensuring that all segment areas point to LVs or PVs that are listed in the respective VG.
2010-12-14 20:51:09 +03:00
log_error(INTERNAL_ERROR
"Referenced LV %s not listed in VG %s.",
lv->name, vg->name);
r = 0;
}
dm_list_iterate_items(lvseg, &lv->segments) {
for (s = 0; s < lvseg->area_count; ++s) {
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (seg_type(lvseg, s) != AREA_PV)
continue;
pv = seg_pv(lvseg, s);
/* look up the reference in vg->pvs */
if (pv != dm_hash_lookup_binary(vhash->pvid, &pv->id,
sizeof(pv->id))) {
log_error(INTERNAL_ERROR
"Referenced PV %s not listed in VG %s.",
pv_dev_name(pv), vg->name);
r = 0;
Add further consistency checking to vg_validate, ensuring that all segment areas point to LVs or PVs that are listed in the respective VG.
2010-12-14 20:51:09 +03:00
}
}
}
return r;
}
metadata: vg_validate lock_args
2015-07-09 21:24:28 +03:00
/*
* Format is <version>:<info>
*/
static int _validate_lock_args_chars(const char *lock_args)
{
gcc: cleanup some sign warnings When comparing unsigned with int, the comparision is made as 'unsigned' type, so make it rather explicit which type is being compared.
2016-02-23 14:18:48 +03:00
unsigned i;
metadata: vg_validate lock_args
2015-07-09 21:24:28 +03:00
char c;
int found_colon = 0;
int r = 1;
for (i = 0; i < strlen(lock_args); i++) {
c = lock_args[i];
if (!isalnum(c) && c != '.' && c != '_' && c != '-' && c != '+' && c != ':') {
gcc: cleanup some sign warnings When comparing unsigned with int, the comparision is made as 'unsigned' type, so make it rather explicit which type is being compared.
2016-02-23 14:18:48 +03:00
log_error(INTERNAL_ERROR "Invalid character at index %u of lock_args \"%s\"",
metadata: vg_validate lock_args
2015-07-09 21:24:28 +03:00
i, lock_args);
r = 0;
}
if (c == ':' && found_colon) {
gcc: cleanup some sign warnings When comparing unsigned with int, the comparision is made as 'unsigned' type, so make it rather explicit which type is being compared.
2016-02-23 14:18:48 +03:00
log_error(INTERNAL_ERROR "Invalid colon at index %u of lock_args \"%s\"",
metadata: vg_validate lock_args
2015-07-09 21:24:28 +03:00
i, lock_args);
r = 0;
}
if (c == ':')
found_colon = 1;
}
return r;
}
static int _validate_vg_lock_args(struct volume_group *vg)
{
if (!_validate_lock_args_chars(vg->lock_args)) {
log_error(INTERNAL_ERROR "VG %s has invalid lock_args chars", vg->name);
return 0;
}
return 1;
}
/*
* For lock_type sanlock, LV lock_args are <version>:<info>
* For lock_type dlm, LV lock_args are not used, and lock_args is
* just set to "dlm".
*/
static int _validate_lv_lock_args(struct logical_volume *lv)
{
int r = 1;
if (!strcmp(lv->vg->lock_type, "sanlock")) {
if (!_validate_lock_args_chars(lv->lock_args)) {
log_error(INTERNAL_ERROR "LV %s/%s has invalid lock_args chars",
lv->vg->name, display_lvname(lv));
return 0;
}
} else if (!strcmp(lv->vg->lock_type, "dlm")) {
if (strcmp(lv->lock_args, "dlm")) {
log_error(INTERNAL_ERROR "LV %s/%s has invalid lock_args \"%s\"",
lv->vg->name, display_lvname(lv), lv->lock_args);
r = 0;
}
}
return r;
}
Fix pvmove segment splitting. Abstract vg_validate.
2005-07-12 23:40:59 +04:00
int vg_validate(struct volume_group *vg)
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
{
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
struct pv_list *pvl;
struct lv_list *lvl;
metadata: also validate historical LVs in VG in vg_validate and check_lv_segments
2016-03-01 17:32:01 +03:00
struct glv_list *glvl;
struct historical_logical_volume *hlv;
Add a validation step for pvmoveN internal LVs to vg_validate.
2010-12-14 20:07:35 +03:00
struct lv_segment *seg;
report: select: refactor: move str_list to libdm The list of strings is used quite frequently and we'd like to reuse this simple structure for report selection support too. Make it part of libdevmapper for general reuse throughout the code. This also simplifies the LVM code a bit since we don't need to include and manage lvm-types.h anymore (the string list was the only structure defined there).
2014-05-29 11:41:03 +04:00
struct dm_str_list *sl;
Use __attribute__ consistently throughout.
2010-07-09 19:34:40 +04:00
char uuid[64] __attribute__((aligned(8)));
vgsplit: Fix VG component of lvid. Fix VG component of lvid in vgsplit and vgmerge Update vg_validate() to detect the error. Call lv_is_active() before moving LV into new VG, not after.
2014-07-03 22:06:04 +04:00
char uuid2[64] __attribute__((aligned(8)));
Add checks for duplicate LV name, lvid and PV id before writing metadata. Report all sanity check failures, not just the first.
2006-08-09 23:33:25 +04:00
int r = 1;
cleanup: use plain unsigned types
2013-07-05 19:10:11 +04:00
unsigned hidden_lv_count = 0, lv_count = 0, lv_visible_count = 0;
unsigned pv_count = 0;
unsigned num_snapshots = 0;
thin: add pool metadata spare lv support Add support for pool's metadata spare volume.
2013-07-05 19:10:47 +04:00
unsigned spare_count = 0;
vg_validate: check size of lv_name + vg_name Since the whole dm device name may not exceed 127 characters, validate no LV names is bigger then this limit.
2014-03-31 13:51:50 +04:00
size_t vg_name_len = strlen(vg->name);
size_t dev_name_len;
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
struct validate_hash vhash = { NULL };
Add checks for duplicate LV name, lvid and PV id before writing metadata. Report all sanity check failures, not just the first.
2006-08-09 23:33:25 +04:00
Extend cling allocation policy to recognise PV tags (cling_by_tags). Add allocation/cling_tag_list to lvm.conf.
2010-11-09 15:34:40 +03:00
if (vg->alloc == ALLOC_CLING_BY_TAGS) {
log_error(INTERNAL_ERROR "VG %s allocation policy set to invalid cling_by_tags.",
vg->name);
r = 0;
}
metadata: Detect internal use of LVM_WRITE_LOCKED. Generate internal error if LVM_WRITE_LOCKED ever appears in struct volume_group: it's only used in external metadata.
2015-03-09 21:56:24 +03:00
if (vg->status & LVM_WRITE_LOCKED) {
log_error(INTERNAL_ERROR "VG %s has external flag LVM_WRITE_LOCKED set internally.",
vg->name);
r = 0;
}
Fix format_text mda_setup pv->size and pv_setup pe_count calculations. (This area of the code needs a lot more work.)
2006-10-06 02:02:52 +04:00
/* FIXME Also check there's no data/metadata overlap */
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (!(vhash.pvid = dm_hash_create(vg->pv_count))) {
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
log_error("Failed to allocate pvid hash.");
return 0;
}
Check all tags and LV names are in a valid form in vg_validate.
2012-02-23 04:11:01 +04:00
dm_list_iterate_items(sl, &vg->tags)
if (!validate_tag(sl->str)) {
log_error(INTERNAL_ERROR "VG %s tag %s has invalid form.",
vg->name, sl->str);
r = 0;
}
Improve vg_validate to detect some loops in lists.
2010-04-01 15:43:24 +04:00
dm_list_iterate_items(pvl, &vg->pvs) {
if (++pv_count > vg->pv_count) {
log_error(INTERNAL_ERROR "PV list corruption detected in VG %s.", vg->name);
/* FIXME Dump list structure? */
r = 0;
}
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
Add pv->vg to solidify link between a pv and a vg. lvm2app needs a link back to the vg in order to use the vg handle for memory allocations as well as other things. This patch adds the field to struct physical_volume, and sets pv->vg when reading a vg from disk or extending a vg by using the helper function previously added, add_pvl_to_vgs(). Moves and renames are handled with separate code inside move_pv() and vgmerge(). Add pv->vg check to vg_validate(). A NULL value in pv->vg signifies membership in the orphan VG. Note though in the case of pv_read() on a device with metadatacopies == 0, more devices may need to be read for an authoritative answer. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-13 21:26:36 +04:00
if (pvl->pv->vg != vg) {
log_error(INTERNAL_ERROR "VG %s PV list entry points "
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
"to different VG %s.", vg->name,
Add pv->vg to solidify link between a pv and a vg. lvm2app needs a link back to the vg in order to use the vg handle for memory allocations as well as other things. This patch adds the field to struct physical_volume, and sets pv->vg when reading a vg from disk or extending a vg by using the helper function previously added, add_pvl_to_vgs(). Moves and renames are handled with separate code inside move_pv() and vgmerge(). Add pv->vg check to vg_validate(). A NULL value in pv->vg signifies membership in the orphan VG. Note though in the case of pv_read() on a device with metadatacopies == 0, more devices may need to be read for an authoritative answer. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-04-13 21:26:36 +04:00
pvl->pv->vg ? pvl->pv->vg->name : "NULL");
r = 0;
}
Fix vgsplit for lvm1 format (set and validate VG name in PVs metadata). Split metadata areas in vgsplit properly.
2007-03-23 15:43:17 +03:00
if (strcmp(pvl->pv->vg_name, vg->name)) {
Just add '.' at the end of error message.
2010-01-07 17:29:53 +03:00
log_error(INTERNAL_ERROR "VG name for PV %s is corrupted.",
Add pv_dev_name() to access PV device name. Patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 18:29:32 +04:00
pv_dev_name(pvl->pv));
Fix vgsplit for lvm1 format (set and validate VG name in PVs metadata). Split metadata areas in vgsplit properly.
2007-03-23 15:43:17 +03:00
r = 0;
}
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (dm_hash_lookup_binary(vhash.pvid, &pvl->pv->id,
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
sizeof(pvl->pv->id))) {
if (!id_write_format(&pvl->pv->id, uuid,
sizeof(uuid)))
stack;
log_error(INTERNAL_ERROR "Duplicate PV id "
"%s detected for %s in %s.",
uuid, pv_dev_name(pvl->pv),
vg->name);
r = 0;
}
Check all tags and LV names are in a valid form in vg_validate.
2012-02-23 04:11:01 +04:00
dm_list_iterate_items(sl, &pvl->pv->tags)
if (!validate_tag(sl->str)) {
log_error(INTERNAL_ERROR "PV %s tag %s has invalid form.",
pv_dev_name(pvl->pv), sl->str);
r = 0;
}
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (!dm_hash_insert_binary(vhash.pvid, &pvl->pv->id,
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
sizeof(pvl->pv->id), pvl->pv)) {
log_error("Failed to hash pvid.");
r = 0;
break;
}
Add checks for duplicate LV name, lvid and PV id before writing metadata. Report all sanity check failures, not just the first.
2006-08-09 23:33:25 +04:00
}
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
Reinstate full PV size when removing from VG. Support loopfiles for testing. Complete the pv_segment support.
2005-05-03 21:28:23 +04:00
if (!check_pv_segments(vg)) {
#define an INTERNAL_ERROR macro and use it throughout LVM.
2009-12-16 22:22:11 +03:00
log_error(INTERNAL_ERROR "PV segments corrupted in %s.",
Reinstate full PV size when removing from VG. Support loopfiles for testing. Complete the pv_segment support.
2005-05-03 21:28:23 +04:00
vg->name);
Add checks for duplicate LV name, lvid and PV id before writing metadata. Report all sanity check failures, not just the first.
2006-08-09 23:33:25 +04:00
r = 0;
}
metadata: vg: add removed_lvs field to collect LVs which have been removed Do not keep dangling LVs if they're removed from the vg->lvs list and move them to vg->removed_lvs instead (this is actually similar to already existing vg->removed_pvs list, just it's for LVs now). Once we have this vg->removed_lvs list indexed so it's possible to do lookups for LVs quickly, we can remove the LV_REMOVED flag as that one won't be needed anymore - instead of checking the flag, we can directly check the vg->removed_lvs list if the LV is present there or not and to say if the LV is removed or not then. For now, we don't have this index, but it may be implemented in the future.
2015-03-23 15:32:00 +03:00
dm_list_iterate_items(lvl, &vg->removed_lvs) {
if (!(lvl->lv->status & LV_REMOVED)) {
log_error(INTERNAL_ERROR "LV %s is not marked as removed while it's part "
"of removed LV list for VG %s", lvl->lv->name, vg->name);
r = 0;
}
}
Merge lv_is_displayable and lv_is_visible. Displayable and visible is the same thing. volumes_count(vg) is now vg_visible_lvs() and always returns number of LVs from user perspective.
2009-05-14 01:27:43 +04:00
/*
* Count all non-snapshot invisible LVs
*/
dm_list_iterate_items(lvl, &vg->lvs) {
Improve vg_validate to detect some loops in lists.
2010-04-01 15:43:24 +04:00
lv_count++;
metadata: vg: add removed_lvs field to collect LVs which have been removed Do not keep dangling LVs if they're removed from the vg->lvs list and move them to vg->removed_lvs instead (this is actually similar to already existing vg->removed_pvs list, just it's for LVs now). Once we have this vg->removed_lvs list indexed so it's possible to do lookups for LVs quickly, we can remove the LV_REMOVED flag as that one won't be needed anymore - instead of checking the flag, we can directly check the vg->removed_lvs list if the LV is present there or not and to say if the LV is removed or not then. For now, we don't have this index, but it may be implemented in the future.
2015-03-23 15:32:00 +03:00
if (lvl->lv->status & LV_REMOVED) {
log_error(INTERNAL_ERROR "LV %s is marked as removed while it's "
"still part of the VG %s", lvl->lv->name, vg->name);
r = 0;
}
metadata: Detect internal use of LVM_WRITE_LOCKED. Generate internal error if LVM_WRITE_LOCKED ever appears in struct volume_group: it's only used in external metadata.
2015-03-09 21:56:24 +03:00
if (lvl->lv->status & LVM_WRITE_LOCKED) {
log_error(INTERNAL_ERROR "LV %s has external flag LVM_WRITE_LOCKED set internally.",
metadata: Fix recent vg_validate message text.
2015-03-17 20:48:56 +03:00
lvl->lv->name);
metadata: Detect internal use of LVM_WRITE_LOCKED. Generate internal error if LVM_WRITE_LOCKED ever appears in struct volume_group: it's only used in external metadata.
2015-03-09 21:56:24 +03:00
r = 0;
}
vg_validate: check size of lv_name + vg_name Since the whole dm device name may not exceed 127 characters, validate no LV names is bigger then this limit.
2014-03-31 13:51:50 +04:00
dev_name_len = strlen(lvl->lv->name) + vg_name_len + 3;
if (dev_name_len >= NAME_LEN) {
log_error(INTERNAL_ERROR "LV name \"%s/%s\" length %"
PRIsize_t " is not supported.",
vg->name, lvl->lv->name, dev_name_len);
r = 0;
}
vgsplit: Fix VG component of lvid. Fix VG component of lvid in vgsplit and vgmerge Update vg_validate() to detect the error. Call lv_is_active() before moving LV into new VG, not after.
2014-07-03 22:06:04 +04:00
if (!id_equal(&lvl->lv->lvid.id[0], &lvl->lv->vg->id)) {
if (!id_write_format(&lvl->lv->lvid.id[0], uuid,
sizeof(uuid)))
stack;
if (!id_write_format(&lvl->lv->vg->id, uuid2,
sizeof(uuid2)))
stack;
log_error(INTERNAL_ERROR "LV %s has VG UUID %s but its VG %s has UUID %s",
lvl->lv->name, uuid, lvl->lv->vg->name, uuid2);
r = 0;
}
vg_validate: check size of lv_name + vg_name Since the whole dm device name may not exceed 127 characters, validate no LV names is bigger then this limit.
2014-03-31 13:51:50 +04:00
validation: move check for spare correctness Check lv feature when we check LVs
2014-09-21 13:34:50 +04:00
if (lv_is_pool_metadata_spare(lvl->lv)) {
if (++spare_count > 1) {
pre-release
2014-11-11 17:13:00 +03:00
log_error(INTERNAL_ERROR "LV %s is extra pool metadata spare volume. %u found but only 1 allowed.",
validation: move check for spare correctness Check lv feature when we check LVs
2014-09-21 13:34:50 +04:00
lvl->lv->name, spare_count);
r = 0;
}
if (vg->pool_metadata_spare_lv != lvl->lv) {
pre-release
2014-11-11 17:13:00 +03:00
log_error(INTERNAL_ERROR "LV %s is not the VG's pool metadata spare volume.",
validation: move check for spare correctness Check lv feature when we check LVs
2014-09-21 13:34:50 +04:00
lvl->lv->name);
r = 0;
}
}
Improve vg_validate to detect some loops in lists.
2010-04-01 15:43:24 +04:00
if (lv_is_cow(lvl->lv))
num_snapshots++;
if (lv_is_visible(lvl->lv))
lv_visible_count++;
Avoid endless loop if lv->segments list is corrupted
2010-04-01 17:08:06 +04:00
if (!check_lv_segments(lvl->lv, 0)) {
log_error(INTERNAL_ERROR "LV segments corrupted in %s.",
lvl->lv->name);
r = 0;
}
Extend cling allocation policy to recognise PV tags (cling_by_tags). Add allocation/cling_tag_list to lvm.conf.
2010-11-09 15:34:40 +03:00
if (lvl->lv->alloc == ALLOC_CLING_BY_TAGS) {
log_error(INTERNAL_ERROR "LV %s allocation policy set to invalid cling_by_tags.",
lvl->lv->name);
r = 0;
}
Check all tags and LV names are in a valid form in vg_validate.
2012-02-23 04:11:01 +04:00
if (!validate_name(lvl->lv->name)) {
log_error(INTERNAL_ERROR "LV name %s has invalid form.", lvl->lv->name);
r = 0;
}
dm_list_iterate_items(sl, &lvl->lv->tags)
if (!validate_tag(sl->str)) {
log_error(INTERNAL_ERROR "LV %s tag %s has invalid form.",
lvl->lv->name, sl->str);
r = 0;
}
Merge lv_is_displayable and lv_is_visible. Displayable and visible is the same thing. volumes_count(vg) is now vg_visible_lvs() and always returns number of LVs from user perspective.
2009-05-14 01:27:43 +04:00
if (lvl->lv->status & VISIBLE_LV)
continue;
/* snapshots */
Fix counting of virtual origin LVs in vg_validate. (mbroz)
2009-05-27 17:19:34 +04:00
if (lv_is_cow(lvl->lv))
continue;
/* virtual origins are always hidden */
if (lv_is_origin(lvl->lv) && !lv_is_virtual_origin(lvl->lv))
Merge lv_is_displayable and lv_is_visible. Displayable and visible is the same thing. volumes_count(vg) is now vg_visible_lvs() and always returns number of LVs from user perspective.
2009-05-14 01:27:43 +04:00
continue;
/* count other non-snapshot invisible volumes */
Fix counting of virtual origin LVs in vg_validate. (mbroz)
2009-05-27 17:19:34 +04:00
hidden_lv_count++;
Merge lv_is_displayable and lv_is_visible. Displayable and visible is the same thing. volumes_count(vg) is now vg_visible_lvs() and always returns number of LVs from user perspective.
2009-05-14 01:27:43 +04:00
/*
* FIXME: add check for unreferenced invisible LVs
* - snapshot cow & origin
* - mirror log & images
* - mirror conversion volumes (_mimagetmp*)
*/
}
/*
* all volumes = visible LVs + snapshot_cows + invisible LVs
*/
Improve vg_validate to detect some loops in lists.
2010-04-01 15:43:24 +04:00
if (lv_count != lv_visible_count + num_snapshots + hidden_lv_count) {
cleanup: use plain unsigned types
2013-07-05 19:10:11 +04:00
log_error(INTERNAL_ERROR "#LVs (%u) != #visible LVs (%u) "
"+ #snapshots (%u) + #internal LVs (%u) in VG %s",
lv_count, lv_visible_count, num_snapshots,
hidden_lv_count, vg->name);
post-release
2008-06-06 23:28:35 +04:00
r = 0;
Check lv_count in vg_validate. Fix internal LV counter when a snapshot is removed.
2008-04-22 16:54:33 +04:00
}
Avoid endless loop if lv->segments list is corrupted
2010-04-01 17:08:06 +04:00
/* Avoid endless loop if lv->segments list is corrupt */
if (!r)
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
goto out;
Avoid endless loop if lv->segments list is corrupted
2010-04-01 17:08:06 +04:00
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (!(vhash.lvname = dm_hash_create(lv_count))) {
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
log_error("Failed to allocate lv_name hash");
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
r = 0;
goto out;
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
}
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (!(vhash.lvid = dm_hash_create(lv_count))) {
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
log_error("Failed to allocate uuid hash");
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
r = 0;
goto out;
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
}
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(lvl, &vg->lvs) {
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (dm_hash_lookup(vhash.lvname, lvl->lv->name)) {
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
log_error(INTERNAL_ERROR
"Duplicate LV name %s detected in %s.",
lvl->lv->name, vg->name);
r = 0;
}
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (dm_hash_lookup_binary(vhash.lvid, &lvl->lv->lvid.id[1],
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
sizeof(lvl->lv->lvid.id[1]))) {
if (!id_write_format(&lvl->lv->lvid.id[1], uuid,
sizeof(uuid)))
stack;
log_error(INTERNAL_ERROR "Duplicate LV id "
"%s detected for %s in %s.",
uuid, lvl->lv->name, vg->name);
r = 0;
Add checks for duplicate LV name, lvid and PV id before writing metadata. Report all sanity check failures, not just the first.
2006-08-09 23:33:25 +04:00
}
Reinstate full PV size when removing from VG. Support loopfiles for testing. Complete the pv_segment support.
2005-05-03 21:28:23 +04:00
Fix new mirror_seg pointer.
2005-10-28 01:51:28 +04:00
if (!check_lv_segments(lvl->lv, 1)) {
#define an INTERNAL_ERROR macro and use it throughout LVM.
2009-12-16 22:22:11 +03:00
log_error(INTERNAL_ERROR "LV segments corrupted in %s.",
Basic support for mirrors.
2005-06-01 20:51:55 +04:00
lvl->lv->name);
Add checks for duplicate LV name, lvid and PV id before writing metadata. Report all sanity check failures, not just the first.
2006-08-09 23:33:25 +04:00
r = 0;
Basic support for mirrors.
2005-06-01 20:51:55 +04:00
}
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (!dm_hash_insert(vhash.lvname, lvl->lv->name, lvl)) {
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
log_error("Failed to hash lvname.");
r = 0;
break;
}
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (!dm_hash_insert_binary(vhash.lvid, &lvl->lv->lvid.id[1],
Missed merge fix in vg_validate patch
2011-03-11 01:39:36 +03:00
sizeof(lvl->lv->lvid.id[1]), lvl->lv)) {
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
log_error("Failed to hash lvid.");
r = 0;
break;
}
Basic support for mirrors.
2005-06-01 20:51:55 +04:00
}
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (!_lv_postorder_vg(vg, _lv_validate_references_single, &vhash)) {
Refactor code for _lv_postoder Add _lv_postorder_vg() - for calling _lv_postorder() for every LV from VG. We use this in 2 places - vg_mark_partial_lvs() and vg_validate() so make it as a one function. Benefit here is - to use only one cleanup code and avoid potentially duplicate scans of same LVs.
2011-03-10 17:40:32 +03:00
stack;
r = 0;
Add further consistency checking to vg_validate, ensuring that all segment areas point to LVs or PVs that are listed in the respective VG.
2010-12-14 20:51:09 +03:00
}
dm_list_iterate_items(lvl, &vg->lvs) {
cleanup: Use lv_is_ macros. Use lv_is_* macros throughout the code base, introducing lv_is_pvmove, lv_is_locked, lv_is_converting and lv_is_merging. lv_is_mirror_type no longer includes pvmove.
2014-09-16 00:33:53 +04:00
if (!lv_is_pvmove(lvl->lv))
Add a validation step for pvmoveN internal LVs to vg_validate.
2010-12-14 20:07:35 +03:00
continue;
dm_list_iterate_items(seg, &lvl->lv->segments) {
if (seg_is_mirrored(seg)) {
if (seg->area_count != 2) {
log_error(INTERNAL_ERROR
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
"Segment in %s is not 2-way.",
lvl->lv->name);
Add a validation step for pvmoveN internal LVs to vg_validate.
2010-12-14 20:07:35 +03:00
r = 0;
}
} else if (seg->area_count != 1) {
log_error(INTERNAL_ERROR
Use hash tables for validating names Accelerate validation loop by using lvname, lvid, pvid hash tables. Also merge pvl loop into one cycle now - no need to scan the list twice. List scan is stopped when dm_hash_insert fails. The error message with loop_counter1 is no longer provided - however the message has been misleading anyway.
2011-03-10 16:11:59 +03:00
"Segment in %s has wrong number of areas: %d.",
lvl->lv->name, seg->area_count);
Add a validation step for pvmoveN internal LVs to vg_validate.
2010-12-14 20:07:35 +03:00
r = 0;
}
}
}
Fix setting of volume limit count if converting to lvm1 format. Fixes problem when after downconvert to lvm1 VG is broken: # lvcreate -n lv1 -l 4 vg_test Invalid LV in extent map (PV /dev/sdb1, PE 0, LV 0, LE 0) ...
2008-08-29 17:41:21 +04:00
if (!(vg->fid->fmt->features & FMT_UNLIMITED_VOLS) &&
(!vg->max_lv || !vg->max_pv)) {
#define an INTERNAL_ERROR macro and use it throughout LVM.
2009-12-16 22:22:11 +03:00
log_error(INTERNAL_ERROR "Volume group %s has limited PV/LV count"
Fix setting of volume limit count if converting to lvm1 format. Fixes problem when after downconvert to lvm1 VG is broken: # lvcreate -n lv1 -l 4 vg_test Invalid LV in extent map (PV /dev/sdb1, PE 0, LV 0, LE 0) ...
2008-08-29 17:41:21 +04:00
" but limit is not set.", vg->name);
r = 0;
}
thin: add pool metadata spare lv support Add support for pool's metadata spare volume.
2013-07-05 19:10:47 +04:00
if (vg->pool_metadata_spare_lv &&
!lv_is_pool_metadata_spare(vg->pool_metadata_spare_lv)) {
log_error(INTERNAL_ERROR "VG references non pool metadata spare LV %s.",
vg->pool_metadata_spare_lv->name);
r = 0;
}
Check max_lv on only place and force the check only for new volume. We can temporarily violate max_lv during mirror conversion etc. (If the operation fails, orphan mirror images are visible to administrator for manual remove for example. Not that this should ever happen:-) Force limit only for lvcreate (and vg merge) command. Patch also adds simple max_lv tests into testsuite
2009-05-14 01:29:10 +04:00
if (vg_max_lv_reached(vg))
stack;
Add lvmlockd
2015-03-05 23:00:44 +03:00
if (!(vhash.lv_lock_args = dm_hash_create(lv_count))) {
log_error("Failed to allocate lv_lock_args hash");
r = 0;
goto out;
}
lvmlockd: primarily use vg_is_shared to check if a vg uses an lvmlockd lock_type, instead of the equivalent but longer is_lockd_type.
2018-06-01 18:04:54 +03:00
if (vg_is_shared(vg)) {
Add lvmlockd
2015-03-05 23:00:44 +03:00
if (!vg->lock_args) {
log_error(INTERNAL_ERROR "VG %s with lock_type %s without lock_args",
vg->name, vg->lock_type);
r = 0;
}
if (vg_is_clustered(vg)) {
log_error(INTERNAL_ERROR "VG %s with lock_type %s is clustered",
vg->name, vg->lock_type);
r = 0;
}
if (vg->system_id && vg->system_id[0]) {
log_error(INTERNAL_ERROR "VG %s with lock_type %s has system_id %s",
vg->name, vg->lock_type, vg->system_id);
r = 0;
}
if (strcmp(vg->lock_type, "sanlock") && strcmp(vg->lock_type, "dlm")) {
log_error(INTERNAL_ERROR "VG %s has unknown lock_type %s",
vg->name, vg->lock_type);
r = 0;
}
metadata: vg_validate lock_args
2015-07-09 21:24:28 +03:00
lockd: disable part of lock_args validation There are at least a couple instances where the lock_args check does not work correctly, (listed in the comment), so disable the NULL check for lock_args until those are resolved.
2015-07-10 19:41:29 +03:00
if (!_validate_vg_lock_args(vg))
metadata: vg_validate lock_args
2015-07-09 21:24:28 +03:00
r = 0;
Add lvmlockd
2015-03-05 23:00:44 +03:00
} else {
if (vg->lock_args) {
log_error(INTERNAL_ERROR "VG %s has lock_args %s without lock_type",
vg->name, vg->lock_args);
r = 0;
}
}
dm_list_iterate_items(lvl, &vg->lvs) {
lvmlockd: primarily use vg_is_shared to check if a vg uses an lvmlockd lock_type, instead of the equivalent but longer is_lockd_type.
2018-06-01 18:04:54 +03:00
if (vg_is_shared(vg)) {
Add lvmlockd
2015-03-05 23:00:44 +03:00
if (lockd_lv_uses_lock(lvl->lv)) {
metadata: vg_validate lock_args
2015-07-09 21:24:28 +03:00
if (vg->skip_validate_lock_args)
Add lvmlockd
2015-03-05 23:00:44 +03:00
continue;
metadata: vg_validate lock_args
2015-07-09 21:24:28 +03:00
lockd: disable part of lock_args validation There are at least a couple instances where the lock_args check does not work correctly, (listed in the comment), so disable the NULL check for lock_args until those are resolved.
2015-07-10 19:41:29 +03:00
/*
* FIXME: make missing lock_args an error.
* There are at least two cases where this
* check doesn't work correctly:
*
* 1. When creating a cow snapshot,
* (lvcreate -s -L1M -n snap1 vg/lv1),
* lockd_lv_uses_lock() uses lv_is_cow()
* which depends on lv->snapshot being
* set, but it's not set at this point,
* so lockd_lv_uses_lock() cannot identify
* the LV as a cow_lv, and thinks it needs
* a lock when it doesn't. To fix this we
* probably need to validate by finding the
* origin LV, then finding all its snapshots
* which will have no lock_args.
*
* 2. When converting an LV to a thin pool
* without using an existing metadata LV,
* (lvconvert --type thin-pool vg/poolX),
* there is an intermediate LV created,
* probably for the metadata LV, and
* validate is called on the VG in this
* intermediate state, which finds the
* newly created LV which is not yet
* identified as a metadata LV, and
* does not have any lock_args. To fix
* this we might be able to find the place
* where the intermediate LV is created,
* and set new variable on it like for vgs,
* lv->skip_validate_lock_args.
*/
metadata: vg_validate lock_args
2015-07-09 21:24:28 +03:00
if (!lvl->lv->lock_args) {
lockd: disable part of lock_args validation There are at least a couple instances where the lock_args check does not work correctly, (listed in the comment), so disable the NULL check for lock_args until those are resolved.
2015-07-10 19:41:29 +03:00
/*
log_verbose("LV %s/%s missing lock_args",
vg->name, lvl->lv->name);
Add lvmlockd
2015-03-05 23:00:44 +03:00
r = 0;
lockd: disable part of lock_args validation There are at least a couple instances where the lock_args check does not work correctly, (listed in the comment), so disable the NULL check for lock_args until those are resolved.
2015-07-10 19:41:29 +03:00
*/
metadata: vg_validate lock_args
2015-07-09 21:24:28 +03:00
continue;
}
if (!_validate_lv_lock_args(lvl->lv)) {
r = 0;
continue;
}
if (!strcmp(vg->lock_type, "sanlock")) {
Add lvmlockd
2015-03-05 23:00:44 +03:00
if (dm_hash_lookup(vhash.lv_lock_args, lvl->lv->lock_args)) {
log_error(INTERNAL_ERROR "LV %s/%s has duplicate lock_args %s.",
vg->name, lvl->lv->name, lvl->lv->lock_args);
r = 0;
}
if (!dm_hash_insert(vhash.lv_lock_args, lvl->lv->lock_args, lvl)) {
log_error("Failed to hash lvname.");
r = 0;
}
}
} else {
lvmlockd: fix cachevol locking When a cachevol LV is attached, have the LV keep it's lock allocated. The lock on the cachevol won't be used while it's attached. When the cachevol is split a new lock does not need to be allocated. (Applies to cachevol usage by both dm-cache and dm-writecache.)
2019-10-25 01:09:00 +03:00
if (lv_is_cache_vol(lvl->lv)) {
log_debug("lock_args will be ignored on cache vol");
} else if (lvl->lv->lock_args) {
Add lvmlockd
2015-03-05 23:00:44 +03:00
log_error(INTERNAL_ERROR "LV %s/%s shouldn't have lock_args",
vg->name, lvl->lv->name);
r = 0;
}
}
} else {
if (lvl->lv->lock_args) {
log_error(INTERNAL_ERROR "LV %s/%s with no lock_type has lock_args %s",
vg->name, lvl->lv->name, lvl->lv->lock_args);
r = 0;
}
}
}
metadata: also validate historical LVs in VG in vg_validate and check_lv_segments
2016-03-01 17:32:01 +03:00
if (!(vhash.historical_lvname = dm_hash_create(dm_list_size(&vg->historical_lvs)))) {
log_error("Failed to allocate historical LV name hash");
r = 0;
goto out;
}
if (!(vhash.historical_lvid = dm_hash_create(dm_list_size(&vg->historical_lvs)))) {
log_error("Failed to allocate historical LV uuid hash");
r = 0;
goto out;
}
dm_list_iterate_items(glvl, &vg->historical_lvs) {
if (!glvl->glv->is_historical) {
log_error(INTERNAL_ERROR "LV %s/%s appearing in VG's historical list is not a historical LV",
vg->name, glvl->glv->live->name);
r = 0;
continue;
}
hlv = glvl->glv->historical;
if (hlv->vg != vg) {
log_error(INTERNAL_ERROR "Historical LV %s points to different VG %s while it is listed in VG %s",
hlv->name, hlv->vg->name, vg->name);
r = 0;
continue;
}
if (!id_equal(&hlv->lvid.id[0], &hlv->vg->id)) {
if (!id_write_format(&hlv->lvid.id[0], uuid, sizeof(uuid)))
stack;
if (!id_write_format(&hlv->vg->id, uuid2, sizeof(uuid2)))
stack;
log_error(INTERNAL_ERROR "Historical LV %s has VG UUID %s but its VG %s has UUID %s",
hlv->name, uuid, hlv->vg->name, uuid2);
r = 0;
continue;
}
if (dm_hash_lookup_binary(vhash.historical_lvid, &hlv->lvid.id[1], sizeof(hlv->lvid.id[1]))) {
if (!id_write_format(&hlv->lvid.id[1], uuid,sizeof(uuid)))
stack;
log_error(INTERNAL_ERROR "Duplicate historical LV id %s detected for %s in %s",
uuid, hlv->name, vg->name);
r = 0;
}
if (dm_hash_lookup(vhash.historical_lvname, hlv->name)) {
log_error(INTERNAL_ERROR "Duplicate historical LV name %s detected in %s", hlv->name, vg->name);
r = 0;
continue;
}
if (!dm_hash_insert(vhash.historical_lvname, hlv->name, hlv)) {
log_error("Failed to hash historical LV name");
r = 0;
break;
}
if (!dm_hash_insert_binary(vhash.historical_lvid, &hlv->lvid.id[1], sizeof(hlv->lvid.id[1]), hlv)) {
log_error("Failed to hash historical LV id");
r = 0;
break;
}
if (dm_hash_lookup(vhash.lvname, hlv->name)) {
log_error(INTERNAL_ERROR "Name %s appears as live and historical LV at the same time in VG %s",
hlv->name, vg->name);
r = 0;
continue;
}
if (!hlv->indirect_origin && !dm_list_size(&hlv->indirect_glvs)) {
log_error(INTERNAL_ERROR "Historical LV %s is not part of any LV chain in VG %s", hlv->name, vg->name);
r = 0;
continue;
}
}
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
out:
if (vhash.lvid)
dm_hash_destroy(vhash.lvid);
if (vhash.lvname)
dm_hash_destroy(vhash.lvname);
metadata: also validate historical LVs in VG in vg_validate and check_lv_segments
2016-03-01 17:32:01 +03:00
if (vhash.historical_lvid)
dm_hash_destroy(vhash.historical_lvid);
if (vhash.historical_lvname)
dm_hash_destroy(vhash.historical_lvname);
Use created hash tables for quick check of LV, PV. Instead of searching linear list of all LVs, PVs - use created hash tables also for quick mapping between LV. (Note - for small number of PVs or LVs the overhead of the hash is bigger). TODO: Use hash tables in volume_group structure directly.
2011-03-30 17:35:51 +04:00
if (vhash.pvid)
dm_hash_destroy(vhash.pvid);
Add lvmlockd
2015-03-05 23:00:44 +03:00
if (vhash.lv_lock_args)
dm_hash_destroy(vhash.lv_lock_args);
Fix lv_count when manipulating with snapshots and max_lv is set. Patch fixes these problems: - during the snapshot creation process, it needs create 2 LVs, one is cow, second becomes snapshot. If the code fails in vg_add_snapshot, code lvcreate will not remove LV cow volume. - if max_lv is set and VG contains snapshot, it can happen that during the activation lv_count is temporarily increased over the limit and VG metadata are not properly processed see https://bugzilla.redhat.com/show_bug.cgi?id=490298 - vgcfgrestore alows restore with max_lv set to lower valuer that actual LV count. This later leads to situation that max_lv is completely ignored. - vgck doesn't call vg_validate(). It should at least try:-) Signed-off-by: Milan Broz <mbroz@redhat.com>
2009-03-16 17:34:57 +03:00
Add checks for duplicate LV name, lvid and PV id before writing metadata. Report all sanity check failures, not just the first.
2006-08-09 23:33:25 +04:00
return r;
Fix pvmove segment splitting. Abstract vg_validate.
2005-07-12 23:40:59 +04:00
}
metadata: use pv_write_list for _check_old_pv_ext_for_vg The _check_old_pv_ext_for_vg() function only needs to do pv_write(), so it can use the simpler pv_list structs on the pv_write_list.
2016-02-16 22:02:00 +03:00
static int _pv_in_pv_list(struct physical_volume *pv, struct dm_list *head)
vg: automatically update to newest PV ext version during vg_write
2016-02-12 16:22:02 +03:00
{
metadata: use pv_write_list for _check_old_pv_ext_for_vg The _check_old_pv_ext_for_vg() function only needs to do pv_write(), so it can use the simpler pv_list structs on the pv_write_list.
2016-02-16 22:02:00 +03:00
struct pv_list *pvl;
vg: automatically update to newest PV ext version during vg_write
2016-02-12 16:22:02 +03:00
metadata: use pv_write_list for _check_old_pv_ext_for_vg The _check_old_pv_ext_for_vg() function only needs to do pv_write(), so it can use the simpler pv_list structs on the pv_write_list.
2016-02-16 22:02:00 +03:00
dm_list_iterate_items(pvl, head) {
if (pvl->pv == pv)
vg: automatically update to newest PV ext version during vg_write
2016-02-12 16:22:02 +03:00
return 1;
}
return 0;
}
metadata: automatically remove invalid (dangling) historical LVs Historical LV is valid as long as there is at least one live LV among its ancestors. If we find any invalid (dangling) historical LVs, remove them automatically.
2016-03-02 14:19:07 +03:00
static int _check_historical_lv_is_valid(struct historical_logical_volume *hlv)
metadata: create historical LVs when LVs are removed and interconnect with live LVs When an LV is being removed, we create an instance of "struct historical_logical_volume" wrapped up in "struct generic_logical_volume". All instances of "struct historical_logical_volume" are then recorded in "historical_lvs" list which is part of "struct volume_group". The "historical LV" is then interconnected with "live LVs" to connect a history chain for the live LV.
2016-03-01 17:20:09 +03:00
{
struct glv_list *glvl;
metadata: automatically remove invalid (dangling) historical LVs Historical LV is valid as long as there is at least one live LV among its ancestors. If we find any invalid (dangling) historical LVs, remove them automatically.
2016-03-02 14:19:07 +03:00
if (hlv->checked)
return hlv->valid;
/*
* Historical LV is valid if there is
* at least one live LV among ancestors.
*/
hlv->valid = 0;
dm_list_iterate_items(glvl, &hlv->indirect_glvs) {
if (!glvl->glv->is_historical ||
_check_historical_lv_is_valid(glvl->glv->historical)) {
hlv->valid = 1;
break;
}
}
hlv->checked = 1;
return hlv->valid;
}
static int _handle_historical_lvs(struct volume_group *vg)
{
struct glv_list *glvl, *tglvl;
metadata: create historical LVs when LVs are removed and interconnect with live LVs When an LV is being removed, we create an instance of "struct historical_logical_volume" wrapped up in "struct generic_logical_volume". All instances of "struct historical_logical_volume" are then recorded in "historical_lvs" list which is part of "struct volume_group". The "historical LV" is then interconnected with "live LVs" to connect a history chain for the live LV.
2016-03-01 17:20:09 +03:00
time_t current_timestamp = 0;
struct historical_logical_volume *hlv;
metadata: automatically remove invalid (dangling) historical LVs Historical LV is valid as long as there is at least one live LV among its ancestors. If we find any invalid (dangling) historical LVs, remove them automatically.
2016-03-02 14:19:07 +03:00
int valid = 1;
dm_list_iterate_items(glvl, &vg->historical_lvs)
glvl->glv->historical->checked = 0;
metadata: create historical LVs when LVs are removed and interconnect with live LVs When an LV is being removed, we create an instance of "struct historical_logical_volume" wrapped up in "struct generic_logical_volume". All instances of "struct historical_logical_volume" are then recorded in "historical_lvs" list which is part of "struct volume_group". The "historical LV" is then interconnected with "live LVs" to connect a history chain for the live LV.
2016-03-01 17:20:09 +03:00
dm_list_iterate_items(glvl, &vg->historical_lvs) {
hlv = glvl->glv->historical;
metadata: automatically remove invalid (dangling) historical LVs Historical LV is valid as long as there is at least one live LV among its ancestors. If we find any invalid (dangling) historical LVs, remove them automatically.
2016-03-02 14:19:07 +03:00
valid &= _check_historical_lv_is_valid(hlv);
metadata: create historical LVs when LVs are removed and interconnect with live LVs When an LV is being removed, we create an instance of "struct historical_logical_volume" wrapped up in "struct generic_logical_volume". All instances of "struct historical_logical_volume" are then recorded in "historical_lvs" list which is part of "struct volume_group". The "historical LV" is then interconnected with "live LVs" to connect a history chain for the live LV.
2016-03-01 17:20:09 +03:00
if (!hlv->timestamp_removed) {
if (!current_timestamp)
current_timestamp = time(NULL);
hlv->timestamp_removed = (uint64_t) current_timestamp;
}
}
metadata: automatically remove invalid (dangling) historical LVs Historical LV is valid as long as there is at least one live LV among its ancestors. If we find any invalid (dangling) historical LVs, remove them automatically.
2016-03-02 14:19:07 +03:00
if (valid)
return 1;
dm_list_iterate_items_safe(glvl, tglvl, &vg->historical_lvs) {
hlv = glvl->glv->historical;
if (hlv->checked && hlv->valid)
continue;
log_print_unless_silent("Automatically removing historical "
"logical volume %s/%s%s.",
vg->name, HISTORICAL_LV_PREFIX, hlv->name);
if (!historical_glv_remove(glvl->glv))
return_0;
}
metadata: create historical LVs when LVs are removed and interconnect with live LVs When an LV is being removed, we create an instance of "struct historical_logical_volume" wrapped up in "struct generic_logical_volume". All instances of "struct historical_logical_volume" are then recorded in "historical_lvs" list which is part of "struct volume_group". The "historical LV" is then interconnected with "live LVs" to connect a history chain for the live LV.
2016-03-01 17:20:09 +03:00
return 1;
}
move wipe_outdated_pvs to vg_write and implement it based on a device, not based on a pv struct (which is not available when the device is not a part of the vg.) currently only the vgremove command wipes outdated pvs until more advanced recovery is added in a subsequent commit
2019-02-06 21:32:26 +03:00
static void _wipe_outdated_pvs(struct cmd_context *cmd, struct volume_group *vg)
{
struct dm_list devs;
struct dm_list *mdas = NULL;
struct device_list *devl;
struct device *dev;
struct metadata_area *mda;
struct label *label;
struct lvmcache_info *info;
uint32_t ext_flags;
dm_list_init(&devs);
/*
* When vg_read selected a good copy of the metadata, it used it to
* update the lvmcache representation of the VG (lvmcache_update_vg).
* At that point outdated PVs were recognized and moved into the
* vginfo->outdated_infos list. Here we clear the PVs on that list.
*/
lvmcache_get_outdated_devs(cmd, vg->name, (const char *)&vg->id, &devs);
dm_list_iterate_items(devl, &devs) {
dev = devl->dev;
lvmcache_get_outdated_mdas(cmd, vg->name, (const char *)&vg->id, dev, &mdas);
if (mdas) {
dm_list_iterate_items(mda, mdas) {
log_warn("WARNING: wiping mda on outdated PV %s", dev_name(dev));
if (!text_wipe_outdated_pv_mda(cmd, dev, mda))
log_warn("WARNING: failed to wipe mda on outdated PV %s", dev_name(dev));
}
}
if (!(label = lvmcache_get_dev_label(dev))) {
log_error("_wipe_outdated_pvs no label for %s", dev_name(dev));
continue;
}
info = label->info;
ext_flags = lvmcache_ext_flags(info);
ext_flags &= ~PV_EXT_USED;
lvmcache_set_ext_version(info, PV_HEADER_EXTENSION_VSN);
lvmcache_set_ext_flags(info, ext_flags);
log_warn("WARNING: wiping header on outdated PV %s", dev_name(dev));
if (!label_write(dev, label))
log_warn("WARNING: failed to wipe header on outdated PV %s", dev_name(dev));
lvmcache_del(info);
}
/*
* A vgremove will involve many vg_write() calls (one for each lv
* removed) but we only need to wipe pvs once, so clear the outdated
* list so it won't be wiped again.
*/
lvmcache_del_outdated_devs(cmd, vg->name, (const char *)&vg->id);
}
Fix pvmove segment splitting. Abstract vg_validate.
2005-07-12 23:40:59 +04:00
/*
* After vg_write() returns success,
* caller MUST call either vg_commit() or vg_revert()
*/
int vg_write(struct volume_group *vg)
{
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
struct dm_list *mdah;
move pv header repairs to vg_write Correct PV header in-use or version fields from vg_write instead of vg_read.
2019-02-06 22:18:45 +03:00
struct pv_list *pvl, *pvl_safe, *new_pvl;
Fix pvmove segment splitting. Abstract vg_validate.
2005-07-12 23:40:59 +04:00
struct metadata_area *mda;
Add lvmlockd
2015-03-05 23:00:44 +03:00
struct lv_list *lvl;
add a warning message when updating old metadata in an mda that had previously not been updated
2019-02-06 22:51:54 +03:00
struct device *mda_dev;
metadata: Make it possible to write partial VGs obtained from lvmetad.
2014-05-26 16:23:33 +04:00
int revert = 0, wrote = 0;
Fix pvmove segment splitting. Abstract vg_validate.
2005-07-12 23:40:59 +04:00
lvmlockd: clarify lock_type use for coverity Make it clearer when vg->lock_type will be used so coverity doesn't worry about it.
2018-06-01 18:12:04 +03:00
if (vg_is_shared(vg)) {
dm_list_iterate_items(lvl, &vg->lvs) {
if (lvl->lv->lock_args && !strcmp(lvl->lv->lock_args, "pending")) {
if (!lockd_init_lv_args(vg->cmd, vg, lvl->lv, vg->lock_type, &lvl->lv->lock_args)) {
log_error("Cannot allocate lock for new LV.");
return 0;
}
lvl->lv->new_lock_args = 1;
Add lvmlockd
2015-03-05 23:00:44 +03:00
}
}
}
metadata: create historical LVs when LVs are removed and interconnect with live LVs When an LV is being removed, we create an instance of "struct historical_logical_volume" wrapped up in "struct generic_logical_volume". All instances of "struct historical_logical_volume" are then recorded in "historical_lvs" list which is part of "struct volume_group". The "historical LV" is then interconnected with "live LVs" to connect a history chain for the live LV.
2016-03-01 17:20:09 +03:00
if (!_handle_historical_lvs(vg)) {
log_error("Failed to handle historical LVs in VG %s.", vg->name);
return 0;
}
Use stack return macros throughout.
2008-01-30 16:19:47 +03:00
if (!vg_validate(vg))
return_0;
Fix pvmove segment splitting. Abstract vg_validate.
2005-07-12 23:40:59 +04:00
Some partial VG support with format_text.
2002-04-30 21:12:37 +04:00
if (vg->status & PARTIAL_VG) {
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
log_error("Cannot update partial volume group %s.", vg->name);
return 0;
}
if (vg_missing_pv_count(vg) && !vg->cmd->handles_missing_pvs) {
log_error("Cannot update volume group %s while physical "
"volumes are missing.", vg->name);
Some partial VG support with format_text.
2002-04-30 21:12:37 +04:00
return 0;
}
lvmcache: renaming functions and variables related to duplicates, no functional changes.
2019-08-01 21:50:04 +03:00
if (lvmcache_has_duplicate_devs() && vg_has_duplicate_pvs(vg) &&
lvmcache: improve duplicate PV handling Wait to compare and choose alternate duplicate devices until after all devices are scanned. During scanning, the first duplicate dev is kept in lvmcache, and others are kept in a new list (_found_duplicate_devs). After all devices are scanned, compare all the duplicates available for a given PVID and decide which is best. If the dev used in lvmcache is changed, drop the old dev from lvmcache entirely and rescan the replacement dev. Previously the VG metadata from the old dev was kept in lvmcache and only the dev was replaced. A new config setting devices/allow_changes_with_duplicate_pvs can be set to 0 which disallows modifying a VG or activating LVs in it when the VG contains PVs with duplicate devices. Set to 1 is the old behavior which allowed the VG to be changed. The logic for which of two devs is preferred has changed. The primary goal is to choose a device that is currently in use if the other isn't, e.g. by an active LV. . prefer dev with fs mounted if the other doesn't, else . prefer dev that is dm if the other isn't, else . prefer dev in subsystem if the other isn't If neither device is preferred by these rules, then don't change devices in lvmcache, leaving the one that was found first. The previous logic for preferring a device was: . prefer dev in subsystem if the other isn't, else . prefer dev without holders if the other has holders, else . prefer dev that is dm if the other isn't
2016-02-09 22:06:27 +03:00
!find_config_tree_bool(vg->cmd, devices_allow_changes_with_duplicate_pvs_CFG, NULL)) {
log_error("Cannot update volume group %s with duplicate PV devices.",
vg->name);
return 0;
}
Handle metadata with unknown segment types more gracefully.
2009-10-16 21:41:49 +04:00
if (vg_has_unknown_segments(vg) && !vg->cmd->handles_unknown_segments) {
log_error("Cannot update volume group %s with unknown segments in it!",
vg->name);
return 0;
}
Remove lvm1 and pool disk formats There are likely more bits of code that can be removed, e.g. lvm1/pool-specific bits of code that were identified using FMT flags. The vgconvert command can likely be reduced further. The lvm1-specific config settings should probably have some other fields set for proper deprecation.
2018-04-28 00:22:46 +03:00
if (!_vg_adjust_ignored_mdas(vg))
Improve various log messages.
2010-06-30 17:51:11 +04:00
return_0;
Handle metadata with unknown segment types more gracefully.
2009-10-16 21:41:49 +04:00
more mda ignore cleanups
2010-06-30 23:28:35 +04:00
if (!vg_mda_used_count(vg)) {
Some new features.
2002-11-18 17:04:08 +03:00
log_error("Aborting vg_write: No metadata areas to write to!");
return 0;
}
move wipe_outdated_pvs to vg_write and implement it based on a device, not based on a pv struct (which is not available when the device is not a part of the vg.) currently only the vgremove command wipes outdated pvs until more advanced recovery is added in a subsequent commit
2019-02-06 21:32:26 +03:00
if (vg->cmd->wipe_outdated_pvs)
_wipe_outdated_pvs(vg->cmd, vg);
Unlock memory for vg_write For write we do not need to hold memory locked. This relaxes many conditions and avoid problems when allocating a lot of memory for writting metadata buffers. (In case of huge MDA size this would lead to mismatch between locked and unlocked memory region size). Add also internal check we are not writing in critical section.
2011-11-18 23:28:00 +04:00
if (critical_section())
log_error(INTERNAL_ERROR
"Writing metadata in critical section.");
/* Unlock memory if possible */
memlock_unlock(vg->cmd);
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
vg->seqno++;
move pv header repairs to vg_write Correct PV header in-use or version fields from vg_write instead of vg_read.
2019-02-06 22:18:45 +03:00
dm_list_iterate_items(pvl, &vg->pvs) {
int update_pv_header = 0;
if (_pv_in_pv_list(pvl->pv, &vg->pv_write_list))
continue;
if (!pvl->pv->fmt->ops->pv_needs_rewrite(pvl->pv->fmt, pvl->pv, &update_pv_header))
continue;
if (!update_pv_header)
continue;
if (!(new_pvl = dm_pool_zalloc(vg->vgmem, sizeof(*new_pvl))))
continue;
new_pvl->pv = pvl->pv;
dm_list_add(&vg->pv_write_list, &new_pvl->list);
log_warn("WARNING: updating PV header on %s for VG %s.", pv_dev_name(pvl->pv), vg->name);
}
vg_write: add list of pvs to write The vg->pv_write_list contains pv_list structs for which vg_write() should call pv_write(). The new list will replace vg->pvs_to_write that contains vg_to_create structs which are used to perform higher-level pvcreate-related operations. The higher level pvcreate operations will be moved out of vg_write() to higher levels.
2016-02-16 21:43:24 +03:00
dm_list_iterate_items_safe(pvl, pvl_safe, &vg->pv_write_list) {
if (!pv_write(vg->cmd, pvl->pv, 1))
return_0;
dm_list_del(&pvl->list);
}
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
/* Write to each copy of the metadata area */
Rename fid->metadata_areas to fid->metadata_areas_in_use. Rename the metadata_areas list to an 'in_use' list to prepare for future 'ignored' list.
2010-06-29 00:32:44 +04:00
dm_list_iterate_items(mda, &vg->fid->metadata_areas_in_use) {
add a warning message when updating old metadata in an mda that had previously not been updated
2019-02-06 22:51:54 +03:00
mda_dev = mda_get_device(mda);
metadata: Consistently skip metadata areas that failed. Even after writing some metadata encountered problems, some commands continue (rightly or wrongly) and attempt to make further changes. Once an mda is marked MDA_FAILED, don't try to use it again. This also applies when reverting, where one loop already skips failed mdas but the other doesn't. This fixes some device open_count warnings on relevant failure paths.
2017-12-12 20:49:35 +03:00
if (mda->status & MDA_FAILED)
continue;
add a warning message when updating old metadata in an mda that had previously not been updated
2019-02-06 22:51:54 +03:00
/*
* When the scan and vg_read find old metadata in an mda, they
* leave the info struct in lvmcache, and leave the mda in
* info->mdas. That means we use the mda here to write new
* metadata into. This means that a command writing a VG will
* automatically update old metadata to the latest.
*
* This can also happen if the metadata was ignored on this
* dev, and then it's later changed to not ignored, and
* we see the old metadata.
*/
if (lvmcache_has_old_metadata(vg->cmd, vg->name, (const char *)&vg->id, mda_dev)) {
log_warn("WARNING: updating old metadata to %u on %s for VG %s.",
vg->seqno, dev_name(mda_dev), vg->name);
}
indent
2004-03-27 00:07:30 +03:00
if (!mda->ops->vg_write) {
Cope better when format functions are missing.
2003-08-27 01:12:06 +04:00
log_error("Format does not support writing volume"
"group metadata areas");
metadata: Make it possible to write partial VGs obtained from lvmetad.
2014-05-26 16:23:33 +04:00
revert = 1;
break;
Cope better when format functions are missing.
2003-08-27 01:12:06 +04:00
}
add a warning message when updating old metadata in an mda that had previously not been updated
2019-02-06 22:51:54 +03:00
Some new features.
2002-11-18 17:04:08 +03:00
if (!mda->ops->vg_write(vg->fid, vg, mda)) {
metadata: Make it possible to write partial VGs obtained from lvmetad.
2014-05-26 16:23:33 +04:00
if (vg->cmd->handles_missing_pvs) {
metadata: add "Failed to write VG <vg_name>." on failed vg_write and revert previous patch Better than previous patch which changed log_warn to log_error - we can have multiple MDAs and if one of them fails to be written, we can still continue with other MDAs if we're in a mode where we can handle missing PVs - so keep the log_warn for single failed MDA write as it was before. However, add log_error with "Failed to write VG <vg_name>." in case we're not handling missing PVs or no MDA was written at all during VG write process. This also prevents an internal error in which the vg_write fails and we're not issuing any other log_error in vg_write caller or above, so we end up with: "Internal error: Failed command did not use log_error".
2015-01-09 16:04:44 +03:00
log_warn("WARNING: Failed to write an MDA of VG %s.", vg->name);
metadata: Make it possible to write partial VGs obtained from lvmetad.
2014-05-26 16:23:33 +04:00
mda->status |= MDA_FAILED;
} else {
stack;
revert = 1;
break;
}
} else
++ wrote;
}
Basic support for mirrors.
2005-06-01 20:51:55 +04:00
metadata: Make it possible to write partial VGs obtained from lvmetad.
2014-05-26 16:23:33 +04:00
if (revert || !wrote) {
metadata: add "Failed to write VG <vg_name>." on failed vg_write and revert previous patch Better than previous patch which changed log_warn to log_error - we can have multiple MDAs and if one of them fails to be written, we can still continue with other MDAs if we're in a mode where we can handle missing PVs - so keep the log_warn for single failed MDA write as it was before. However, add log_error with "Failed to write VG <vg_name>." in case we're not handling missing PVs or no MDA was written at all during VG write process. This also prevents an internal error in which the vg_write fails and we're not issuing any other log_error in vg_write caller or above, so we end up with: "Internal error: Failed command did not use log_error".
2015-01-09 16:04:44 +03:00
log_error("Failed to write VG %s.", vg->name);
metadata: Make it possible to write partial VGs obtained from lvmetad.
2014-05-26 16:23:33 +04:00
dm_list_uniterate(mdah, &vg->fid->metadata_areas_in_use, &mda->list) {
mda = dm_list_item(mdah, struct metadata_area);
metadata: Consistently skip metadata areas that failed. Even after writing some metadata encountered problems, some commands continue (rightly or wrongly) and attempt to make further changes. Once an mda is marked MDA_FAILED, don't try to use it again. This also applies when reverting, where one loop already skips failed mdas but the other doesn't. This fixes some device open_count warnings on relevant failure paths.
2017-12-12 20:49:35 +03:00
if (mda->status & MDA_FAILED)
continue;
metadata: Make it possible to write partial VGs obtained from lvmetad.
2014-05-26 16:23:33 +04:00
if (mda->ops->vg_revert &&
!mda->ops->vg_revert(vg->fid, vg, mda)) {
stack;
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
}
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
}
metadata: Make it possible to write partial VGs obtained from lvmetad.
2014-05-26 16:23:33 +04:00
return 0;
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
}
Move from 2-step to 3-step on-disk metadata commit.
2005-04-06 22:59:55 +04:00
/* Now pre-commit each copy of the new metadata */
Rename fid->metadata_areas to fid->metadata_areas_in_use. Rename the metadata_areas list to an 'in_use' list to prepare for future 'ignored' list.
2010-06-29 00:32:44 +04:00
dm_list_iterate_items(mda, &vg->fid->metadata_areas_in_use) {
metadata: Make it possible to write partial VGs obtained from lvmetad.
2014-05-26 16:23:33 +04:00
if (mda->status & MDA_FAILED)
continue;
Move from 2-step to 3-step on-disk metadata commit.
2005-04-06 22:59:55 +04:00
if (mda->ops->vg_precommit &&
!mda->ops->vg_precommit(vg->fid, vg, mda)) {
stack;
/* Revert */
Rename fid->metadata_areas to fid->metadata_areas_in_use. Rename the metadata_areas list to an 'in_use' list to prepare for future 'ignored' list.
2010-06-29 00:32:44 +04:00
dm_list_iterate_items(mda, &vg->fid->metadata_areas_in_use) {
metadata: Make it possible to write partial VGs obtained from lvmetad.
2014-05-26 16:23:33 +04:00
if (mda->status & MDA_FAILED)
continue;
Move from 2-step to 3-step on-disk metadata commit.
2005-04-06 22:59:55 +04:00
if (mda->ops->vg_revert &&
!mda->ops->vg_revert(vg->fid, vg, mda)) {
stack;
}
}
return 0;
}
}
metadata: Eliminate redundant nested VG metadata Only lv_committed() now uses vg->vg_committed and it appears redundant if its contents match the enclosing VG so don't waste cycles creating it when that's known to be true when no write lock is held so the struct won't get modified.
2017-11-14 18:38:55 +03:00
if (!_vg_update_embedded_copy(vg, &vg->vg_precommitted)) /* prepare precommited */
lvmetad: more reuse precommit buffer This patch moves more allocation to vg_write (as started in 8c878438f5a75310d09a1ee035d9caf8e2b9e50b) TODO: relocate also communication. (in-release update)
2014-03-01 01:40:00 +04:00
return_0;
lvmlockd: update VG lock version earlier Have commands send lvmlockd the update message in vg_write instead of vg_commit, so that it's not done while LVs are suspended. If the vg_write is not committed, and the seqno sent to lvmlockd is not used, then lvmlockd can detect this when the next update uses the same seqno.
2015-12-16 01:14:49 +03:00
lockd_vg_update(vg);
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
return 1;
}
Refactor vg_commit() to add _vg_commit_mdas(). Factor out calling mda->ops->vg_commit() for each mda. No functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:35:33 +04:00
static int _vg_commit_mdas(struct volume_group *vg)
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
{
Before committing each mda, arrange mdas so ignored mdas get committed first. Arrange mdas so mdas that are to be ignored come first. This is an optimization that ensures consistency on disk for the longest period of time. This was noted by agk in review of the v4 patchset of pvchange-based mda balance. Note the following example for an explanation of the background: Assume the initial state on disk is as follows: PV0 (v1, non-ignored) PV1 (v1, non-ignored) PV2 (v1, non-ignored) PV3 (v1, non-ignored) If we did not sort the list, we would have a commit sequence something like this: PV0 (v2, non-ignored) PV1 (v2, ignored) PV2 (v2, ignored) PV3 (v2, non-ignored) After the commit of PV0's mdas, we'd have an on-disk state like this: PV0 (v2, non-ignored) PV1 (v1, non-ignored) PV2 (v1, non-ignored) PV3 (v1, non-ignored) This is an inconsistent state of the disk. If the machine fails, the next time it was brought back up, the auto-correct mechanism in vg_read would update the metadata on PV1-PV3. However, if possible we try to avoid inconsistent on-disk states. Clearly, because we did not sort, we have a greater chance of on-disk inconsistency - from the time the commit of PV0 is complete until the time PV3 is complete. We could improve the amount of time the on-disk state is consistent by simply sorting the commit order as follows: PV1 (v2, ignored) PV2 (v2, ignored) PV0 (v2, non-ignored) PV3 (v2, non-ignored) Thus, after the first PV is committed (in this case PV1), on-disk we would have: PV0 (v1, non-ignored) PV1 (v2, ignored) PV2 (v1, non-ignored) PV3 (v1, non-ignored) This is clearly a consistent state. PV1 will be read but the mda will be ignored. All other PVs contain v1 metadata, and no auto-correct will be required. In fact, if we commit all PVs with ignored mdas first, we'll only have an inconsistent state when we start writing non-ignored PVs, and thus the chances we'll get an inconsistent state on disk is much less with the sorted method. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:35:49 +04:00
struct metadata_area *mda, *tmda;
struct dm_list ignored;
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
int failed = 0;
fix vg_commit return value The existing comment was desribing the correct behavior, but the code didn't match. The commit is successful if one mda was committed. Making it depend on the result of the internal lvmcache update was wrong.
2019-02-05 23:02:24 +03:00
int good = 0;
Refactor vg_commit() to add _vg_commit_mdas(). Factor out calling mda->ops->vg_commit() for each mda. No functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:35:33 +04:00
int cache_updated = 0;
Add check to vg_commit() to ensure lock is held before writing new VG metadata.
2008-04-04 19:41:20 +04:00
Before committing each mda, arrange mdas so ignored mdas get committed first. Arrange mdas so mdas that are to be ignored come first. This is an optimization that ensures consistency on disk for the longest period of time. This was noted by agk in review of the v4 patchset of pvchange-based mda balance. Note the following example for an explanation of the background: Assume the initial state on disk is as follows: PV0 (v1, non-ignored) PV1 (v1, non-ignored) PV2 (v1, non-ignored) PV3 (v1, non-ignored) If we did not sort the list, we would have a commit sequence something like this: PV0 (v2, non-ignored) PV1 (v2, ignored) PV2 (v2, ignored) PV3 (v2, non-ignored) After the commit of PV0's mdas, we'd have an on-disk state like this: PV0 (v2, non-ignored) PV1 (v1, non-ignored) PV2 (v1, non-ignored) PV3 (v1, non-ignored) This is an inconsistent state of the disk. If the machine fails, the next time it was brought back up, the auto-correct mechanism in vg_read would update the metadata on PV1-PV3. However, if possible we try to avoid inconsistent on-disk states. Clearly, because we did not sort, we have a greater chance of on-disk inconsistency - from the time the commit of PV0 is complete until the time PV3 is complete. We could improve the amount of time the on-disk state is consistent by simply sorting the commit order as follows: PV1 (v2, ignored) PV2 (v2, ignored) PV0 (v2, non-ignored) PV3 (v2, non-ignored) Thus, after the first PV is committed (in this case PV1), on-disk we would have: PV0 (v1, non-ignored) PV1 (v2, ignored) PV2 (v1, non-ignored) PV3 (v1, non-ignored) This is clearly a consistent state. PV1 will be read but the mda will be ignored. All other PVs contain v1 metadata, and no auto-correct will be required. In fact, if we commit all PVs with ignored mdas first, we'll only have an inconsistent state when we start writing non-ignored PVs, and thus the chances we'll get an inconsistent state on disk is much less with the sorted method. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:35:49 +04:00
/* Rearrange the metadata_areas_in_use so ignored mdas come first. */
dm_list_init(&ignored);
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
dm_list_iterate_items_safe(mda, tmda, &vg->fid->metadata_areas_in_use)
Before committing each mda, arrange mdas so ignored mdas get committed first. Arrange mdas so mdas that are to be ignored come first. This is an optimization that ensures consistency on disk for the longest period of time. This was noted by agk in review of the v4 patchset of pvchange-based mda balance. Note the following example for an explanation of the background: Assume the initial state on disk is as follows: PV0 (v1, non-ignored) PV1 (v1, non-ignored) PV2 (v1, non-ignored) PV3 (v1, non-ignored) If we did not sort the list, we would have a commit sequence something like this: PV0 (v2, non-ignored) PV1 (v2, ignored) PV2 (v2, ignored) PV3 (v2, non-ignored) After the commit of PV0's mdas, we'd have an on-disk state like this: PV0 (v2, non-ignored) PV1 (v1, non-ignored) PV2 (v1, non-ignored) PV3 (v1, non-ignored) This is an inconsistent state of the disk. If the machine fails, the next time it was brought back up, the auto-correct mechanism in vg_read would update the metadata on PV1-PV3. However, if possible we try to avoid inconsistent on-disk states. Clearly, because we did not sort, we have a greater chance of on-disk inconsistency - from the time the commit of PV0 is complete until the time PV3 is complete. We could improve the amount of time the on-disk state is consistent by simply sorting the commit order as follows: PV1 (v2, ignored) PV2 (v2, ignored) PV0 (v2, non-ignored) PV3 (v2, non-ignored) Thus, after the first PV is committed (in this case PV1), on-disk we would have: PV0 (v1, non-ignored) PV1 (v2, ignored) PV2 (v1, non-ignored) PV3 (v1, non-ignored) This is clearly a consistent state. PV1 will be read but the mda will be ignored. All other PVs contain v1 metadata, and no auto-correct will be required. In fact, if we commit all PVs with ignored mdas first, we'll only have an inconsistent state when we start writing non-ignored PVs, and thus the chances we'll get an inconsistent state on disk is much less with the sorted method. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:35:49 +04:00
if (mda_is_ignored(mda))
dm_list_move(&ignored, &mda->list);
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
dm_list_iterate_items_safe(mda, tmda, &ignored)
Before committing each mda, arrange mdas so ignored mdas get committed first. Arrange mdas so mdas that are to be ignored come first. This is an optimization that ensures consistency on disk for the longest period of time. This was noted by agk in review of the v4 patchset of pvchange-based mda balance. Note the following example for an explanation of the background: Assume the initial state on disk is as follows: PV0 (v1, non-ignored) PV1 (v1, non-ignored) PV2 (v1, non-ignored) PV3 (v1, non-ignored) If we did not sort the list, we would have a commit sequence something like this: PV0 (v2, non-ignored) PV1 (v2, ignored) PV2 (v2, ignored) PV3 (v2, non-ignored) After the commit of PV0's mdas, we'd have an on-disk state like this: PV0 (v2, non-ignored) PV1 (v1, non-ignored) PV2 (v1, non-ignored) PV3 (v1, non-ignored) This is an inconsistent state of the disk. If the machine fails, the next time it was brought back up, the auto-correct mechanism in vg_read would update the metadata on PV1-PV3. However, if possible we try to avoid inconsistent on-disk states. Clearly, because we did not sort, we have a greater chance of on-disk inconsistency - from the time the commit of PV0 is complete until the time PV3 is complete. We could improve the amount of time the on-disk state is consistent by simply sorting the commit order as follows: PV1 (v2, ignored) PV2 (v2, ignored) PV0 (v2, non-ignored) PV3 (v2, non-ignored) Thus, after the first PV is committed (in this case PV1), on-disk we would have: PV0 (v1, non-ignored) PV1 (v2, ignored) PV2 (v1, non-ignored) PV3 (v1, non-ignored) This is clearly a consistent state. PV1 will be read but the mda will be ignored. All other PVs contain v1 metadata, and no auto-correct will be required. In fact, if we commit all PVs with ignored mdas first, we'll only have an inconsistent state when we start writing non-ignored PVs, and thus the chances we'll get an inconsistent state on disk is much less with the sorted method. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:35:49 +04:00
dm_list_move(&vg->fid->metadata_areas_in_use, &mda->list);
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
/* Commit to each copy of the metadata area */
Rename fid->metadata_areas to fid->metadata_areas_in_use. Rename the metadata_areas list to an 'in_use' list to prepare for future 'ignored' list.
2010-06-29 00:32:44 +04:00
dm_list_iterate_items(mda, &vg->fid->metadata_areas_in_use) {
metadata: Make it possible to write partial VGs obtained from lvmetad.
2014-05-26 16:23:33 +04:00
if (mda->status & MDA_FAILED)
continue;
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
failed = 0;
Some new features.
2002-11-18 17:04:08 +03:00
if (mda->ops->vg_commit &&
!mda->ops->vg_commit(vg->fid, vg, mda)) {
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
stack;
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
failed = 1;
fix vg_commit return value The existing comment was desribing the correct behavior, but the code didn't match. The commit is successful if one mda was committed. Making it depend on the result of the internal lvmcache update was wrong.
2019-02-05 23:02:24 +03:00
} else
good++;
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
/* Update cache first time we succeed */
if (!failed && !cache_updated) {
create separate lvmcache update functions for read and write The vg read and vg write cases need to update lvmcache differently, so create separate functions for them. The read case now handles checking for outdated mdas and moves them aside into a new list to be repaired in a subsequent commit.
2019-02-06 21:10:13 +03:00
lvmcache_update_vg_from_write(vg);
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
cache_updated = 1;
}
}
fix vg_commit return value The existing comment was desribing the correct behavior, but the code didn't match. The commit is successful if one mda was committed. Making it depend on the result of the internal lvmcache update was wrong.
2019-02-05 23:02:24 +03:00
if (good)
return 1;
return 0;
Refactor vg_commit() to add _vg_commit_mdas(). Factor out calling mda->ops->vg_commit() for each mda. No functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:35:33 +04:00
}
/* Commit pending changes */
int vg_commit(struct volume_group *vg)
{
vgmerge: Fix intermediate metadata corruption vgmerge suffers from a similar problem to the one fixed in commit 8146548d25e9104f0d530d943290d448c1994c0a ("vgsplit: Fix intermediate metadata corruption.") When merging, splitting or renaming VGs, use a new PV status flag PV_MOVED_VG to mark the PVs that hold metadata with the old VG name and use this to provide PV-level granularity instead of incorrectly assuming all PVs in the VG are the same.
2017-10-06 04:12:42 +03:00
struct pv_list *pvl;
fix vg_commit return value The existing comment was desribing the correct behavior, but the code didn't match. The commit is successful if one mda was committed. Making it depend on the result of the internal lvmcache update was wrong.
2019-02-05 23:02:24 +03:00
int ret;
Refactor vg_commit() to add _vg_commit_mdas(). Factor out calling mda->ops->vg_commit() for each mda. No functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:35:33 +04:00
fix vg_commit return value The existing comment was desribing the correct behavior, but the code didn't match. The commit is successful if one mda was committed. Making it depend on the result of the internal lvmcache update was wrong.
2019-02-05 23:02:24 +03:00
ret = _vg_commit_mdas(vg);
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
dbus: add notification from commands When a command modifies a PV or VG, or changes the activation state of an LV, it will send a dbus notification when the command is finished. This can be enabled/disabled with a config setting.
2016-02-22 18:42:03 +03:00
set_vg_notify(vg->cmd);
fix vg_commit return value The existing comment was desribing the correct behavior, but the code didn't match. The commit is successful if one mda was committed. Making it depend on the result of the internal lvmcache update was wrong.
2019-02-05 23:02:24 +03:00
if (ret) {
Do not reset position in metadata ring buffer on vgrename and vgcfgrestore. We should write metadata into next position in the ring buffer while calling vgrename and vgcfgrestore. At this code level (_vg_write_raw), we were not able to determine if this is a rename or not. If yes, then accompanying VG structure passed here has a new name set, not the old one. When looking for a location where to put metadata next, we were given a NULL value because of failed VG name comparison (in _find_vg_rlocn) between the name in existing metadata and metadata we're just about to write. This resets the position in the ring buffer, overwriting any existing metadata (and also incorrectly updates the cache to "orphan" afterwards). This patch just adds old_name item in struct volume_group that we can check and use if necessary and detect renames at lower layers as well. The same applies for vgcfgrestore, but here we're using a special value of old_name, an empty string, to disable the check with existing metadata totally.
2010-04-14 17:09:16 +04:00
/*
* We need to clear old_name after a successful commit.
* The volume_group structure could be reused later.
*/
vg->old_name = NULL;
vgmerge: Fix intermediate metadata corruption vgmerge suffers from a similar problem to the one fixed in commit 8146548d25e9104f0d530d943290d448c1994c0a ("vgsplit: Fix intermediate metadata corruption.") When merging, splitting or renaming VGs, use a new PV status flag PV_MOVED_VG to mark the PVs that hold metadata with the old VG name and use this to provide PV-level granularity instead of incorrectly assuming all PVs in the VG are the same.
2017-10-06 04:12:42 +03:00
dm_list_iterate_items(pvl, &vg->pvs)
pvl->pv->status &= ~PV_MOVED_VG;
Add and track an "ondisk" pointer to struct volume_group. This allows us to get the current on-disk version of the metadata whenever we have the current in-flight version, without a recourse to scanning or lvmcache.
2013-03-17 19:27:44 +04:00
/* This *is* the original now that it's commited. */
metadata: Eliminate redundant nested VG metadata Only lv_committed() now uses vg->vg_committed and it appears redundant if its contents match the enclosing VG so don't waste cycles creating it when that's known to be true when no write lock is held so the struct won't get modified.
2017-11-14 18:38:55 +03:00
_vg_move_cached_precommitted_to_committed(vg);
Do not reset position in metadata ring buffer on vgrename and vgcfgrestore. We should write metadata into next position in the ring buffer while calling vgrename and vgcfgrestore. At this code level (_vg_write_raw), we were not able to determine if this is a rename or not. If yes, then accompanying VG structure passed here has a new name set, not the old one. When looking for a location where to put metadata next, we were given a NULL value because of failed VG name comparison (in _find_vg_rlocn) between the name in existing metadata and metadata we're just about to write. This resets the position in the ring buffer, overwriting any existing metadata (and also incorrectly updates the cache to "orphan" afterwards). This patch just adds old_name item in struct volume_group that we can check and use if necessary and detect renames at lower layers as well. The same applies for vgcfgrestore, but here we're using a special value of old_name, an empty string, to disable the check with existing metadata totally.
2010-04-14 17:09:16 +04:00
}
Propagate commit and revert metadata event to other nodes in cluster. This patch tries to correctly track changes in lvmcache related to commit/revert. For vg_commit: if there is cached precommitted metadata, after successfull commit these metadata must be tracked as committed. For vg_revert: remote nodes must drop precommitted metadata and its flag in lvmcache. (N.B. Patch do not touch LV locks here in any way.) All this machinery is needed to properly solve remote node cache invalidaton which cause several problems recently observed.
2010-01-05 19:09:33 +03:00
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
/* If at least one mda commit succeeded, it was committed */
fix vg_commit return value The existing comment was desribing the correct behavior, but the code didn't match. The commit is successful if one mda was committed. Making it depend on the result of the internal lvmcache update was wrong.
2019-02-05 23:02:24 +03:00
return ret;
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
}
/* Don't commit any pending changes */
Abort if _finish_pvmove suspend_lvs fails instead of cleaning up incompletely. Change suspend_lvs to call vg_revert internally. Change vg_revert to void and remove superfluous calls after failed vg_commit.
2011-09-27 21:09:42 +04:00
void vg_revert(struct volume_group *vg)
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
{
struct metadata_area *mda;
Add lvmlockd
2015-03-05 23:00:44 +03:00
struct lv_list *lvl;
dm_list_iterate_items(lvl, &vg->lvs) {
if (lvl->lv->new_lock_args) {
lockd_free_lv(vg->cmd, vg, lvl->lv->name, &lvl->lv->lvid.id[1], lvl->lv->lock_args);
lvl->lv->new_lock_args = 0;
}
}
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
metadata: Eliminate redundant nested VG metadata Only lv_committed() now uses vg->vg_committed and it appears redundant if its contents match the enclosing VG so don't waste cycles creating it when that's known to be true when no write lock is held so the struct won't get modified.
2017-11-14 18:38:55 +03:00
_vg_wipe_cached_precommitted(vg); /* VG is no longer needed */
metadata: move vg parsing to vg_write Parsing vg structure during supend/commit/resume may require a lot of memory - so move this into vg_write. FIXME: there are now multiple cache layers which our doing some thing multiple times at different levels. Moreover there is now different caching path with and without lvmetad - this should be unified and both path should use same mechanism.
2014-02-22 04:44:21 +04:00
Rename fid->metadata_areas to fid->metadata_areas_in_use. Rename the metadata_areas list to an 'in_use' list to prepare for future 'ignored' list.
2010-06-29 00:32:44 +04:00
dm_list_iterate_items(mda, &vg->fid->metadata_areas_in_use) {
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
if (mda->ops->vg_revert &&
!mda->ops->vg_revert(vg->fid, vg, mda)) {
stack;
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
}
}
}
Move lvmcache data structures behind an API (making the structures private to lvmcache.c). No functional change.
2012-02-10 05:28:27 +04:00
struct _vg_read_orphan_baton {
metadata: _vg_read: check if PV_EXT_USED flag is set correctly for orphan PVs and do a repair if needed If we know that the PV is orphan, meaning there's at least one MDA on that PV which does not reference any VG and at the same time there's PV_EXT_USED flag set, we're certainly in an inconsistent state and we need to fix this. For example, such situation can happen during vgremove/vgreduce if we removed/reduced the VG, but we haven't written PV headers yet because vgremove stopped abruptly for whatever reason just before writing new PV headers with updated state, including PV extension flags (and so the PV_EXT_USED flag). However, in case the PV has no MDAs at all, we can't double-check whether the PV_EXT_USED is correct or not - if that PV is marked as used, it's either: - really used (but other disks with MDAs are missing) - or the error state as described above is hit User needs to overwrite the PV header directly if it's really clear the PV having no MDAs does not belong to any VG and at the same time it's still marked as being in use (pvcreate -ff <dev_name> will fix this). For example - /dev/sda here has 1 MDA, orphan and is incorrectly marked with PV_EXT_USED flag: $ pvs --binary -o+pv_in_use WARNING: Found inconsistent standalone Physical Volumes. WARNING: Repairing flag incorrectly marking Physical Volume /dev/sda as used. PV VG Fmt Attr PSize PFree InUse /dev/sda lvm2 --- 128.00m 128.00m 0
2015-03-11 18:18:42 +03:00
struct cmd_context *cmd;
Move lvmcache data structures behind an API (making the structures private to lvmcache.c). No functional change.
2012-02-10 05:28:27 +04:00
struct volume_group *vg;
label_scan: remove extra label scan and read for orphan PVs When process_each_pv() calls vg_read() on the orphan VG, the internal implementation was doing an unnecessary lvmcache_label_scan() and two unnecessary label_read() calls on each orphan. Some of those unnecessary label scans/reads would sometimes be skipped due to caching, but the code was always doing at least one unnecessary read on each orphan. The common format_text case was also unecessarily calling into the format-specific pv_read() function which actually did nothing. By analyzing each case in which vg_read() was being called on the orphan VG, we can say that all of the label scans/reads in vg_read_orphans are unnecessary: 1. reporting commands: the information saved in lvmcache by the original label scan can be reported. There is no advantage to repeating the label scan on the orphans a second time before reporting it. 2. pvcreate/vgcreate/vgextend: these all share a common implementation in pvcreate_each_device(). That function already rescans labels after acquiring the orphan VG lock, which ensures that the command is using valid lvmcache information.
2017-11-06 21:09:52 +03:00
const struct format_type *fmt;
Move lvmcache data structures behind an API (making the structures private to lvmcache.c). No functional change.
2012-02-10 05:28:27 +04:00
};
static int _vg_read_orphan_pv(struct lvmcache_info *info, void *baton)
{
struct _vg_read_orphan_baton *b = baton;
struct physical_volume *pv = NULL;
struct pv_list *pvl;
print warning about in-use orphans Warn about a PV that has the in-use flag set, but appears in the orphan VG (no VG was found referencing it.) There are a number of conditions that could lead to this: . The PV was created with no mdas and is used in a VG with other PVs (with metadata) that have not yet appeared on the system. So, no VG metadata is found by lvm which references the in-use PV with no mdas. . vgremove could have failed after clearing mdas but before clearing the in-use flag. In this case, the in-use flag needs to be manually cleared on the PV. . The PV may have damanged/unrecognized VG metadata that lvm could not read. . The PV may have no mdas, and the PVs with the metadata may have damaged/unrecognized metadata.
2017-06-01 19:10:09 +03:00
uint32_t ext_version;
uint32_t ext_flags;
Move lvmcache data structures behind an API (making the structures private to lvmcache.c). No functional change.
2012-02-10 05:28:27 +04:00
label_scan: remove extra label scan and read for orphan PVs When process_each_pv() calls vg_read() on the orphan VG, the internal implementation was doing an unnecessary lvmcache_label_scan() and two unnecessary label_read() calls on each orphan. Some of those unnecessary label scans/reads would sometimes be skipped due to caching, but the code was always doing at least one unnecessary read on each orphan. The common format_text case was also unecessarily calling into the format-specific pv_read() function which actually did nothing. By analyzing each case in which vg_read() was being called on the orphan VG, we can say that all of the label scans/reads in vg_read_orphans are unnecessary: 1. reporting commands: the information saved in lvmcache by the original label scan can be reported. There is no advantage to repeating the label scan on the orphans a second time before reporting it. 2. pvcreate/vgcreate/vgextend: these all share a common implementation in pvcreate_each_device(). That function already rescans labels after acquiring the orphan VG lock, which ensures that the command is using valid lvmcache information.
2017-11-06 21:09:52 +03:00
if (!(pv = _pv_read(b->cmd, b->fmt, b->vg, info))) {
cleanup: add some missing stack backtraces
2012-03-14 00:36:02 +04:00
stack;
Move lvmcache data structures behind an API (making the structures private to lvmcache.c). No functional change.
2012-02-10 05:28:27 +04:00
return 1;
}
The lvmetad client-side integration. Only active when use_lvmetad = 1 is set in lvm.conf *and* lvmetad is running.
2012-02-23 17:11:07 +04:00
Move lvmcache data structures behind an API (making the structures private to lvmcache.c). No functional change.
2012-02-10 05:28:27 +04:00
if (!(pvl = dm_pool_zalloc(b->vg->vgmem, sizeof(*pvl)))) {
log_error("pv_list allocation failed");
free_pv_fid(pv);
return 0;
}
pvl->pv = pv;
add_pvl_to_vgs(b->vg, pvl);
metadata: _vg_read: check if PV_EXT_USED flag is set correctly for orphan PVs and do a repair if needed If we know that the PV is orphan, meaning there's at least one MDA on that PV which does not reference any VG and at the same time there's PV_EXT_USED flag set, we're certainly in an inconsistent state and we need to fix this. For example, such situation can happen during vgremove/vgreduce if we removed/reduced the VG, but we haven't written PV headers yet because vgremove stopped abruptly for whatever reason just before writing new PV headers with updated state, including PV extension flags (and so the PV_EXT_USED flag). However, in case the PV has no MDAs at all, we can't double-check whether the PV_EXT_USED is correct or not - if that PV is marked as used, it's either: - really used (but other disks with MDAs are missing) - or the error state as described above is hit User needs to overwrite the PV header directly if it's really clear the PV having no MDAs does not belong to any VG and at the same time it's still marked as being in use (pvcreate -ff <dev_name> will fix this). For example - /dev/sda here has 1 MDA, orphan and is incorrectly marked with PV_EXT_USED flag: $ pvs --binary -o+pv_in_use WARNING: Found inconsistent standalone Physical Volumes. WARNING: Repairing flag incorrectly marking Physical Volume /dev/sda as used. PV VG Fmt Attr PSize PFree InUse /dev/sda lvm2 --- 128.00m 128.00m 0
2015-03-11 18:18:42 +03:00
disable repairing in-use flag on orphan PVs A PV holding VG metadata that lvm can't understand (e.g. damaged, checksum error, unrecognized flag) will appear as an in-use orphan, and will be cleared by this repair code. Disable this repair until the code can keep track of these problematic PVs, and distinguish them from actual in-use orphans.
2017-05-26 21:26:09 +03:00
/*
* FIXME: this bit of code that does the auto repair is disabled
* until we can distinguish cases where the repair should not
* happen, i.e. the VG metadata could not be read/parsed.
*
* A PV holding VG metadata that lvm can't understand
* (e.g. damaged, checksum error, unrecognized flag)
* will appear as an in-use orphan, and would be cleared
* by this repair code. Disable this repair until the
* code can keep track of these problematic PVs, and
* distinguish them from actual in-use orphans.
*/
/*
metadata: _vg_read: check if PV_EXT_USED flag is set correctly for orphan PVs and do a repair if needed If we know that the PV is orphan, meaning there's at least one MDA on that PV which does not reference any VG and at the same time there's PV_EXT_USED flag set, we're certainly in an inconsistent state and we need to fix this. For example, such situation can happen during vgremove/vgreduce if we removed/reduced the VG, but we haven't written PV headers yet because vgremove stopped abruptly for whatever reason just before writing new PV headers with updated state, including PV extension flags (and so the PV_EXT_USED flag). However, in case the PV has no MDAs at all, we can't double-check whether the PV_EXT_USED is correct or not - if that PV is marked as used, it's either: - really used (but other disks with MDAs are missing) - or the error state as described above is hit User needs to overwrite the PV header directly if it's really clear the PV having no MDAs does not belong to any VG and at the same time it's still marked as being in use (pvcreate -ff <dev_name> will fix this). For example - /dev/sda here has 1 MDA, orphan and is incorrectly marked with PV_EXT_USED flag: $ pvs --binary -o+pv_in_use WARNING: Found inconsistent standalone Physical Volumes. WARNING: Repairing flag incorrectly marking Physical Volume /dev/sda as used. PV VG Fmt Attr PSize PFree InUse /dev/sda lvm2 --- 128.00m 128.00m 0
2015-03-11 18:18:42 +03:00
if (!_check_or_repair_orphan_pv_ext(pv, info, baton)) {
stack;
return 0;
}
disable repairing in-use flag on orphan PVs A PV holding VG metadata that lvm can't understand (e.g. damaged, checksum error, unrecognized flag) will appear as an in-use orphan, and will be cleared by this repair code. Disable this repair until the code can keep track of these problematic PVs, and distinguish them from actual in-use orphans.
2017-05-26 21:26:09 +03:00
*/
metadata: _vg_read: check if PV_EXT_USED flag is set correctly for orphan PVs and do a repair if needed If we know that the PV is orphan, meaning there's at least one MDA on that PV which does not reference any VG and at the same time there's PV_EXT_USED flag set, we're certainly in an inconsistent state and we need to fix this. For example, such situation can happen during vgremove/vgreduce if we removed/reduced the VG, but we haven't written PV headers yet because vgremove stopped abruptly for whatever reason just before writing new PV headers with updated state, including PV extension flags (and so the PV_EXT_USED flag). However, in case the PV has no MDAs at all, we can't double-check whether the PV_EXT_USED is correct or not - if that PV is marked as used, it's either: - really used (but other disks with MDAs are missing) - or the error state as described above is hit User needs to overwrite the PV header directly if it's really clear the PV having no MDAs does not belong to any VG and at the same time it's still marked as being in use (pvcreate -ff <dev_name> will fix this). For example - /dev/sda here has 1 MDA, orphan and is incorrectly marked with PV_EXT_USED flag: $ pvs --binary -o+pv_in_use WARNING: Found inconsistent standalone Physical Volumes. WARNING: Repairing flag incorrectly marking Physical Volume /dev/sda as used. PV VG Fmt Attr PSize PFree InUse /dev/sda lvm2 --- 128.00m 128.00m 0
2015-03-11 18:18:42 +03:00
print warning about in-use orphans Warn about a PV that has the in-use flag set, but appears in the orphan VG (no VG was found referencing it.) There are a number of conditions that could lead to this: . The PV was created with no mdas and is used in a VG with other PVs (with metadata) that have not yet appeared on the system. So, no VG metadata is found by lvm which references the in-use PV with no mdas. . vgremove could have failed after clearing mdas but before clearing the in-use flag. In this case, the in-use flag needs to be manually cleared on the PV. . The PV may have damanged/unrecognized VG metadata that lvm could not read. . The PV may have no mdas, and the PVs with the metadata may have damaged/unrecognized metadata.
2017-06-01 19:10:09 +03:00
/*
* Nothing to do if PV header extension < 2:
* - version 0 is PV header without any extensions,
* - version 1 has bootloader area support only and
* we're not checking anything for that one here.
*/
ext_version = lvmcache_ext_version(info);
ext_flags = lvmcache_ext_flags(info);
/*
* Warn about a PV that has the in-use flag set, but appears in
* the orphan VG (no VG was found referencing it.)
* There are a number of conditions that could lead to this:
*
* . The PV was created with no mdas and is used in a VG with
* other PVs (with metadata) that have not yet appeared on
* the system. So, no VG metadata is found by lvm which
* references the in-use PV with no mdas.
*
* . vgremove could have failed after clearing mdas but
* before clearing the in-use flag. In this case, the
* in-use flag needs to be manually cleared on the PV.
*
* . The PV may have damanged/unrecognized VG metadata
* that lvm could not read.
*
* . The PV may have no mdas, and the PVs with the metadata
* may have damaged/unrecognized metadata.
*/
if ((ext_version >= 2) && (ext_flags & PV_EXT_USED)) {
log_warn("WARNING: PV %s is marked in use but no VG was found using it.", pv_dev_name(pv));
log_warn("WARNING: PV %s might need repairing.", pv_dev_name(pv));
}
Move lvmcache data structures behind an API (making the structures private to lvmcache.c). No functional change.
2012-02-10 05:28:27 +04:00
return 1;
}
Fix leak of FID structure
2012-03-01 13:46:38 +04:00
/* Make orphan PVs look like a VG. */
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
struct volume_group *vg_read_orphans(struct cmd_context *cmd, const char *orphan_vgname)
Some new features.
2002-11-18 17:04:08 +03:00
{
vgsplit: simplify vg creation The way that this command now uses the global lock followed by a label scan, it can simply check if the new VG name exists, and if not lock it and create it.
2019-06-07 22:30:03 +03:00
const struct format_type *fmt = cmd->fmt;
Another sync point - numerous fixes & clean ups.
2003-07-05 02:34:56 +04:00
struct lvmcache_vginfo *vginfo;
Various cleanups for fid mem and ref_count changes. Missing free_vg on error_path in lvmcache_get_vg fn. Call destroy_instance only if the fid is not part of the vg in backup_read_vg fn (otherwise it's part of the VG we're returning and we definitely don't want to destroy it!).
2011-03-11 18:08:31 +03:00
struct volume_group *vg = NULL;
Move lvmcache data structures behind an API (making the structures private to lvmcache.c). No functional change.
2012-02-10 05:28:27 +04:00
struct _vg_read_orphan_baton baton;
cleanup: use safe iterator Simplify code and use dm_list_iterate_items_safe() and avoid scanning the list mutliple times. Use dm_list_move().
2013-11-22 16:18:02 +04:00
struct pv_list *pvl, *tpvl;
metadata: Fix handling of orphan PV linking & re-linking.
2013-02-19 06:13:59 +04:00
struct pv_list head;
Some new features.
2002-11-18 17:04:08 +03:00
metadata: Fix handling of orphan PV linking & re-linking.
2013-02-19 06:13:59 +04:00
dm_list_init(&head.list);
Fix vgreduce to use vg_split_mdas to check sufficient mdas remain. Add (empty) orphan VGs to lvmcache during initialisation. Fix orphan VG name used for format_pool.
2008-04-08 16:49:21 +04:00
Move lvmcache data structures behind an API (making the structures private to lvmcache.c). No functional change.
2012-02-10 05:28:27 +04:00
if (!(vginfo = lvmcache_vginfo_from_vgname(orphan_vgname, NULL)))
return_NULL;
Keep a global (per-format) orphan_vg and keep any and all orphan PVs linked to it. Avoids the need for FMT_INSTANCE_PV and enables further simplifications. No functional change, internal refactor only.
2012-02-10 06:53:03 +04:00
vg = fmt->orphan_vg;
cleanup: use safe iterator Simplify code and use dm_list_iterate_items_safe() and avoid scanning the list mutliple times. Use dm_list_move().
2013-11-22 16:18:02 +04:00
dm_list_iterate_items_safe(pvl, tpvl, &vg->pvs)
if (pvl->pv->status & UNLABELLED_PV )
dm_list_move(&head.list, &pvl->list);
else
metadata: Fix handling of orphan PV linking & re-linking.
2013-02-19 06:13:59 +04:00
pv_set_fid(pvl->pv, NULL);
cleanup: use safe iterator Simplify code and use dm_list_iterate_items_safe() and avoid scanning the list mutliple times. Use dm_list_move().
2013-11-22 16:18:02 +04:00
Fix a whitespace bug in the last checkin.
2012-02-29 04:19:14 +04:00
dm_list_init(&vg->pvs);
Attempt a fix for lvm shell accumulating copies of orphan PVs with each "pvs" invocation.
2012-02-29 04:18:27 +04:00
vg->pv_count = 0;
metadata: Fix handling of orphan PV linking & re-linking.
2013-02-19 06:13:59 +04:00
vg->extent_count = 0;
vg->free_count = 0;
create fids for internal orphan VGs
2008-04-08 02:12:37 +04:00
metadata: _vg_read: check if PV_EXT_USED flag is set correctly for orphan PVs and do a repair if needed If we know that the PV is orphan, meaning there's at least one MDA on that PV which does not reference any VG and at the same time there's PV_EXT_USED flag set, we're certainly in an inconsistent state and we need to fix this. For example, such situation can happen during vgremove/vgreduce if we removed/reduced the VG, but we haven't written PV headers yet because vgremove stopped abruptly for whatever reason just before writing new PV headers with updated state, including PV extension flags (and so the PV_EXT_USED flag). However, in case the PV has no MDAs at all, we can't double-check whether the PV_EXT_USED is correct or not - if that PV is marked as used, it's either: - really used (but other disks with MDAs are missing) - or the error state as described above is hit User needs to overwrite the PV header directly if it's really clear the PV having no MDAs does not belong to any VG and at the same time it's still marked as being in use (pvcreate -ff <dev_name> will fix this). For example - /dev/sda here has 1 MDA, orphan and is incorrectly marked with PV_EXT_USED flag: $ pvs --binary -o+pv_in_use WARNING: Found inconsistent standalone Physical Volumes. WARNING: Repairing flag incorrectly marking Physical Volume /dev/sda as used. PV VG Fmt Attr PSize PFree InUse /dev/sda lvm2 --- 128.00m 128.00m 0
2015-03-11 18:18:42 +03:00
baton.cmd = cmd;
label_scan: remove extra label scan and read for orphan PVs When process_each_pv() calls vg_read() on the orphan VG, the internal implementation was doing an unnecessary lvmcache_label_scan() and two unnecessary label_read() calls on each orphan. Some of those unnecessary label scans/reads would sometimes be skipped due to caching, but the code was always doing at least one unnecessary read on each orphan. The common format_text case was also unecessarily calling into the format-specific pv_read() function which actually did nothing. By analyzing each case in which vg_read() was being called on the orphan VG, we can say that all of the label scans/reads in vg_read_orphans are unnecessary: 1. reporting commands: the information saved in lvmcache by the original label scan can be reported. There is no advantage to repeating the label scan on the orphans a second time before reporting it. 2. pvcreate/vgcreate/vgextend: these all share a common implementation in pvcreate_each_device(). That function already rescans labels after acquiring the orphan VG lock, which ensures that the command is using valid lvmcache information.
2017-11-06 21:09:52 +03:00
baton.fmt = fmt;
Move lvmcache data structures behind an API (making the structures private to lvmcache.c). No functional change.
2012-02-10 05:28:27 +04:00
baton.vg = vg;
label_scan: remove extra label scan and read for orphan PVs When process_each_pv() calls vg_read() on the orphan VG, the internal implementation was doing an unnecessary lvmcache_label_scan() and two unnecessary label_read() calls on each orphan. Some of those unnecessary label scans/reads would sometimes be skipped due to caching, but the code was always doing at least one unnecessary read on each orphan. The common format_text case was also unecessarily calling into the format-specific pv_read() function which actually did nothing. By analyzing each case in which vg_read() was being called on the orphan VG, we can say that all of the label scans/reads in vg_read_orphans are unnecessary: 1. reporting commands: the information saved in lvmcache by the original label scan can be reported. There is no advantage to repeating the label scan on the orphans a second time before reporting it. 2. pvcreate/vgcreate/vgextend: these all share a common implementation in pvcreate_each_device(). That function already rescans labels after acquiring the orphan VG lock, which ensures that the command is using valid lvmcache information.
2017-11-06 21:09:52 +03:00
/*
* vg_read for a normal VG will rescan labels for all the devices
* in the VG, in case something changed on disk between the initial
* label scan and acquiring the VG lock. We don't rescan labels
* here because this is only called in two ways:
*
* 1. for reporting, in which case it doesn't matter if something
* changed between the label scan and printing the PVs here
*
* 2. pvcreate_each_device() for pvcreate//vgcreate/vgextend,
* which already does the label rescan after taking the
* orphan lock.
*/
Add missing test for failure of lvmcache_foreach_pv
2012-02-13 14:58:20 +04:00
cleanup: drop extra dm_list_empty Since dm_list_first has this check already include, skip extra call in while(). Moreover analyzers are then sure pvl is not NULL.
2013-11-24 22:03:29 +04:00
while ((pvl = (struct pv_list *) dm_list_first(&head.list))) {
metadata: Fix handling of orphan PV linking & re-linking.
2013-02-19 06:13:59 +04:00
dm_list_del(&pvl->list);
add_pvl_to_vgs(vg, pvl);
vg->extent_count += pvl->pv->pe_count;
vg->free_count += pvl->pv->pe_count;
}
Add missing test for failure of lvmcache_foreach_pv
2012-02-13 14:58:20 +04:00
if (!lvmcache_foreach_pv(vginfo, _vg_read_orphan_pv, &baton))
Fix leak of FID structure
2012-03-01 13:46:38 +04:00
return_NULL;
Some new features.
2002-11-18 17:04:08 +03:00
return vg;
}
Fix possible format instance memory leaks and premature releases in _vg_read.
2011-08-11 20:31:40 +04:00
static void _destroy_fid(struct format_instance **fid)
{
if (*fid) {
(*fid)->fmt->ops->destroy_instance(*fid);
*fid = NULL;
}
}
Remove pv_t, vg_t & lv_t handles from lib. Only liblvm uses them. Rename lvm.h to lvm2app.h for now.
2009-07-29 17:26:01 +04:00
int vg_missing_pv_count(const struct volume_group *vg)
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
{
int ret = 0;
struct pv_list *pvl;
Right, a simple build (without options) is working again.
2008-11-04 01:14:30 +03:00
dm_list_iterate_items(pvl, &vg->pvs) {
Introduce is_missing_pv().
2010-03-16 17:37:38 +03:00
if (is_missing_pv(pvl->pv))
Improve the way VGs with PVs missing are handled so manual intervention is required in fewer circumstances. (mornfall)
2008-09-19 10:42:00 +04:00
++ ret;
}
return ret;
}
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
#define DEV_LIST_DELIM ", "
metadata: use own mem pool to report PV device mismatch in VG
2016-03-21 16:38:49 +03:00
static int _check_devs_used_correspond_with_lv(struct dm_pool *mem, struct dm_list *list, struct logical_volume *lv)
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
{
struct device_list *dl;
int found_inconsistent = 0;
struct device *dev;
struct lv_segment *seg;
uint32_t s;
metadata: log warning instead of error if device not found while checking used and assumed devs When checking assumed PVs against real devices used for LVs and if there's no device assigned for an assumed PV (e.g. due to filters), do log_warn instead of log_error and continue checking LV segments and associated assumed PVs further, just like we do log_warn elsewhere in this situation. This way user will see the warning for each LV which couldn't be checked completely against real PVs used. Before, we logged only the very first occurence of missing device for an LV in a VG and we returned from the function doing this check for all the LVs in VG immediately which may be a bit misleading because it didn't tell user about all the other LVs and whether they could be checked or not. For example, we have this setup: [0] fedora/~ # pvs PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m /dev/vda2 fedora lvm2 a-- 19.49g 0 [0] fedora/~ # lvs -o+devices LV VG Attr LSize Devices root fedora -wi-ao---- 19.00g /dev/vda2(0) swap fedora -wi-ao---- 500.00m /dev/vda2(4864) Before this patch (only the very first LV in a VG is logged to have a problem while checking used and assumed devices): [0] fedora/~ # pvs --config 'devices/filter=["a|/dev/sda|", "r|.*|"]' WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. Couldn't find device for segment belonging to fedora/root while checking used and assumed devices. PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m [unknown] fedora lvm2 a-m 19.49g 0 With this patch applied (all LVs where we hit problem while checking used and assumed devices are logged and it's warning, not error): [0] fedora/~ # pvs --config 'devices/filter=["a|/dev/sda|", "r|.*|"]' WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Couldn't find device for segment belonging to fedora/root while checking used and assumed devices. WARNING: Couldn't find device for segment belonging to fedora/swap while checking used and assumed devices. PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m [unknown] fedora lvm2 a-m 19.49g 0
2016-04-25 12:15:44 +03:00
int warned_about_no_dev = 0;
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
char *used_devnames = NULL, *assumed_devnames = NULL;
if (!(list = dev_cache_get_dev_list_for_lvid(lv->lvid.s + ID_LEN)))
return 1;
dm_list_iterate_items(dl, list) {
dev = dl->dev;
if (!(dev->flags & DEV_ASSUMED_FOR_LV)) {
if (!found_inconsistent) {
coverity: add some tests for function results Even though they cannot normally happen...
2016-07-12 17:43:12 +03:00
if (!dm_pool_begin_object(mem, 32))
return_0;
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
found_inconsistent = 1;
coverity: fix some issues reported by coverity for recent code
2016-03-22 18:03:51 +03:00
} else {
if (!dm_pool_grow_object(mem, DEV_LIST_DELIM, sizeof(DEV_LIST_DELIM) - 1))
coverity: drop abadoing object As mempool is destroyed on by caller don't bother for mempool freeing here.
2016-04-21 21:55:23 +03:00
return_0;
coverity: fix some issues reported by coverity for recent code
2016-03-22 18:03:51 +03:00
}
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
if (!dm_pool_grow_object(mem, dev_name(dev), 0))
coverity: drop abadoing object As mempool is destroyed on by caller don't bother for mempool freeing here.
2016-04-21 21:55:23 +03:00
return_0;
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
}
}
if (!found_inconsistent)
return 1;
coverity: fix some issues reported by coverity for recent code
2016-03-22 18:03:51 +03:00
if (!dm_pool_grow_object(mem, "\0", 1))
coverity: drop abadoing object As mempool is destroyed on by caller don't bother for mempool freeing here.
2016-04-21 21:55:23 +03:00
return_0;
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
used_devnames = dm_pool_end_object(mem);
found_inconsistent = 0;
dm_list_iterate_items(seg, &lv->segments) {
for (s = 0; s < seg->area_count; s++) {
if (seg_type(seg, s) == AREA_PV) {
if (!(dev = seg_dev(seg, s))) {
metadata: log warning instead of error if device not found while checking used and assumed devs When checking assumed PVs against real devices used for LVs and if there's no device assigned for an assumed PV (e.g. due to filters), do log_warn instead of log_error and continue checking LV segments and associated assumed PVs further, just like we do log_warn elsewhere in this situation. This way user will see the warning for each LV which couldn't be checked completely against real PVs used. Before, we logged only the very first occurence of missing device for an LV in a VG and we returned from the function doing this check for all the LVs in VG immediately which may be a bit misleading because it didn't tell user about all the other LVs and whether they could be checked or not. For example, we have this setup: [0] fedora/~ # pvs PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m /dev/vda2 fedora lvm2 a-- 19.49g 0 [0] fedora/~ # lvs -o+devices LV VG Attr LSize Devices root fedora -wi-ao---- 19.00g /dev/vda2(0) swap fedora -wi-ao---- 500.00m /dev/vda2(4864) Before this patch (only the very first LV in a VG is logged to have a problem while checking used and assumed devices): [0] fedora/~ # pvs --config 'devices/filter=["a|/dev/sda|", "r|.*|"]' WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. Couldn't find device for segment belonging to fedora/root while checking used and assumed devices. PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m [unknown] fedora lvm2 a-m 19.49g 0 With this patch applied (all LVs where we hit problem while checking used and assumed devices are logged and it's warning, not error): [0] fedora/~ # pvs --config 'devices/filter=["a|/dev/sda|", "r|.*|"]' WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Couldn't find device for segment belonging to fedora/root while checking used and assumed devices. WARNING: Couldn't find device for segment belonging to fedora/swap while checking used and assumed devices. PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m [unknown] fedora lvm2 a-m 19.49g 0
2016-04-25 12:15:44 +03:00
if (!warned_about_no_dev) {
cleanup: do not mention segment in warning message if device not found for a PV when checking used/assumed devs for an LV [0] fedora/~ # pvs --config 'devices/filter=["a|/dev/sda|", "r|.*|"]' WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Couldn't find device for segment belonging to fedora/root while checking used and assumed devices. WARNING: Couldn't find device for segment belonging to fedora/swap while checking used and assumed devices. PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m [unknown] fedora lvm2 a-m 19.49g 0 Probably not worth mentioning "segments" here, just state that devices for an LV can't be all found during the check - it's less mysterious for user then: [0] fedora/~ # pvs --config 'devices/filter=["a|/dev/sda|", "r|.*|"]' WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Couldn't find all devices for LV fedora/root while checking used and assumed devices. WARNING: Couldn't find all devices for LV fedora/swap while checking used and assumed devices. PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m [unknown] fedora lvm2 a-m 19.49g 0
2016-04-25 12:41:36 +03:00
log_warn("WARNING: Couldn't find all devices for LV %s "
"while checking used and assumed devices.",
metadata: log warning instead of error if device not found while checking used and assumed devs When checking assumed PVs against real devices used for LVs and if there's no device assigned for an assumed PV (e.g. due to filters), do log_warn instead of log_error and continue checking LV segments and associated assumed PVs further, just like we do log_warn elsewhere in this situation. This way user will see the warning for each LV which couldn't be checked completely against real PVs used. Before, we logged only the very first occurence of missing device for an LV in a VG and we returned from the function doing this check for all the LVs in VG immediately which may be a bit misleading because it didn't tell user about all the other LVs and whether they could be checked or not. For example, we have this setup: [0] fedora/~ # pvs PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m /dev/vda2 fedora lvm2 a-- 19.49g 0 [0] fedora/~ # lvs -o+devices LV VG Attr LSize Devices root fedora -wi-ao---- 19.00g /dev/vda2(0) swap fedora -wi-ao---- 500.00m /dev/vda2(4864) Before this patch (only the very first LV in a VG is logged to have a problem while checking used and assumed devices): [0] fedora/~ # pvs --config 'devices/filter=["a|/dev/sda|", "r|.*|"]' WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. Couldn't find device for segment belonging to fedora/root while checking used and assumed devices. PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m [unknown] fedora lvm2 a-m 19.49g 0 With this patch applied (all LVs where we hit problem while checking used and assumed devices are logged and it's warning, not error): [0] fedora/~ # pvs --config 'devices/filter=["a|/dev/sda|", "r|.*|"]' WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Couldn't find device for segment belonging to fedora/root while checking used and assumed devices. WARNING: Couldn't find device for segment belonging to fedora/swap while checking used and assumed devices. PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m [unknown] fedora lvm2 a-m 19.49g 0
2016-04-25 12:15:44 +03:00
display_lvname(lv));
warned_about_no_dev = 1;
}
continue;
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
}
if (!(dev->flags & DEV_USED_FOR_LV)) {
if (!found_inconsistent) {
coverity: add some tests for function results Even though they cannot normally happen...
2016-07-12 17:43:12 +03:00
if (!dm_pool_begin_object(mem, 32))
return_0;
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
found_inconsistent = 1;
} else {
coverity: fix some issues reported by coverity for recent code
2016-03-22 18:03:51 +03:00
if (!dm_pool_grow_object(mem, DEV_LIST_DELIM, sizeof(DEV_LIST_DELIM) - 1))
coverity: drop abadoing object As mempool is destroyed on by caller don't bother for mempool freeing here.
2016-04-21 21:55:23 +03:00
return_0;
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
}
if (!dm_pool_grow_object(mem, dev_name(dev), 0))
coverity: drop abadoing object As mempool is destroyed on by caller don't bother for mempool freeing here.
2016-04-21 21:55:23 +03:00
return_0;
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
}
}
}
}
if (found_inconsistent) {
coverity: fix some issues reported by coverity for recent code
2016-03-22 18:03:51 +03:00
if (!dm_pool_grow_object(mem, "\0", 1))
coverity: drop abadoing object As mempool is destroyed on by caller don't bother for mempool freeing here.
2016-04-21 21:55:23 +03:00
return_0;
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
assumed_devnames = dm_pool_end_object(mem);
metadata: log warning instead of error if device not found while checking used and assumed devs When checking assumed PVs against real devices used for LVs and if there's no device assigned for an assumed PV (e.g. due to filters), do log_warn instead of log_error and continue checking LV segments and associated assumed PVs further, just like we do log_warn elsewhere in this situation. This way user will see the warning for each LV which couldn't be checked completely against real PVs used. Before, we logged only the very first occurence of missing device for an LV in a VG and we returned from the function doing this check for all the LVs in VG immediately which may be a bit misleading because it didn't tell user about all the other LVs and whether they could be checked or not. For example, we have this setup: [0] fedora/~ # pvs PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m /dev/vda2 fedora lvm2 a-- 19.49g 0 [0] fedora/~ # lvs -o+devices LV VG Attr LSize Devices root fedora -wi-ao---- 19.00g /dev/vda2(0) swap fedora -wi-ao---- 500.00m /dev/vda2(4864) Before this patch (only the very first LV in a VG is logged to have a problem while checking used and assumed devices): [0] fedora/~ # pvs --config 'devices/filter=["a|/dev/sda|", "r|.*|"]' WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. Couldn't find device for segment belonging to fedora/root while checking used and assumed devices. PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m [unknown] fedora lvm2 a-m 19.49g 0 With this patch applied (all LVs where we hit problem while checking used and assumed devices are logged and it's warning, not error): [0] fedora/~ # pvs --config 'devices/filter=["a|/dev/sda|", "r|.*|"]' WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Device for PV Qcxpcy-XgtP-UD3s-PmG0-qLyE-Z0ho-DYsxoz not found or rejected by a filter. WARNING: Couldn't find device for segment belonging to fedora/root while checking used and assumed devices. WARNING: Couldn't find device for segment belonging to fedora/swap while checking used and assumed devices. PV VG Fmt Attr PSize PFree /dev/sda lvm2 --- 128.00m 128.00m [unknown] fedora lvm2 a-m 19.49g 0
2016-04-25 12:15:44 +03:00
log_warn("WARNING: Device mismatch detected for %s which is accessing %s instead of %s.",
display_lvname(lv), used_devnames, assumed_devnames);
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
}
return 1;
}
static int _check_devs_used_correspond_with_vg(struct volume_group *vg)
{
metadata: use own mem pool to report PV device mismatch in VG
2016-03-21 16:38:49 +03:00
struct dm_pool *mem;
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
char vgid[ID_LEN + 1];
struct pv_list *pvl;
struct lv_list *lvl;
struct dm_list *list;
struct device_list *dl;
int found_inconsistent = 0;
strncpy(vgid, (const char *) vg->id.uuid, sizeof(vgid));
vgid[ID_LEN] = '\0';
/* Mark all PVs in VG as used. */
dm_list_iterate_items(pvl, &vg->pvs) {
metadata: fix segfault when filters reject devices Checking for devices uses is_missing_pv() to check if there is a device for the PV. is_missing_pv() is based on the MISSING_PV flag, which does not always correspond to !pv->dev. When using lvmetad, a command like: pvs --config 'devices/filter=["a|/dev/sdb|", "r|.*|"]' will cause a number of PVs to have NULL pv->dev, but not the MISSING_PV flag. So, NULL pv->dev needs to also be checked.
2016-04-27 20:13:26 +03:00
/*
* FIXME: It's not clear if the meaning
* of "missing" should always include the
* !pv->dev case, or if "missing" is the
* more narrow case where VG metadata has
* been written with the MISSING flag.
*/
if (!pvl->pv->dev)
continue;
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
if (is_missing_pv(pvl->pv))
continue;
pvl->pv->dev->flags |= DEV_ASSUMED_FOR_LV;
}
if (!(list = dev_cache_get_dev_list_for_vgid(vgid)))
return 1;
dm_list_iterate_items(dl, list) {
if (!(dl->dev->flags & DEV_OPEN_FAILURE) &&
!(dl->dev->flags & DEV_ASSUMED_FOR_LV)) {
found_inconsistent = 1;
break;
}
}
if (found_inconsistent) {
metadata: use own mem pool to report PV device mismatch in VG
2016-03-21 16:38:49 +03:00
if (!(mem = dm_pool_create("vg_devs_check", 1024)))
return_0;
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
dm_list_iterate_items(lvl, &vg->lvs) {
metadata: use own mem pool to report PV device mismatch in VG
2016-03-21 16:38:49 +03:00
if (!_check_devs_used_correspond_with_lv(mem, list, lvl->lv)) {
dm_pool_destroy(mem);
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
return_0;
metadata: use own mem pool to report PV device mismatch in VG
2016-03-21 16:38:49 +03:00
}
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
}
metadata: use own mem pool to report PV device mismatch in VG
2016-03-21 16:38:49 +03:00
dm_pool_destroy(mem);
dev: detect mismatch between devices used and devices assumed for an LV It's possible for an LVM LV to use a device during activation which then differs from device which LVM assumes based on metadata later on. For example, such device mismatch can occur if LVM doesn't have complete view of devices during activation or if filters are misbehaving or they're incorrectly set during activation. This patch adds code that can detect this mismatch by creating VG UUID and LV UUID index while scanning devices for device cache. The VG UUID index maps VG UUID to a device list. Each device in the list has a device layered above as a holder which is an LVM LV device and for which we know the VG UUID (and similarly for LV UUID index). We can acquire VG and LV UUID by reading /sys/block/<dm_dev_name>/dm/uuid. So these indices represent the actual state of PV device use in the system by LVs and then we compare that to what LVM assumes based on metadata. For example: [0] fedora/~ # lsblk /dev/sdq /dev/sdr /dev/sds /dev/sdt NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdq 65:0 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev1 253:3 0 104M 0 mpath sdr 65:16 0 104M 0 disk `-mpath_dev1 253:3 0 104M 0 mpath sds 65:32 0 104M 0 disk |-vg-lvol0 253:2 0 200M 0 lvm `-mpath_dev2 253:4 0 104M 0 mpath sdt 65:48 0 104M 0 disk `-mpath_dev2 253:4 0 104M 0 mpath In this case the vg-lvol0 is mapped onto sdq and sds becauset this is what was available and seen during activation. Then later on, sdr and sdt appeared and mpath devices were created out of sdq+sdr (mpath_dev1) and sds+sdt (mpath_dev2). Now, LVM assumes (correctly) that mpath_dev1 and mpath_dev2 are the PVs that should be used, not the mpath components (sdq/sdr, sds/sdt). [0] fedora/~ # pvs Found duplicate PV xSUix1GJ2SK82ACFuKzFLAQi8xMfFxnO: using /dev/mapper/mpath_dev1 not /dev/sdq Using duplicate PV /dev/mapper/mpath_dev1 from subsystem DM, replacing /dev/sdq Found duplicate PV MvHyMVabtSqr33AbkUrobq1LjP8oiTRm: using /dev/mapper/mpath_dev2 not /dev/sds Using duplicate PV /dev/mapper/mpath_dev2 from subsystem DM, ignoring /dev/sds WARNING: Device mismatch detected for vg/lvol0 which is accessing /dev/sdq, /dev/sds instead of /dev/mapper/mpath_dev1, /dev/mapper/mpath_dev2. PV VG Fmt Attr PSize PFree /dev/mapper/mpath_dev1 vg lvm2 a-- 100.00m 0 /dev/mapper/mpath_dev2 vg lvm2 a-- 100.00m 0
2016-03-16 16:01:26 +03:00
}
return 1;
}
Add new free_pv_fid fn and use it throughout to free all attached fids. Since format instances will use own memory pool, it's necessary to properly deallocate it. For now, only fid is deallocated. The PV structure itself still uses cmd mempool mostly, but anytime we'd like to add a mempool in the struct physical_volume, we can just rename this fn to free_pv and add the code (like we have free_vg fn for VGs).
2011-03-11 17:56:56 +03:00
void free_pv_fid(struct physical_volume *pv)
{
if (!pv)
return;
fid: fix reset of PV fid Avoid hitting memory corruption (double free) in code path, where PV FID has been already destroyed and the released pointer was left in PV structure and could have been tried to be released from there 2nd. time with final context destruction.
2013-05-29 14:42:09 +04:00
pv_set_fid(pv, NULL);
Add new free_pv_fid fn and use it throughout to free all attached fids. Since format instances will use own memory pool, it's necessary to properly deallocate it. For now, only fid is deallocated. The PV structure itself still uses cmd mempool mostly, but anytime we'd like to add a mempool in the struct physical_volume, we can just rename this fn to free_pv and add the code (like we have free_vg fn for VGs).
2011-03-11 17:56:56 +03:00
}
Some whitespace tidy-ups.
2008-01-30 17:00:02 +03:00
static struct physical_volume *_pv_read(struct cmd_context *cmd,
label_scan: remove extra label scan and read for orphan PVs When process_each_pv() calls vg_read() on the orphan VG, the internal implementation was doing an unnecessary lvmcache_label_scan() and two unnecessary label_read() calls on each orphan. Some of those unnecessary label scans/reads would sometimes be skipped due to caching, but the code was always doing at least one unnecessary read on each orphan. The common format_text case was also unecessarily calling into the format-specific pv_read() function which actually did nothing. By analyzing each case in which vg_read() was being called on the orphan VG, we can say that all of the label scans/reads in vg_read_orphans are unnecessary: 1. reporting commands: the information saved in lvmcache by the original label scan can be reported. There is no advantage to repeating the label scan on the orphans a second time before reporting it. 2. pvcreate/vgcreate/vgextend: these all share a common implementation in pvcreate_each_device(). That function already rescans labels after acquiring the orphan VG lock, which ensures that the command is using valid lvmcache information.
2017-11-06 21:09:52 +03:00
const struct format_type *fmt,
struct volume_group *vg,
struct lvmcache_info *info)
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
{
struct physical_volume *pv;
label_scan: remove extra label scan and read for orphan PVs When process_each_pv() calls vg_read() on the orphan VG, the internal implementation was doing an unnecessary lvmcache_label_scan() and two unnecessary label_read() calls on each orphan. Some of those unnecessary label scans/reads would sometimes be skipped due to caching, but the code was always doing at least one unnecessary read on each orphan. The common format_text case was also unecessarily calling into the format-specific pv_read() function which actually did nothing. By analyzing each case in which vg_read() was being called on the orphan VG, we can say that all of the label scans/reads in vg_read_orphans are unnecessary: 1. reporting commands: the information saved in lvmcache by the original label scan can be reported. There is no advantage to repeating the label scan on the orphans a second time before reporting it. 2. pvcreate/vgcreate/vgextend: these all share a common implementation in pvcreate_each_device(). That function already rescans labels after acquiring the orphan VG lock, which ensures that the command is using valid lvmcache information.
2017-11-06 21:09:52 +03:00
struct device *dev = lvmcache_device(info);
Some new features.
2002-11-18 17:04:08 +03:00
label_scan: remove extra label scan and read for orphan PVs When process_each_pv() calls vg_read() on the orphan VG, the internal implementation was doing an unnecessary lvmcache_label_scan() and two unnecessary label_read() calls on each orphan. Some of those unnecessary label scans/reads would sometimes be skipped due to caching, but the code was always doing at least one unnecessary read on each orphan. The common format_text case was also unecessarily calling into the format-specific pv_read() function which actually did nothing. By analyzing each case in which vg_read() was being called on the orphan VG, we can say that all of the label scans/reads in vg_read_orphans are unnecessary: 1. reporting commands: the information saved in lvmcache by the original label scan can be reported. There is no advantage to repeating the label scan on the orphans a second time before reporting it. 2. pvcreate/vgcreate/vgextend: these all share a common implementation in pvcreate_each_device(). That function already rescans labels after acquiring the orphan VG lock, which ensures that the command is using valid lvmcache information.
2017-11-06 21:09:52 +03:00
if (!(pv = _alloc_pv(vg->vgmem, NULL))) {
log_error("pv allocation failed");
initial pv_segment defns
2005-04-20 00:52:35 +04:00
return NULL;
Some new features.
2002-11-18 17:04:08 +03:00
}
label_scan: remove extra label scan and read for orphan PVs When process_each_pv() calls vg_read() on the orphan VG, the internal implementation was doing an unnecessary lvmcache_label_scan() and two unnecessary label_read() calls on each orphan. Some of those unnecessary label scans/reads would sometimes be skipped due to caching, but the code was always doing at least one unnecessary read on each orphan. The common format_text case was also unecessarily calling into the format-specific pv_read() function which actually did nothing. By analyzing each case in which vg_read() was being called on the orphan VG, we can say that all of the label scans/reads in vg_read_orphans are unnecessary: 1. reporting commands: the information saved in lvmcache by the original label scan can be reported. There is no advantage to repeating the label scan on the orphans a second time before reporting it. 2. pvcreate/vgcreate/vgextend: these all share a common implementation in pvcreate_each_device(). That function already rescans labels after acquiring the orphan VG lock, which ensures that the command is using valid lvmcache information.
2017-11-06 21:09:52 +03:00
if (fmt->ops->pv_read) {
/* format1 and pool */
if (!(fmt->ops->pv_read(fmt, dev_name(dev), pv, 0))) {
log_error("Failed to read existing physical volume '%s'", dev_name(dev));
goto bad;
}
} else {
/* format text */
if (!lvmcache_populate_pv_fields(info, vg, pv))
goto_bad;
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
}
label_scan: remove extra label scan and read for orphan PVs When process_each_pv() calls vg_read() on the orphan VG, the internal implementation was doing an unnecessary lvmcache_label_scan() and two unnecessary label_read() calls on each orphan. Some of those unnecessary label scans/reads would sometimes be skipped due to caching, but the code was always doing at least one unnecessary read on each orphan. The common format_text case was also unecessarily calling into the format-specific pv_read() function which actually did nothing. By analyzing each case in which vg_read() was being called on the orphan VG, we can say that all of the label scans/reads in vg_read_orphans are unnecessary: 1. reporting commands: the information saved in lvmcache by the original label scan can be reported. There is no advantage to repeating the label scan on the orphans a second time before reporting it. 2. pvcreate/vgcreate/vgextend: these all share a common implementation in pvcreate_each_device(). That function already rescans labels after acquiring the orphan VG lock, which ensures that the command is using valid lvmcache information.
2017-11-06 21:09:52 +03:00
if (!alloc_pv_segment_whole_pv(vg->vgmem, pv))
Call _alloc_pv() inside _pv_read() and clean up error paths. We should be consistent with pv constructors so call _alloc_pv() here as we do from pv_create().
2010-01-22 00:09:23 +03:00
goto_bad;
initial pv_segment defns
2005-04-20 00:52:35 +04:00
label_scan: remove extra label scan and read for orphan PVs When process_each_pv() calls vg_read() on the orphan VG, the internal implementation was doing an unnecessary lvmcache_label_scan() and two unnecessary label_read() calls on each orphan. Some of those unnecessary label scans/reads would sometimes be skipped due to caching, but the code was always doing at least one unnecessary read on each orphan. The common format_text case was also unecessarily calling into the format-specific pv_read() function which actually did nothing. By analyzing each case in which vg_read() was being called on the orphan VG, we can say that all of the label scans/reads in vg_read_orphans are unnecessary: 1. reporting commands: the information saved in lvmcache by the original label scan can be reported. There is no advantage to repeating the label scan on the orphans a second time before reporting it. 2. pvcreate/vgcreate/vgextend: these all share a common implementation in pvcreate_each_device(). That function already rescans labels after acquiring the orphan VG lock, which ensures that the command is using valid lvmcache information.
2017-11-06 21:09:52 +03:00
lvmcache_fid_add_mdas(info, vg->fid, (const char *) &pv->id, ID_LEN);
pv_set_fid(pv, vg->fid);
initial pv_segment defns
2005-04-20 00:52:35 +04:00
return pv;
Call _alloc_pv() inside _pv_read() and clean up error paths. We should be consistent with pv constructors so call _alloc_pv() here as we do from pv_create().
2010-01-22 00:09:23 +03:00
bad:
Add new free_pv_fid fn and use it throughout to free all attached fids. Since format instances will use own memory pool, it's necessary to properly deallocate it. For now, only fid is deallocated. The PV structure itself still uses cmd mempool mostly, but anytime we'd like to add a mempool in the struct physical_volume, we can just rename this fn to free_pv and add the code (like we have free_vg fn for VGs).
2011-03-11 17:56:56 +03:00
free_pv_fid(pv);
label_scan: remove extra label scan and read for orphan PVs When process_each_pv() calls vg_read() on the orphan VG, the internal implementation was doing an unnecessary lvmcache_label_scan() and two unnecessary label_read() calls on each orphan. Some of those unnecessary label scans/reads would sometimes be skipped due to caching, but the code was always doing at least one unnecessary read on each orphan. The common format_text case was also unecessarily calling into the format-specific pv_read() function which actually did nothing. By analyzing each case in which vg_read() was being called on the orphan VG, we can say that all of the label scans/reads in vg_read_orphans are unnecessary: 1. reporting commands: the information saved in lvmcache by the original label scan can be reported. There is no advantage to repeating the label scan on the orphans a second time before reporting it. 2. pvcreate/vgcreate/vgextend: these all share a common implementation in pvcreate_each_device(). That function already rescans labels after acquiring the orphan VG lock, which ensures that the command is using valid lvmcache information.
2017-11-06 21:09:52 +03:00
dm_pool_free(vg->vgmem, pv);
Call _alloc_pv() inside _pv_read() and clean up error paths. We should be consistent with pv constructors so call _alloc_pv() here as we do from pv_create().
2010-01-22 00:09:23 +03:00
return NULL;
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
}
separate code for setting devices from metadata parsing Pull the code that sets devs for PVs out of the metadata parsing code and call it separately.
2019-02-05 20:32:54 +03:00
/*
* FIXME: we only want to print the warnings when this is called from
* vg_read, not from import_vg_from_metadata, so do the warnings elsewhere
* or avoid calling this from import_vg_from.
*/
static void _set_pv_device(struct format_instance *fid,
struct volume_group *vg,
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
struct physical_volume *pv,
int *found_md_component)
separate code for setting devices from metadata parsing Pull the code that sets devs for PVs out of the metadata parsing code and call it separately.
2019-02-05 20:32:54 +03:00
{
char buffer[64] __attribute__((aligned(8)));
md component detection for differing PV and device sizes This check was mistakenly removed when shifting code in commit "separate code for setting devices from metadata parsing". Put it back with some new conditions.
2019-07-09 21:32:41 +03:00
struct cmd_context *cmd = fid->fmt->cmd;
struct device *dev;
separate code for setting devices from metadata parsing Pull the code that sets devs for PVs out of the metadata parsing code and call it separately.
2019-02-05 20:32:54 +03:00
uint64_t size;
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
int do_check = 0;
separate code for setting devices from metadata parsing Pull the code that sets devs for PVs out of the metadata parsing code and call it separately.
2019-02-05 20:32:54 +03:00
md component detection for differing PV and device sizes This check was mistakenly removed when shifting code in commit "separate code for setting devices from metadata parsing". Put it back with some new conditions.
2019-07-09 21:32:41 +03:00
if (!(dev = lvmcache_device_from_pvid(cmd, &pv->id, &pv->label_sector))) {
separate code for setting devices from metadata parsing Pull the code that sets devs for PVs out of the metadata parsing code and call it separately.
2019-02-05 20:32:54 +03:00
if (!id_write_format(&pv->id, buffer, sizeof(buffer)))
buffer[0] = '\0';
drop warnings about missing pvs in foreign vgs When a foreign VG is ignored, don't print warnings that it is missing PVs.
2019-12-11 21:56:15 +03:00
if (cmd && !cmd->pvscan_cache_single &&
(!vg_is_foreign(vg) && !cmd->include_foreign_vgs))
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
log_warn("WARNING: Couldn't find device with uuid %s.", buffer);
separate code for setting devices from metadata parsing Pull the code that sets devs for PVs out of the metadata parsing code and call it separately.
2019-02-05 20:32:54 +03:00
else
log_debug_metadata("Couldn't find device with uuid %s.", buffer);
}
md component detection for differing PV and device sizes This check was mistakenly removed when shifting code in commit "separate code for setting devices from metadata parsing". Put it back with some new conditions.
2019-07-09 21:32:41 +03:00
/*
* If the device and PV are not the size, it's a clue that we might
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
* be reading an MD component (but not necessarily). Skip this check
* if md component detection is disabled or if we are already doing
* full a md check in label scan
md component detection for differing PV and device sizes This check was mistakenly removed when shifting code in commit "separate code for setting devices from metadata parsing". Put it back with some new conditions.
2019-07-09 21:32:41 +03:00
*/
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
if (dev && cmd && cmd->md_component_detection && !cmd->use_full_md_check) {
get dev size when setting pv device In some cases the dev size may not have been read yet in set_pv_devices(). In this case get the dev size before comparing the dev size with the pv size.
2020-10-22 00:21:50 +03:00
uint64_t devsize = dev->size;
if (!devsize && !dev_get_size(dev, &devsize))
log_debug("No size for %s when setting PV dev.", dev_name(dev));
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
/* PV larger than dev not common, check for md component */
get dev size when setting pv device In some cases the dev size may not have been read yet in set_pv_devices(). In this case get the dev size before comparing the dev size with the pv size.
2020-10-22 00:21:50 +03:00
else if (pv->size > devsize)
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
do_check = 1;
/* dev larger than PV can be common, limit check to auto mode */
get dev size when setting pv device In some cases the dev size may not have been read yet in set_pv_devices(). In this case get the dev size before comparing the dev size with the pv size.
2020-10-22 00:21:50 +03:00
else if ((pv->size < devsize) && !strcmp(cmd->md_component_checks, "auto"))
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
do_check = 1;
if (do_check && dev_is_md_component(dev, NULL, 1)) {
md component detection for differing PV and device sizes This check was mistakenly removed when shifting code in commit "separate code for setting devices from metadata parsing". Put it back with some new conditions.
2019-07-09 21:32:41 +03:00
log_warn("WARNING: device %s is an md component, not setting device for PV.",
dev_name(dev));
dev = NULL;
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
if (found_md_component)
*found_md_component = 1;
md component detection for differing PV and device sizes This check was mistakenly removed when shifting code in commit "separate code for setting devices from metadata parsing". Put it back with some new conditions.
2019-07-09 21:32:41 +03:00
}
}
pv->dev = dev;
separate code for setting devices from metadata parsing Pull the code that sets devs for PVs out of the metadata parsing code and call it separately.
2019-02-05 20:32:54 +03:00
/*
* A previous command wrote the VG while this dev was missing, so
* the MISSING flag was included in the PV.
*/
if ((pv->status & MISSING_PV) && pv->dev)
log_warn("WARNING: VG %s was previously updated while PV %s was missing.", vg->name, dev_name(pv->dev));
/*
* If this command writes the VG, we want the MISSING flag to be
* written for this PV with no device.
*/
if (!pv->dev)
pv->status |= MISSING_PV;
/* is this correct? */
if ((pv->status & MISSING_PV) && pv->dev && (pv_mda_used_count(pv) == 0)) {
pv->status &= ~MISSING_PV;
log_info("Found a previously MISSING PV %s with no MDAs.", pv_dev_name(pv));
}
/* Fix up pv size if missing or impossibly large */
if ((!pv->size || pv->size > (1ULL << 62)) && pv->dev) {
if (!dev_get_size(pv->dev, &pv->size)) {
log_error("%s: Couldn't get size.", pv_dev_name(pv));
return;
}
log_verbose("Fixing up missing size (%s) for PV %s", display_size(fid->fmt->cmd, pv->size),
pv_dev_name(pv));
size = pv->pe_count * (uint64_t) vg->extent_size + pv->pe_start;
if (size > pv->size)
log_warn("WARNING: Physical Volume %s is too large "
"for underlying device", pv_dev_name(pv));
}
}
/*
* Finds the 'struct device' that correponds to each PV in the metadata,
* and may make some adjustments to vg fields based on the dev properties.
*/
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
void set_pv_devices(struct format_instance *fid, struct volume_group *vg, int *found_md_component)
separate code for setting devices from metadata parsing Pull the code that sets devs for PVs out of the metadata parsing code and call it separately.
2019-02-05 20:32:54 +03:00
{
struct pv_list *pvl;
dm_list_iterate_items(pvl, &vg->pvs)
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
_set_pv_device(fid, vg, pvl->pv, found_md_component);
separate code for setting devices from metadata parsing Pull the code that sets devs for PVs out of the metadata parsing code and call it separately.
2019-02-05 20:32:54 +03:00
}
pvscan: use process_each_vg for autoactivate This refactors the code for autoactivation. Previously, as each PV was found, it would be sent to lvmetad, and the VG would be autoactivated using a non-standard VG processing function (the "activation_handler") called via a function pointer from within the lvmetad notification path. Now, any scanning that the command needs to do (scanning only the named device args, or scanning all devices when there are no args), is done first, before any activation is attempted. During the scans, the VG names are saved. After scanning is complete, process_each_vg is used to do autoactivation of the saved VG names. This makes pvscan activation much more similar to activation done with vgchange or lvchange. The separate autoactivate phase also means that if lvmetad is disabled (either before or during the scan), the command can continue with the activation step by simply not using lvmetad and reverting to disk scanning to do the activation.
2016-04-28 17:37:03 +03:00
int pv_write(struct cmd_context *cmd,
Allow non-orphan PVs with two metadata areas to be resized. We allow writing non-orphan PVs only for resize now. The "orphan PV" assert in pv_write fn uses the "allow_non_orphan" parameter to control this assert. However, we should find a more elaborate solution so we can remove this restriction altogether (pv_write together with vg_write is not atomic, we need to find a safe mechanism so there's an easy revert possible in case of an error).
2011-02-28 16:19:02 +03:00
struct physical_volume *pv, int allow_non_orphan)
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
{
Cope better when format functions are missing.
2003-08-27 01:12:06 +04:00
if (!pv->fmt->ops->pv_write) {
log_error("Format does not support writing physical volumes");
return 0;
}
Allow only orphan PVs to be resized even with two metadata areas.
2011-02-25 17:08:54 +03:00
/*
* FIXME: Try to remove this restriction. This requires checking
* that the PV and the VG are in a consistent state. We need
* to provide some revert mechanism since PV label together
* with VG metadata write is not atomic.
*/
Allow non-orphan PVs with two metadata areas to be resized. We allow writing non-orphan PVs only for resize now. The "orphan PV" assert in pv_write fn uses the "allow_non_orphan" parameter to control this assert. However, we should find a more elaborate solution so we can remove this restriction altogether (pv_write together with vg_write is not atomic, we need to find a safe mechanism so there's an easy revert possible in case of an error).
2011-02-28 16:19:02 +03:00
if (!allow_non_orphan &&
(!is_orphan_vg(pv->vg_name) || pv->pe_alloc_count)) {
Some new features.
2002-11-18 17:04:08 +03:00
log_error("Assertion failed: can't _pv_write non-orphan PV "
pv_vg_name: do not expose internal orphans to lvm2 users Check for orphan VG name and return just empty string, Use internally pv->vg_name if the real orphan name is needed.
2014-03-19 02:54:46 +04:00
"(in VG %s)", pv_vg_name(pv));
Some new features.
2002-11-18 17:04:08 +03:00
return 0;
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
}
pass cmd struct through more functions no functional change
2020-01-28 19:33:15 +03:00
if (!pv->fmt->ops->pv_write(cmd, pv->fmt, pv))
Use stack return macros throughout.
2008-01-30 16:19:47 +03:00
return_0;
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
metadata: use PV's internal UNLABELLED_PV flag more consistently Set when new PV created, cleared on PV write.
2013-03-25 19:21:59 +04:00
pv->status &= ~UNLABELLED_PV;
Merge with text format branch. Lots of changes/very little testing so far => there'll be bugs! Use 'vgcreate -M text' to create a volume group with its metadata stored in text files. Text format metadata changes should be reasonably atomic, with a (basic) automatic recovery mechanism if the system crashes while a change is in progress. Add a metadata section to lvm.conf to specify multiple directories if you want (recommended) to keep multiple copies of the metadata (eg on different filesystems). e.g. metadata { dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"] } Plenty of refinements still in the pipeline.
2002-04-24 22:20:51 +04:00
return 1;
}
If a PV reappears after it was removed from its VG, make it an orphan.
2007-02-07 16:29:52 +03:00
int pv_write_orphan(struct cmd_context *cmd, struct physical_volume *pv)
{
const char *old_vg_name = pv->vg_name;
split orphan VG by format type
2008-02-06 18:47:28 +03:00
pv->vg_name = cmd->fmt->orphan_vg_name;
If a PV reappears after it was removed from its VG, make it an orphan.
2007-02-07 16:29:52 +03:00
pv->status = ALLOCATABLE_PV;
Fix handling of PVs which reappeared with old metadata version.
2008-09-25 19:59:10 +04:00
pv->pe_alloc_count = 0;
If a PV reappears after it was removed from its VG, make it an orphan.
2007-02-07 16:29:52 +03:00
if (!dev_get_size(pv->dev, &pv->size)) {
Add pv_dev_name() to access PV device name. Patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 18:29:32 +04:00
log_error("%s: Couldn't get size.", pv_dev_name(pv));
If a PV reappears after it was removed from its VG, make it an orphan.
2007-02-07 16:29:52 +03:00
return 0;
}
Allow non-orphan PVs with two metadata areas to be resized. We allow writing non-orphan PVs only for resize now. The "orphan PV" assert in pv_write fn uses the "allow_non_orphan" parameter to control this assert. However, we should find a more elaborate solution so we can remove this restriction altogether (pv_write together with vg_write is not atomic, we need to find a safe mechanism so there's an easy revert possible in case of an error).
2011-02-28 16:19:02 +03:00
if (!pv_write(cmd, pv, 0)) {
If a PV reappears after it was removed from its VG, make it an orphan.
2007-02-07 16:29:52 +03:00
log_error("Failed to clear metadata from physical "
"volume \"%s\" after removal from \"%s\"",
Add pv_dev_name() to access PV device name. Patch by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
2007-10-12 18:29:32 +04:00
pv_dev_name(pv), old_vg_name);
If a PV reappears after it was removed from its VG, make it an orphan.
2007-02-07 16:29:52 +03:00
return 0;
}
return 1;
}
convert pv->vg_name[0] to \!is_orphan(pv)
2007-06-14 19:48:05 +04:00
Add is_orphan_vg() and change all hardcoded checks to use it.
2007-11-02 16:06:42 +03:00
/**
* is_orphan_vg - Determine whether a vg_name is an orphan
* @vg_name: pointer to the vg_name
*/
int is_orphan_vg(const char *vg_name)
{
Add is_global_vg and split out from is_orphan_vg.
2010-05-19 06:36:33 +04:00
return (vg_name && !strncmp(vg_name, ORPHAN_PREFIX, sizeof(ORPHAN_PREFIX) - 1)) ? 1 : 0;
Add is_orphan_vg() and change all hardcoded checks to use it.
2007-11-02 16:06:42 +03:00
}
Replace fs_unlock by sync_local_dev_names to notify local clvmd. (2.02.80) Introduce sync_local_dev_names and CLVMD_CMD_SYNC_NAMES to issue fs_unlock.
2011-01-12 23:42:50 +03:00
/*
* Exclude pseudo VG names used for locking.
*/
int is_real_vg(const char *vg_name)
{
return (vg_name && *vg_name != '#');
}
Add an open_mode to the vg struct for liblvm - enforce read / write semantics. For now, a simple way to enforce the read/write semantics is to just save the open mode of the VG. If the caller uses lvm_vg_create, the mode is write. The caller using lvm_vg_open can use either read or write to open the VG. Once we have this, we enforce the permissions on each API call and don't allow a caller to modify a VG that has not been opened properly. This may be better combined with the locking mode, but I view that as future cleanup, past this initial release. The intial release should enforce the basic object semantics though, as described in the lvm.h file. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2009-07-28 19:14:56 +04:00
/* FIXME: remove / combine this with locking? */
Remove pv_t, vg_t & lv_t handles from lib. Only liblvm uses them. Rename lvm.h to lvm2app.h for now.
2009-07-29 17:26:01 +04:00
int vg_check_write_mode(struct volume_group *vg)
Add an open_mode to the vg struct for liblvm - enforce read / write semantics. For now, a simple way to enforce the read/write semantics is to just save the open mode of the VG. If the caller uses lvm_vg_create, the mode is write. The caller using lvm_vg_open can use either read or write to open the VG. Once we have this, we enforce the permissions on each API call and don't allow a caller to modify a VG that has not been opened properly. This may be better combined with the locking mode, but I view that as future cleanup, past this initial release. The intial release should enforce the basic object semantics though, as described in the lvm.h file. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2009-07-28 19:14:56 +04:00
{
if (vg->open_mode != 'w') {
\n
2009-07-29 00:41:41 +04:00
log_errno(EPERM, "Attempt to modify a read-only VG");
Add an open_mode to the vg struct for liblvm - enforce read / write semantics. For now, a simple way to enforce the read/write semantics is to just save the open mode of the VG. If the caller uses lvm_vg_create, the mode is write. The caller using lvm_vg_open can use either read or write to open the VG. Once we have this, we enforce the permissions on each API call and don't allow a caller to modify a VG that has not been opened properly. This may be better combined with the locking mode, but I view that as future cleanup, past this initial release. The intial release should enforce the basic object semantics though, as described in the lvm.h file. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2009-07-28 19:14:56 +04:00
return 0;
}
return 1;
}
system_id: make new VGs read-only for old lvm versions Previous versions of lvm will not obey the restrictions imposed by the new system_id, and would allow such a VG to be written. So, a VG with a new system_id is further changed to force previous lvm versions to treat it as read-only. This is done by removing the WRITE flag from the metadata status line of these VGs, and putting a new WRITE_LOCKED flag in the flags line of the metadata. Versions of lvm that recognize WRITE_LOCKED, also obey the new system_id. For these lvm versions, WRITE_LOCKED is identical to WRITE, and the rules associated with matching system_id's are imposed. A new VG lock_type field is also added that causes the same WRITE/WRITE_LOCKED transformation when set. A previous version of lvm will also see a VG with lock_type as read-only. Versions of lvm that recognize WRITE_LOCKED, must also obey the lock_type setting. Until the lock_type feature is added, lvm will fail to read any VG with lock_type set and report an error about an unsupported lock_type. Once the lock_type feature is added, lvm will allow VGs with lock_type to be used according to the rules imposed by the lock_type. When both system_id and lock_type settings are removed, a VG is written with the old WRITE status flag, and without the new WRITE_LOCKED flag. This allows old versions of lvm to use the VG as before.
2015-03-04 20:30:53 +03:00
/*
* Return 1 if the VG metadata should be written
comments: Use full flag names.
2015-03-09 21:53:22 +03:00
* *without* the LVM_WRITE flag in the status line, and
* *with* the LVM_WRITE_LOCKED flag in the flags line.
system_id: make new VGs read-only for old lvm versions Previous versions of lvm will not obey the restrictions imposed by the new system_id, and would allow such a VG to be written. So, a VG with a new system_id is further changed to force previous lvm versions to treat it as read-only. This is done by removing the WRITE flag from the metadata status line of these VGs, and putting a new WRITE_LOCKED flag in the flags line of the metadata. Versions of lvm that recognize WRITE_LOCKED, also obey the new system_id. For these lvm versions, WRITE_LOCKED is identical to WRITE, and the rules associated with matching system_id's are imposed. A new VG lock_type field is also added that causes the same WRITE/WRITE_LOCKED transformation when set. A previous version of lvm will also see a VG with lock_type as read-only. Versions of lvm that recognize WRITE_LOCKED, must also obey the lock_type setting. Until the lock_type feature is added, lvm will fail to read any VG with lock_type set and report an error about an unsupported lock_type. Once the lock_type feature is added, lvm will allow VGs with lock_type to be used according to the rules imposed by the lock_type. When both system_id and lock_type settings are removed, a VG is written with the old WRITE status flag, and without the new WRITE_LOCKED flag. This allows old versions of lvm to use the VG as before.
2015-03-04 20:30:53 +03:00
*
* If this is done for a VG, it forces previous versions
comments: Use full flag names.
2015-03-09 21:53:22 +03:00
* of lvm (before the LVM_WRITE_LOCKED flag was added), to view
* the VG and its LVs as read-only (because the LVM_WRITE flag
system_id: make new VGs read-only for old lvm versions Previous versions of lvm will not obey the restrictions imposed by the new system_id, and would allow such a VG to be written. So, a VG with a new system_id is further changed to force previous lvm versions to treat it as read-only. This is done by removing the WRITE flag from the metadata status line of these VGs, and putting a new WRITE_LOCKED flag in the flags line of the metadata. Versions of lvm that recognize WRITE_LOCKED, also obey the new system_id. For these lvm versions, WRITE_LOCKED is identical to WRITE, and the rules associated with matching system_id's are imposed. A new VG lock_type field is also added that causes the same WRITE/WRITE_LOCKED transformation when set. A previous version of lvm will also see a VG with lock_type as read-only. Versions of lvm that recognize WRITE_LOCKED, must also obey the lock_type setting. Until the lock_type feature is added, lvm will fail to read any VG with lock_type set and report an error about an unsupported lock_type. Once the lock_type feature is added, lvm will allow VGs with lock_type to be used according to the rules imposed by the lock_type. When both system_id and lock_type settings are removed, a VG is written with the old WRITE status flag, and without the new WRITE_LOCKED flag. This allows old versions of lvm to use the VG as before.
2015-03-04 20:30:53 +03:00
* is missing). Versions of lvm that understand the
comments: Use full flag names.
2015-03-09 21:53:22 +03:00
* LVM_WRITE_LOCKED flag know to check the other methods of
system_id: make new VGs read-only for old lvm versions Previous versions of lvm will not obey the restrictions imposed by the new system_id, and would allow such a VG to be written. So, a VG with a new system_id is further changed to force previous lvm versions to treat it as read-only. This is done by removing the WRITE flag from the metadata status line of these VGs, and putting a new WRITE_LOCKED flag in the flags line of the metadata. Versions of lvm that recognize WRITE_LOCKED, also obey the new system_id. For these lvm versions, WRITE_LOCKED is identical to WRITE, and the rules associated with matching system_id's are imposed. A new VG lock_type field is also added that causes the same WRITE/WRITE_LOCKED transformation when set. A previous version of lvm will also see a VG with lock_type as read-only. Versions of lvm that recognize WRITE_LOCKED, must also obey the lock_type setting. Until the lock_type feature is added, lvm will fail to read any VG with lock_type set and report an error about an unsupported lock_type. Once the lock_type feature is added, lvm will allow VGs with lock_type to be used according to the rules imposed by the lock_type. When both system_id and lock_type settings are removed, a VG is written with the old WRITE status flag, and without the new WRITE_LOCKED flag. This allows old versions of lvm to use the VG as before.
2015-03-04 20:30:53 +03:00
* access control for the VG, specifically system_id and lock_type.
*
* So, if a VG has a system_id or lock_type, then the
* system_id and lock_type control access to the VG in
* addition to its basic writable status. Because previous
* lvm versions do not know about system_id or lock_type,
comments: Use full flag names.
2015-03-09 21:53:22 +03:00
* VGs depending on either of these should have LVM_WRITE_LOCKED
* instead of LVM_WRITE to prevent the previous lvm versions from
system_id: make new VGs read-only for old lvm versions Previous versions of lvm will not obey the restrictions imposed by the new system_id, and would allow such a VG to be written. So, a VG with a new system_id is further changed to force previous lvm versions to treat it as read-only. This is done by removing the WRITE flag from the metadata status line of these VGs, and putting a new WRITE_LOCKED flag in the flags line of the metadata. Versions of lvm that recognize WRITE_LOCKED, also obey the new system_id. For these lvm versions, WRITE_LOCKED is identical to WRITE, and the rules associated with matching system_id's are imposed. A new VG lock_type field is also added that causes the same WRITE/WRITE_LOCKED transformation when set. A previous version of lvm will also see a VG with lock_type as read-only. Versions of lvm that recognize WRITE_LOCKED, must also obey the lock_type setting. Until the lock_type feature is added, lvm will fail to read any VG with lock_type set and report an error about an unsupported lock_type. Once the lock_type feature is added, lvm will allow VGs with lock_type to be used according to the rules imposed by the lock_type. When both system_id and lock_type settings are removed, a VG is written with the old WRITE status flag, and without the new WRITE_LOCKED flag. This allows old versions of lvm to use the VG as before.
2015-03-04 20:30:53 +03:00
* assuming they can write the VG and its LVs.
*/
int vg_flag_write_locked(struct volume_group *vg)
{
if (vg->system_id && vg->system_id[0])
return 1;
if (vg->lock_type && vg->lock_type[0] && strcmp(vg->lock_type, "none"))
return 1;
return 0;
}
cleanup: reuse existing code
2016-12-25 01:10:06 +03:00
static int _access_vg_clustered(struct cmd_context *cmd, const struct volume_group *vg)
{
Remove clvmd and associated code More code reduction and simplification can follow.
2018-06-05 18:47:01 +03:00
if (vg_is_clustered(vg)) {
lvmlockd: update method for changing clustered VG The previous method for forcibly changing a clustered VG to a local VG involved using -cn and locking_type 0. Since those options are deprecated, replace it with the same command used for other forced lock type changes: vgchange --locktype none --lockopt force.
2018-06-13 23:30:28 +03:00
/*
* force_access_clustered is only set when forcibly
* converting a clustered vg to lock type none.
*/
if (cmd->force_access_clustered) {
log_debug("Allowing forced access to clustered vg %s", vg->name);
return 1;
}
Drop --ignoreskippedcluster option It's no longer needed. Clustered VGs are now handled in the same way as foreign VGs, and as shared VGs that can't be accessed: - A command processing all VGs sees a clustered VG, prints a message ("Skipping clustered VG foo."), skips it, and does not fail. - A command where the clustered VG is explicitly named on the command line, prints a message and fails. "Cannot access clustered VG foo, see lvmlockd(8)." The option is listed in the set of ignored options for the commands that previously accepted it. (Removing it entirely would cause commands/scripts to fail if they set it.)
2018-06-15 23:43:59 +03:00
log_verbose("Skipping clustered VG %s.", vg->name);
cleanup: reuse existing code
2016-12-25 01:10:06 +03:00
return 0;
}
return 1;
}
Replace internal vg_check_status() implementation. (mornfall)
2009-01-27 01:42:59 +03:00
/*
* Performs a set of checks against a VG according to bits set in status
* and returns FAILED_* bits for those that aren't acceptable.
*
* FIXME Remove the unnecessary duplicate definitions and return bits directly.
*/
liblvm: Move lib code used exclusively into metadata-liblvm.c Also remove some redundant function definitions from metadata.h.
2017-10-18 21:29:32 +03:00
uint32_t vg_bad_status_bits(const struct volume_group *vg, uint64_t status)
Introduce as-yet-unused _vg_lock_and_read() and associated header file definitions.
2009-01-27 01:13:22 +03:00
{
uint32_t failure = 0;
Add vg_check_status to consolidate vg status flags checks and error messages.
2007-06-06 23:40:28 +04:00
cleanup: reuse existing code
2016-12-25 01:10:06 +03:00
if ((status & CLUSTERED) && !_access_vg_clustered(vg->cmd, vg))
Introduce as-yet-unused _vg_lock_and_read() and associated header file definitions.
2009-01-27 01:13:22 +03:00
/* Return because other flags are considered undefined. */
return FAILED_CLUSTERED;
if ((status & LVM_WRITE) &&
!(vg->status & LVM_WRITE)) {
log_error("Volume group %s is read-only", vg->name);
failure |= FAILED_READ_ONLY;
}
if ((status & RESIZEABLE_VG) &&
Add vg_is_resizeable() and cleanup references. Clean up VG_RESIZEABLE flag by creating vg_is_resizeable(). Update comment - we no longer have ALLOW_RESIZEABLE. Also use vg_is_exported() in one place missed by earlier patch. Should be no functional change.
2009-09-15 22:35:13 +04:00
!vg_is_resizeable(vg)) {
Introduce as-yet-unused _vg_lock_and_read() and associated header file definitions.
2009-01-27 01:13:22 +03:00
log_error("Volume group %s is not resizeable.", vg->name);
failure |= FAILED_RESIZEABLE;
}
return failure;
}
Add vg_check_status to consolidate vg status flags checks and error messages.
2007-06-06 23:40:28 +04:00
/**
* vg_check_status - check volume group status flags and log error
* @vg - volume group to check status flags
exported vg handling The exported VG checking/enforcement was scattered and inconsistent. This centralizes it and makes it consistent, following the existing approach for foreign and shared VGs/PVs, which are very similar to exported VGs/PVs. The access policy that now applies to foreign/shared/exported VGs/PVs, is that if a foreign/shared/exported VG/PV is named on the command line (i.e. explicitly requested by the user), and the command is not permitted to operate on it because it is foreign/shared/exported, then an access error is reported and the command exits with an error. But, if the command is processing all VGs/PVs, and happens to come across a foreign/shared/exported VG/PV (that is not explicitly named on the command line), then the command silently skips it and does not produce an error. A command using tags or --select handles inaccessible VGs/PVs the same way as a command processing all VGs/PVs, and will not report/return errors if these inaccessible VGs/PVs exist. The new policy fixes the exit codes on a somewhat random set of commands that previously exited with an error if they were looking at all VGs/PVs and an exported VG existed on the system. There should be no change to which commands are allowed/disallowed on exported VGs/PVs. Certain LV commands (lvs/lvdisplay/lvscan) would previously not display LVs from an exported VG (for unknown reasons). This has not changed. The lvm fullreport command would previously report info about an exported VG but not about the LVs in it. This has changed to include all info from the exported VG.
2019-06-21 21:37:11 +03:00
* @status - specific status flags to check
Add vg_check_status to consolidate vg status flags checks and error messages.
2007-06-06 23:40:28 +04:00
*/
Switch status from 32-bit to 64-bit The physical_volume, volume_group, logical_volume and lv_segment structures' 'status' member is now uint64_t. The alignment of these structures was also audited to remove holes. The movement of some members in 'volume_group' and 'lv_segment' eliminates holes. The 'physical_volume' structure still has one 4-byte hole after 'pe_size'; the other structures no longer have any holes. Each structures' size has not changed.
2009-11-25 01:55:55 +03:00
int vg_check_status(const struct volume_group *vg, uint64_t status)
Add vg_check_status to consolidate vg status flags checks and error messages.
2007-06-06 23:40:28 +04:00
{
liblvm: Move lib code used exclusively into metadata-liblvm.c Also remove some redundant function definitions from metadata.h.
2017-10-18 21:29:32 +03:00
return !vg_bad_status_bits(vg, status);
Add vg_check_status to consolidate vg status flags checks and error messages.
2007-06-06 23:40:28 +04:00
}
Add get_pv_* functions to return PV fields in prep for external LVM library
2007-06-13 01:20:20 +04:00
metadata: change function name to _allow_extra_system_id The previous name was misleading since this is not the primary system_id check, only the "extra" check.
2015-07-14 22:42:18 +03:00
static int _allow_extra_system_id(struct cmd_context *cmd, const char *system_id)
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
{
const struct dm_config_node *cn;
const struct dm_config_value *cv;
const char *str;
config: replace find_config_tree_node with find_config_tree_array where appropriate
2015-07-08 12:22:24 +03:00
if (!(cn = find_config_tree_array(cmd, local_extra_system_ids_CFG, NULL)))
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
return 0;
for (cv = cn->v; cv; cv = cv->next) {
if (cv->type == DM_CFG_EMPTY_ARRAY)
break;
config: Rename allow_system_id to extra_system_ids. Add warnings to the config file templates and briefly document each value. Configure lvmlocal.conf and install in /etc/lvm.
2015-02-24 01:19:08 +03:00
/* Ignore invalid data: Warning message already issued by config.c */
if (cv->type != DM_CFG_STRING)
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
continue;
str = cv->v.str;
config: Rename allow_system_id to extra_system_ids. Add warnings to the config file templates and briefly document each value. Configure lvmlocal.conf and install in /etc/lvm.
2015-02-24 01:19:08 +03:00
if (!*str)
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
continue;
if (!strcmp(str, system_id))
return 1;
}
return 0;
}
Add lvmlockd
2015-03-05 23:00:44 +03:00
static int _access_vg_lock_type(struct cmd_context *cmd, struct volume_group *vg,
lockd: fix error message after a failing to get lock There are two different failure conditions detected in access_vg_lock_type() that should have different error messages. This adds another failure flag so the two cases can be distinguished to avoid printing a misleading error message.
2015-07-14 19:36:04 +03:00
uint32_t lockd_state, uint32_t *failure)
system_id: make new VGs read-only for old lvm versions Previous versions of lvm will not obey the restrictions imposed by the new system_id, and would allow such a VG to be written. So, a VG with a new system_id is further changed to force previous lvm versions to treat it as read-only. This is done by removing the WRITE flag from the metadata status line of these VGs, and putting a new WRITE_LOCKED flag in the flags line of the metadata. Versions of lvm that recognize WRITE_LOCKED, also obey the new system_id. For these lvm versions, WRITE_LOCKED is identical to WRITE, and the rules associated with matching system_id's are imposed. A new VG lock_type field is also added that causes the same WRITE/WRITE_LOCKED transformation when set. A previous version of lvm will also see a VG with lock_type as read-only. Versions of lvm that recognize WRITE_LOCKED, must also obey the lock_type setting. Until the lock_type feature is added, lvm will fail to read any VG with lock_type set and report an error about an unsupported lock_type. Once the lock_type feature is added, lvm will allow VGs with lock_type to be used according to the rules imposed by the lock_type. When both system_id and lock_type settings are removed, a VG is written with the old WRITE status flag, and without the new WRITE_LOCKED flag. This allows old versions of lvm to use the VG as before.
2015-03-04 20:30:53 +03:00
{
Add lvmlockd
2015-03-05 23:00:44 +03:00
if (cmd->lockd_vg_disable)
return 1;
system_id: make new VGs read-only for old lvm versions Previous versions of lvm will not obey the restrictions imposed by the new system_id, and would allow such a VG to be written. So, a VG with a new system_id is further changed to force previous lvm versions to treat it as read-only. This is done by removing the WRITE flag from the metadata status line of these VGs, and putting a new WRITE_LOCKED flag in the flags line of the metadata. Versions of lvm that recognize WRITE_LOCKED, also obey the new system_id. For these lvm versions, WRITE_LOCKED is identical to WRITE, and the rules associated with matching system_id's are imposed. A new VG lock_type field is also added that causes the same WRITE/WRITE_LOCKED transformation when set. A previous version of lvm will also see a VG with lock_type as read-only. Versions of lvm that recognize WRITE_LOCKED, must also obey the lock_type setting. Until the lock_type feature is added, lvm will fail to read any VG with lock_type set and report an error about an unsupported lock_type. Once the lock_type feature is added, lvm will allow VGs with lock_type to be used according to the rules imposed by the lock_type. When both system_id and lock_type settings are removed, a VG is written with the old WRITE status flag, and without the new WRITE_LOCKED flag. This allows old versions of lvm to use the VG as before.
2015-03-04 20:30:53 +03:00
/*
Add lvmlockd
2015-03-05 23:00:44 +03:00
* Local VG requires no lock from lvmlockd.
*/
lvmlockd: primarily use vg_is_shared to check if a vg uses an lvmlockd lock_type, instead of the equivalent but longer is_lockd_type.
2018-06-01 18:04:54 +03:00
if (!vg_is_shared(vg))
Add lvmlockd
2015-03-05 23:00:44 +03:00
return 1;
/*
* When lvmlockd is not used, lockd VGs are ignored by lvm
* and cannot be used, with two exceptions:
*
* . The --shared option allows them to be revealed with
* reporting/display commands.
*
* . If a command asks to operate on one specifically
* by name, then an error is printed.
system_id: make new VGs read-only for old lvm versions Previous versions of lvm will not obey the restrictions imposed by the new system_id, and would allow such a VG to be written. So, a VG with a new system_id is further changed to force previous lvm versions to treat it as read-only. This is done by removing the WRITE flag from the metadata status line of these VGs, and putting a new WRITE_LOCKED flag in the flags line of the metadata. Versions of lvm that recognize WRITE_LOCKED, also obey the new system_id. For these lvm versions, WRITE_LOCKED is identical to WRITE, and the rules associated with matching system_id's are imposed. A new VG lock_type field is also added that causes the same WRITE/WRITE_LOCKED transformation when set. A previous version of lvm will also see a VG with lock_type as read-only. Versions of lvm that recognize WRITE_LOCKED, must also obey the lock_type setting. Until the lock_type feature is added, lvm will fail to read any VG with lock_type set and report an error about an unsupported lock_type. Once the lock_type feature is added, lvm will allow VGs with lock_type to be used according to the rules imposed by the lock_type. When both system_id and lock_type settings are removed, a VG is written with the old WRITE status flag, and without the new WRITE_LOCKED flag. This allows old versions of lvm to use the VG as before.
2015-03-04 20:30:53 +03:00
*/
Add lvmlockd
2015-03-05 23:00:44 +03:00
if (!lvmlockd_use()) {
/*
* Some reporting/display commands have the --shared option
* (like --foreign) to allow them to reveal lockd VGs that
* are otherwise ignored. The --shared option must only be
* permitted in commands that read the VG for report or display,
* not any that write the VG or activate LVs.
*/
if (cmd->include_shared_vgs)
return 1;
/*
* Some commands want the error printed by vg_read, others by ignore_vg.
* Those using ignore_vg may choose to skip the error.
*/
if (cmd->vg_read_print_access_error) {
log_error("Cannot access VG %s with lock type %s that requires lvmlockd.",
vg->name, vg->lock_type);
}
lockd: fix error message after a failing to get lock There are two different failure conditions detected in access_vg_lock_type() that should have different error messages. This adds another failure flag so the two cases can be distinguished to avoid printing a misleading error message.
2015-07-14 19:36:04 +03:00
*failure |= FAILED_LOCK_TYPE;
system_id: make new VGs read-only for old lvm versions Previous versions of lvm will not obey the restrictions imposed by the new system_id, and would allow such a VG to be written. So, a VG with a new system_id is further changed to force previous lvm versions to treat it as read-only. This is done by removing the WRITE flag from the metadata status line of these VGs, and putting a new WRITE_LOCKED flag in the flags line of the metadata. Versions of lvm that recognize WRITE_LOCKED, also obey the new system_id. For these lvm versions, WRITE_LOCKED is identical to WRITE, and the rules associated with matching system_id's are imposed. A new VG lock_type field is also added that causes the same WRITE/WRITE_LOCKED transformation when set. A previous version of lvm will also see a VG with lock_type as read-only. Versions of lvm that recognize WRITE_LOCKED, must also obey the lock_type setting. Until the lock_type feature is added, lvm will fail to read any VG with lock_type set and report an error about an unsupported lock_type. Once the lock_type feature is added, lvm will allow VGs with lock_type to be used according to the rules imposed by the lock_type. When both system_id and lock_type settings are removed, a VG is written with the old WRITE status flag, and without the new WRITE_LOCKED flag. This allows old versions of lvm to use the VG as before.
2015-03-04 20:30:53 +03:00
return 0;
}
Add lvmlockd
2015-03-05 23:00:44 +03:00
/*
* The lock request from lvmlockd failed. If the lock was ex,
* we cannot continue. If the lock was sh, we could also fail
* to continue but since the lock was sh, it means the VG is
* only being read, and it doesn't hurt to allow reading with
* no lock.
*/
if (lockd_state & LDST_FAIL) {
vgchange/lvchange: enforce the shared VG lock from lvmlockd The vgchange/lvchange activation commands read the VG, and don't write it, so they acquire a shared VG lock from lvmlockd. When other commands fail to acquire a shared VG lock from lvmlockd, a warning is printed and they continue without it. (Without it, the VG metadata they display from lvmetad may not be up to date.) vgchange/lvchange -a shouldn't continue without the shared lock for a couple reasons: . Usually they will just continue on and fail to acquire the LV locks for activation, so continuing is pointless. . More importantly, without the sh VG lock, the VG metadata used by the command may be stale, and the LV locks shown in the VG metadata may no longer be current. In the case of sanlock, this would result in odd, unpredictable errors when lvmlockd doesn't find the expected lock on disk. In the case of dlm, the invalid LV lock could be granted for the non-existing LV. The solution is to not continue after the shared lock fails, in the same way that a command fails if an exclusive lock fails.
2015-07-17 23:13:22 +03:00
if ((lockd_state & LDST_EX) || cmd->lockd_vg_enforce_sh) {
Add lvmlockd
2015-03-05 23:00:44 +03:00
log_error("Cannot access VG %s due to failed lock.", vg->name);
lockd: fix error message after a failing to get lock There are two different failure conditions detected in access_vg_lock_type() that should have different error messages. This adds another failure flag so the two cases can be distinguished to avoid printing a misleading error message.
2015-07-14 19:36:04 +03:00
*failure |= FAILED_LOCK_MODE;
Add lvmlockd
2015-03-05 23:00:44 +03:00
return 0;
}
tidy: prefer not using else after return clang-tidy: avoid using 'else' after return - give more readable code, and also saves indention level.
2017-07-19 17:16:12 +03:00
log_warn("Reading VG %s without a lock.", vg->name);
return 1;
Add lvmlockd
2015-03-05 23:00:44 +03:00
}
lvmlockd: test mode doesn't work The --test option is not yet compatible with shared VGs because changes are made in lvmlockd that cannot be reversed or faked.
2016-12-15 20:49:19 +03:00
if (test_mode()) {
log_error("Test mode is not yet supported with lock type %s.", vg->lock_type);
lvmlockd: set failure flag for test mode Set a failure flag when vg_read returns an error for test mode. The caller can segfault if there's an error with no flag set.
2019-10-04 18:07:24 +03:00
*failure |= FAILED_LOCK_TYPE;
lvmlockd: test mode doesn't work The --test option is not yet compatible with shared VGs because changes are made in lvmlockd that cannot be reversed or faked.
2016-12-15 20:49:19 +03:00
return 0;
}
system_id: make new VGs read-only for old lvm versions Previous versions of lvm will not obey the restrictions imposed by the new system_id, and would allow such a VG to be written. So, a VG with a new system_id is further changed to force previous lvm versions to treat it as read-only. This is done by removing the WRITE flag from the metadata status line of these VGs, and putting a new WRITE_LOCKED flag in the flags line of the metadata. Versions of lvm that recognize WRITE_LOCKED, also obey the new system_id. For these lvm versions, WRITE_LOCKED is identical to WRITE, and the rules associated with matching system_id's are imposed. A new VG lock_type field is also added that causes the same WRITE/WRITE_LOCKED transformation when set. A previous version of lvm will also see a VG with lock_type as read-only. Versions of lvm that recognize WRITE_LOCKED, must also obey the lock_type setting. Until the lock_type feature is added, lvm will fail to read any VG with lock_type set and report an error about an unsupported lock_type. Once the lock_type feature is added, lvm will allow VGs with lock_type to be used according to the rules imposed by the lock_type. When both system_id and lock_type settings are removed, a VG is written with the old WRITE status flag, and without the new WRITE_LOCKED flag. This allows old versions of lvm to use the VG as before.
2015-03-04 20:30:53 +03:00
return 1;
}
system_id: refactor check for allowed system_id Refactor the code that checks for an allowable system_id so that it can be used from other places.
2015-11-30 20:46:55 +03:00
int is_system_id_allowed(struct cmd_context *cmd, const char *system_id)
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
{
lvm1: Reenable sys ID. Move the lvm1 sys ID into vg->lvm1_system_id and reenable the #if 0 LVM1 code. Still display the new-style system ID in the same reporting field, though, as only one can be set. Add a format feature flag FMT_SYSTEM_ON_PVS for LVM1 and disallow access to LVM1 VGs if a new-style system ID has been set. Treat the new vg->system_id as const.
2015-02-24 02:03:52 +03:00
/*
system_id: refactor check for allowed system_id Refactor the code that checks for an allowable system_id so that it can be used from other places.
2015-11-30 20:46:55 +03:00
* A VG without a system_id can be accessed by anyone.
lvm1: Reenable sys ID. Move the lvm1 sys ID into vg->lvm1_system_id and reenable the #if 0 LVM1 code. Still display the new-style system ID in the same reporting field, though, as only one can be set. Add a format feature flag FMT_SYSTEM_ON_PVS for LVM1 and disallow access to LVM1 VGs if a new-style system ID has been set. Treat the new vg->system_id as const.
2015-02-24 02:03:52 +03:00
*/
system_id: refactor check for allowed system_id Refactor the code that checks for an allowable system_id so that it can be used from other places.
2015-11-30 20:46:55 +03:00
if (!system_id || !system_id[0])
return 1;
lvm1: Reenable sys ID. Move the lvm1 sys ID into vg->lvm1_system_id and reenable the #if 0 LVM1 code. Still display the new-style system ID in the same reporting field, though, as only one can be set. Add a format feature flag FMT_SYSTEM_ON_PVS for LVM1 and disallow access to LVM1 VGs if a new-style system ID has been set. Treat the new vg->system_id as const.
2015-02-24 02:03:52 +03:00
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
/*
system_id: refactor check for allowed system_id Refactor the code that checks for an allowable system_id so that it can be used from other places.
2015-11-30 20:46:55 +03:00
* Allowed if the host and VG system_id's match.
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
*/
system_id: refactor check for allowed system_id Refactor the code that checks for an allowable system_id so that it can be used from other places.
2015-11-30 20:46:55 +03:00
if (cmd->system_id && !strcmp(cmd->system_id, system_id))
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
return 1;
/*
system_id: refactor check for allowed system_id Refactor the code that checks for an allowable system_id so that it can be used from other places.
2015-11-30 20:46:55 +03:00
* Allowed if a host's extra system_id matches.
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
*/
system_id: refactor check for allowed system_id Refactor the code that checks for an allowable system_id so that it can be used from other places.
2015-11-30 20:46:55 +03:00
if (cmd->system_id && _allow_extra_system_id(cmd, system_id))
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
return 1;
systemid: Fix access restrictions. When checking whether the system ID permits access to a VG, check for each permitted situation first, and only then issue the appropriate error message. Always issue a message for now. (We'll try to suppress some of those later when the VG concerned wasn't explicitly requested.) Add more messages to try to ensure every return code is checked and every error path (and only an error path) contains a log_error(). Add self-correction to vgchange -c to deal with situations where the cluster state and system ID state are out-of-sync (e.g. if old tools were used).
2015-02-24 02:19:36 +03:00
/*
system_id: refactor check for allowed system_id Refactor the code that checks for an allowable system_id so that it can be used from other places.
2015-11-30 20:46:55 +03:00
* Not allowed if the host does not have a system_id
* and the VG does, or if the host and VG's system_id's
* do not match.
systemid: Fix access restrictions. When checking whether the system ID permits access to a VG, check for each permitted situation first, and only then issue the appropriate error message. Always issue a message for now. (We'll try to suppress some of those later when the VG concerned wasn't explicitly requested.) Add more messages to try to ensure every return code is checked and every error path (and only an error path) contains a log_error(). Add self-correction to vgchange -c to deal with situations where the cluster state and system ID state are out-of-sync (e.g. if old tools were used).
2015-02-24 02:19:36 +03:00
*/
system_id: refactor check for allowed system_id Refactor the code that checks for an allowable system_id so that it can be used from other places.
2015-11-30 20:46:55 +03:00
return 0;
}
static int _access_vg_systemid(struct cmd_context *cmd, struct volume_group *vg)
{
/*
* A few commands allow read-only access to foreign VGs.
*/
if (cmd->include_foreign_vgs)
return 1;
if (is_system_id_allowed(cmd, vg->system_id))
systemid: Fix access restrictions. When checking whether the system ID permits access to a VG, check for each permitted situation first, and only then issue the appropriate error message. Always issue a message for now. (We'll try to suppress some of those later when the VG concerned wasn't explicitly requested.) Add more messages to try to ensure every return code is checked and every error path (and only an error path) contains a log_error(). Add self-correction to vgchange -c to deal with situations where the cluster state and system ID state are out-of-sync (e.g. if old tools were used).
2015-02-24 02:19:36 +03:00
return 1;
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
/*
* Allow VG access if the local host has active LVs in it.
*/
if (lvs_in_vg_activated(vg)) {
system_id: apply consistent naming In log messages refer to it as system ID (not System ID). Do not put quotes around the system_id string when printing. On the command line use systemid. In code, metadata, and config files use system_id. In lvmsystemid refer to the concept/entity as system_id.
2015-02-27 22:32:00 +03:00
log_warn("WARNING: Found LVs active in VG %s with foreign system ID %s. Possible data corruption.",
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
vg->name, vg->system_id);
lvchange: deactivate is always possible in foreign vgs The only realistic way for a host to have active LVs in a foreign VG is if the host's system_id (or system_id_source) is changed while LVs are active. In this case, the active LVs produce an warning, and access to the VG is implicitly allowed (without requiring --foreign.) This allows the active LVs to be deactivated. In this case, rescanning PVs for the VG offers no benefit. It is not possible that rescanning would reveal an LV that is active but wasn't previously in the VG metadata.
2015-02-25 20:33:11 +03:00
if (cmd->include_active_foreign_vgs)
return 1;
return 0;
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
}
/*
system_id: refactor check for allowed system_id Refactor the code that checks for an allowable system_id so that it can be used from other places.
2015-11-30 20:46:55 +03:00
* Print an error when reading a VG that has a system_id
* and the host system_id is unknown.
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
*/
if (!cmd->system_id || cmd->unknown_system_id) {
system_id: apply consistent naming In log messages refer to it as system ID (not System ID). Do not put quotes around the system_id string when printing. On the command line use systemid. In code, metadata, and config files use system_id. In lvmsystemid refer to the concept/entity as system_id.
2015-02-27 22:32:00 +03:00
log_error("Cannot access VG %s with system ID %s with unknown local system ID.",
systemid: Fix access restrictions. When checking whether the system ID permits access to a VG, check for each permitted situation first, and only then issue the appropriate error message. Always issue a message for now. (We'll try to suppress some of those later when the VG concerned wasn't explicitly requested.) Add more messages to try to ensure every return code is checked and every error path (and only an error path) contains a log_error(). Add self-correction to vgchange -c to deal with situations where the cluster state and system ID state are out-of-sync (e.g. if old tools were used).
2015-02-24 02:19:36 +03:00
vg->name, vg->system_id);
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
return 0;
}
systemid: silently ignore foreign vgs unless named A foreign VG should be silently ignored by a reporting/display command like 'vgs'. If the reporting/display command specifies a foreign VG by name on the command line, it should produce an error message. Scanning commands pvscan/vgscan/lvscan are always allowed to read and update caches from all PVs, including those that belong to foreign VGs. Other non-report/display/scan commands always ignore a foreign VG, or report an error if they attempt to use a foreign VG. vgimport should always invalidate the lvmetad cache because lvmetad likely holds a pre-vgexported copy of the VG. (This is unrelated to using foreign VGs; the pre-vgexported VG may have had no system_id at all.)
2015-02-25 19:44:42 +03:00
/*
Add lvmlockd
2015-03-05 23:00:44 +03:00
* Some commands want the error printed by vg_read, others by ignore_vg.
* Those using ignore_vg may choose to skip the error.
systemid: silently ignore foreign vgs unless named A foreign VG should be silently ignored by a reporting/display command like 'vgs'. If the reporting/display command specifies a foreign VG by name on the command line, it should produce an error message. Scanning commands pvscan/vgscan/lvscan are always allowed to read and update caches from all PVs, including those that belong to foreign VGs. Other non-report/display/scan commands always ignore a foreign VG, or report an error if they attempt to use a foreign VG. vgimport should always invalidate the lvmetad cache because lvmetad likely holds a pre-vgexported copy of the VG. (This is unrelated to using foreign VGs; the pre-vgexported VG may have had no system_id at all.)
2015-02-25 19:44:42 +03:00
*/
Add lvmlockd
2015-03-05 23:00:44 +03:00
if (cmd->vg_read_print_access_error) {
system_id: apply consistent naming In log messages refer to it as system ID (not System ID). Do not put quotes around the system_id string when printing. On the command line use systemid. In code, metadata, and config files use system_id. In lvmsystemid refer to the concept/entity as system_id.
2015-02-27 22:32:00 +03:00
log_error("Cannot access VG %s with system ID %s with local system ID %s.",
systemid: silently ignore foreign vgs unless named A foreign VG should be silently ignored by a reporting/display command like 'vgs'. If the reporting/display command specifies a foreign VG by name on the command line, it should produce an error message. Scanning commands pvscan/vgscan/lvscan are always allowed to read and update caches from all PVs, including those that belong to foreign VGs. Other non-report/display/scan commands always ignore a foreign VG, or report an error if they attempt to use a foreign VG. vgimport should always invalidate the lvmetad cache because lvmetad likely holds a pre-vgexported copy of the VG. (This is unrelated to using foreign VGs; the pre-vgexported VG may have had no system_id at all.)
2015-02-25 19:44:42 +03:00
vg->name, vg->system_id, cmd->system_id);
return 0;
}
Add lvmlockd
2015-03-05 23:00:44 +03:00
/* Silently ignore foreign vgs. */
system_id: use for VG ownership See included lvmsystemid(7) for full description.
2014-10-24 21:29:04 +04:00
return 0;
}
exported vg handling The exported VG checking/enforcement was scattered and inconsistent. This centralizes it and makes it consistent, following the existing approach for foreign and shared VGs/PVs, which are very similar to exported VGs/PVs. The access policy that now applies to foreign/shared/exported VGs/PVs, is that if a foreign/shared/exported VG/PV is named on the command line (i.e. explicitly requested by the user), and the command is not permitted to operate on it because it is foreign/shared/exported, then an access error is reported and the command exits with an error. But, if the command is processing all VGs/PVs, and happens to come across a foreign/shared/exported VG/PV (that is not explicitly named on the command line), then the command silently skips it and does not produce an error. A command using tags or --select handles inaccessible VGs/PVs the same way as a command processing all VGs/PVs, and will not report/return errors if these inaccessible VGs/PVs exist. The new policy fixes the exit codes on a somewhat random set of commands that previously exited with an error if they were looking at all VGs/PVs and an exported VG existed on the system. There should be no change to which commands are allowed/disallowed on exported VGs/PVs. Certain LV commands (lvs/lvdisplay/lvscan) would previously not display LVs from an exported VG (for unknown reasons). This has not changed. The lvm fullreport command would previously report info about an exported VG but not about the LVs in it. This has changed to include all info from the exported VG.
2019-06-21 21:37:11 +03:00
static int _access_vg_exported(struct cmd_context *cmd, struct volume_group *vg)
{
if (!vg_is_exported(vg))
return 1;
if (cmd->include_exported_vgs)
return 1;
/*
* Some commands want the error printed by vg_read, others by ignore_vg.
* Those using ignore_vg may choose to skip the error.
*/
if (cmd->vg_read_print_access_error) {
log_error("Volume group %s is exported", vg->name);
return 0;
}
/* Silently ignore exported vgs. */
return 0;
}
Use new alloc_fid fn for common format instance initialisation.
2011-03-11 17:30:27 +03:00
struct format_instance *alloc_fid(const struct format_type *fmt,
const struct format_instance_ctx *fic)
{
Add mem and ref_count fields to struct format_instance for own mempool use. Format instances can be created anytime on demand and it contains metadata area information mostly (at least for now, but in the future, we may store more things here to update/edit in a PV/VG). In case we have lots of metadata areas, memory consumption will rise. Using cmd context mempool is not quite optimal here because it is destroyed too late. So let's use a separate mempool for format instances. Reference counting is used because fids could be shared, e.g. each PV has either a PV-based fid or VG-based fid. If it's VG-based, each PV has a shared fid with the VG - a reference to VG's fid.
2011-03-11 17:38:38 +03:00
struct dm_pool *mem;
Use new alloc_fid fn for common format instance initialisation.
2011-03-11 17:30:27 +03:00
struct format_instance *fid;
Add mem and ref_count fields to struct format_instance for own mempool use. Format instances can be created anytime on demand and it contains metadata area information mostly (at least for now, but in the future, we may store more things here to update/edit in a PV/VG). In case we have lots of metadata areas, memory consumption will rise. Using cmd context mempool is not quite optimal here because it is destroyed too late. So let's use a separate mempool for format instances. Reference counting is used because fids could be shared, e.g. each PV has either a PV-based fid or VG-based fid. If it's VG-based, each PV has a shared fid with the VG - a reference to VG's fid.
2011-03-11 17:38:38 +03:00
if (!(mem = dm_pool_create("format_instance", 1024)))
return_NULL;
Use format instance mempool where possible and adequate.
2011-03-11 18:10:16 +03:00
if (!(fid = dm_pool_zalloc(mem, sizeof(*fid)))) {
Use new alloc_fid fn for common format instance initialisation.
2011-03-11 17:30:27 +03:00
log_error("Couldn't allocate format_instance object.");
Add mem and ref_count fields to struct format_instance for own mempool use. Format instances can be created anytime on demand and it contains metadata area information mostly (at least for now, but in the future, we may store more things here to update/edit in a PV/VG). In case we have lots of metadata areas, memory consumption will rise. Using cmd context mempool is not quite optimal here because it is destroyed too late. So let's use a separate mempool for format instances. Reference counting is used because fids could be shared, e.g. each PV has either a PV-based fid or VG-based fid. If it's VG-based, each PV has a shared fid with the VG - a reference to VG's fid.
2011-03-11 17:38:38 +03:00
goto bad;
Use new alloc_fid fn for common format instance initialisation.
2011-03-11 17:30:27 +03:00
}
Add mem and ref_count fields to struct format_instance for own mempool use. Format instances can be created anytime on demand and it contains metadata area information mostly (at least for now, but in the future, we may store more things here to update/edit in a PV/VG). In case we have lots of metadata areas, memory consumption will rise. Using cmd context mempool is not quite optimal here because it is destroyed too late. So let's use a separate mempool for format instances. Reference counting is used because fids could be shared, e.g. each PV has either a PV-based fid or VG-based fid. If it's VG-based, each PV has a shared fid with the VG - a reference to VG's fid.
2011-03-11 17:38:38 +03:00
fid->ref_count = 1;
fid->mem = mem;
Use new alloc_fid fn for common format instance initialisation.
2011-03-11 17:30:27 +03:00
fid->type = fic->type;
Add mem and ref_count fields to struct format_instance for own mempool use. Format instances can be created anytime on demand and it contains metadata area information mostly (at least for now, but in the future, we may store more things here to update/edit in a PV/VG). In case we have lots of metadata areas, memory consumption will rise. Using cmd context mempool is not quite optimal here because it is destroyed too late. So let's use a separate mempool for format instances. Reference counting is used because fids could be shared, e.g. each PV has either a PV-based fid or VG-based fid. If it's VG-based, each PV has a shared fid with the VG - a reference to VG's fid.
2011-03-11 17:38:38 +03:00
fid->fmt = fmt;
Use new alloc_fid fn for common format instance initialisation.
2011-03-11 17:30:27 +03:00
dm_list_init(&fid->metadata_areas_in_use);
dm_list_init(&fid->metadata_areas_ignored);
return fid;
Add mem and ref_count fields to struct format_instance for own mempool use. Format instances can be created anytime on demand and it contains metadata area information mostly (at least for now, but in the future, we may store more things here to update/edit in a PV/VG). In case we have lots of metadata areas, memory consumption will rise. Using cmd context mempool is not quite optimal here because it is destroyed too late. So let's use a separate mempool for format instances. Reference counting is used because fids could be shared, e.g. each PV has either a PV-based fid or VG-based fid. If it's VG-based, each PV has a shared fid with the VG - a reference to VG's fid.
2011-03-11 17:38:38 +03:00
bad:
dm_pool_destroy(mem);
return NULL;
Use new alloc_fid fn for common format instance initialisation.
2011-03-11 17:30:27 +03:00
}
Use only vg_set_fid and new pv_set_fid fn to assign the format instance. This is essential for proper format instance ref_count support. We must use these functions to set the fid everywhere from now on, even the NULL value!
2011-03-11 17:50:13 +03:00
void pv_set_fid(struct physical_volume *pv,
struct format_instance *fid)
{
If the same fid is already same avoid ref_counting
2012-02-13 15:01:34 +04:00
if (fid == pv->fid)
return;
Cleanup fid finalization code in free_vg and allow exactly the same fid to be set again for a PV/VG. Actually, we can call vg_set_fid(vg, NULL) instead of calling destroy_instance for all PV structs and a VG struct - it's the same code we already have in the vg_set_fid. Also, allow exactly the same fid to be set again for the same PV/VG Before, this could end up with the fid destroyed because we destroyed existing fid first and then we used the new one and we didn't care whether existing one == new one by chance.
2011-04-01 18:54:20 +04:00
if (fid)
fid->ref_count++;
Use only vg_set_fid and new pv_set_fid fn to assign the format instance. This is essential for proper format instance ref_count support. We must use these functions to set the fid everywhere from now on, even the NULL value!
2011-03-11 17:50:13 +03:00
if (pv->fid)
pv->fid->fmt->ops->destroy_instance(pv->fid);
pv->fid = fid;
}
Add vg_set_fid function to change VG format instance. This function also sets a reference to a new VG format instance for all PVs that are part of the VG so the PV-VG interconnection is consistent after the change.
2011-02-21 15:10:58 +03:00
void vg_set_fid(struct volume_group *vg,
struct format_instance *fid)
{
struct pv_list *pvl;
If the same fid is already same avoid ref_counting
2012-02-13 15:01:34 +04:00
if (fid == vg->fid)
return;
Use only vg_set_fid and new pv_set_fid fn to assign the format instance. This is essential for proper format instance ref_count support. We must use these functions to set the fid everywhere from now on, even the NULL value!
2011-03-11 17:50:13 +03:00
if (fid)
fid->ref_count++;
Add vg_set_fid function to change VG format instance. This function also sets a reference to a new VG format instance for all PVs that are part of the VG so the PV-VG interconnection is consistent after the change.
2011-02-21 15:10:58 +03:00
dm_list_iterate_items(pvl, &vg->pvs)
Use only vg_set_fid and new pv_set_fid fn to assign the format instance. This is essential for proper format instance ref_count support. We must use these functions to set the fid everywhere from now on, even the NULL value!
2011-03-11 17:50:13 +03:00
pv_set_fid(pvl->pv, fid);
Cleanup fid finalization code in free_vg and allow exactly the same fid to be set again for a PV/VG. Actually, we can call vg_set_fid(vg, NULL) instead of calling destroy_instance for all PV structs and a VG struct - it's the same code we already have in the vg_set_fid. Also, allow exactly the same fid to be set again for the same PV/VG Before, this could end up with the fid destroyed because we destroyed existing fid first and then we used the new one and we didn't care whether existing one == new one by chance.
2011-04-01 18:54:20 +04:00
dm_list_iterate_items(pvl, &vg->removed_pvs)
pv_set_fid(pvl->pv, fid);
if (vg->fid)
vg->fid->fmt->ops->destroy_instance(vg->fid);
vg->fid = fid;
Add vg_set_fid function to change VG format instance. This function also sets a reference to a new VG format instance for all PVs that are part of the VG so the PV-VG interconnection is consistent after the change.
2011-02-21 15:10:58 +03:00
}
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
static int _convert_key_to_string(const char *key, size_t key_len,
unsigned sub_key, char *buf, size_t buf_len)
Add metadata_areas_ignored list and functions to manage ignored mdas. Add a second mda list, metadata_areas_ignored to fid, and a couple functions, fid_add_mda() and fid_add_mdas() to help manage the lists. These functions are needed to properly count the ignored mdas and manage the lists attached to the 'fid' and ultimately the 'vg'. Ensure metadata_areas_ignored is initialized in other formats, even if the list is never used. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:33:22 +04:00
{
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
memcpy(buf, key, key_len);
buf += key_len;
buf_len -= key_len;
if ((dm_snprintf(buf, buf_len, "_%u", sub_key) == -1))
return_0;
return 1;
}
int fid_add_mda(struct format_instance *fid, struct metadata_area *mda,
const char *key, size_t key_len, const unsigned sub_key)
{
Replace dynamic buffer allocations for PATH_MAX Use static buffer instead of stack allocated buffer. This reduces stack size usage of lvm tool and the change is very simple. Since the whole library is not thread safe - it should not add any new problems - and if there will be some conversion it's easy to convert this to use some preallocated buffer.
2011-11-18 23:31:09 +04:00
static char full_key[PATH_MAX];
The lvmetad client-side integration. Only active when use_lvmetad = 1 is set in lvm.conf *and* lvmetad is running.
2012-02-23 17:11:07 +04:00
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
dm_list_add(mda_is_ignored(mda) ? &fid->metadata_areas_ignored :
The lvmetad client-side integration. Only active when use_lvmetad = 1 is set in lvm.conf *and* lvmetad is running.
2012-02-23 17:11:07 +04:00
&fid->metadata_areas_in_use, &mda->list);
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
/* Return if the mda is not supposed to be indexed. */
if (!key)
return 1;
metadata: Make the fid mda routines a little more resilient.
2013-11-18 21:00:49 +04:00
if (!fid->metadata_areas_index)
return_0;
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
/* Add metadata area to index. */
FMT_INSTANCE_VG is redundant now
2012-02-13 03:01:19 +04:00
if (!_convert_key_to_string(key, key_len, sub_key,
full_key, sizeof(full_key)))
The lvmetad client-side integration. Only active when use_lvmetad = 1 is set in lvm.conf *and* lvmetad is running.
2012-02-23 17:11:07 +04:00
return_0;
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
Test dm_hash_insert() failures mem failures
2012-02-28 15:12:58 +04:00
if (!dm_hash_insert(fid->metadata_areas_index,
full_key, mda)) {
log_error("Failed to hash mda.");
return 0;
}
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
return 1;
Add metadata_areas_ignored list and functions to manage ignored mdas. Add a second mda list, metadata_areas_ignored to fid, and a couple functions, fid_add_mda() and fid_add_mdas() to help manage the lists. These functions are needed to properly count the ignored mdas and manage the lists attached to the 'fid' and ultimately the 'vg'. Ensure metadata_areas_ignored is initialized in other formats, even if the list is never used. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:33:22 +04:00
}
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
int fid_add_mdas(struct format_instance *fid, struct dm_list *mdas,
const char *key, size_t key_len)
Add metadata_areas_ignored list and functions to manage ignored mdas. Add a second mda list, metadata_areas_ignored to fid, and a couple functions, fid_add_mda() and fid_add_mdas() to help manage the lists. These functions are needed to properly count the ignored mdas and manage the lists attached to the 'fid' and ultimately the 'vg'. Ensure metadata_areas_ignored is initialized in other formats, even if the list is never used. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:33:22 +04:00
{
struct metadata_area *mda, *mda_new;
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
unsigned mda_index = 0;
Add metadata_areas_ignored list and functions to manage ignored mdas. Add a second mda list, metadata_areas_ignored to fid, and a couple functions, fid_add_mda() and fid_add_mdas() to help manage the lists. These functions are needed to properly count the ignored mdas and manage the lists attached to the 'fid' and ultimately the 'vg'. Ensure metadata_areas_ignored is initialized in other formats, even if the list is never used. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:33:22 +04:00
dm_list_iterate_items(mda, mdas) {
Use format instance mempool where possible and adequate.
2011-03-11 18:10:16 +03:00
mda_new = mda_copy(fid->mem, mda);
Add metadata_areas_ignored list and functions to manage ignored mdas. Add a second mda list, metadata_areas_ignored to fid, and a couple functions, fid_add_mda() and fid_add_mdas() to help manage the lists. These functions are needed to properly count the ignored mdas and manage the lists attached to the 'fid' and ultimately the 'vg'. Ensure metadata_areas_ignored is initialized in other formats, even if the list is never used. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:33:22 +04:00
if (!mda_new)
return_0;
Read PV metadata information from cache if pv_setup called with pv->fid == vg->fid. If the PV is already part of the VG (so the pv->fid == vg->fid), it makes no sense to attach the mdas information from PV to a VG. Instead, we read new PV metadata information from cache and attach it to the VG fid.
2011-02-25 16:59:47 +03:00
fid_remove_mda(fid, NULL, key, key_len, mda_index);
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
fid_add_mda(fid, mda_new, key, key_len, mda_index);
mda_index++;
Add metadata_areas_ignored list and functions to manage ignored mdas. Add a second mda list, metadata_areas_ignored to fid, and a couple functions, fid_add_mda() and fid_add_mdas() to help manage the lists. These functions are needed to properly count the ignored mdas and manage the lists attached to the 'fid' and ultimately the 'vg'. Ensure metadata_areas_ignored is initialized in other formats, even if the list is never used. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:33:22 +04:00
}
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
return 1;
}
struct metadata_area *fid_get_mda_indexed(struct format_instance *fid,
const char *key, size_t key_len,
const unsigned sub_key)
{
Replace dynamic buffer allocations for PATH_MAX Use static buffer instead of stack allocated buffer. This reduces stack size usage of lvm tool and the change is very simple. Since the whole library is not thread safe - it should not add any new problems - and if there will be some conversion it's easy to convert this to use some preallocated buffer.
2011-11-18 23:31:09 +04:00
static char full_key[PATH_MAX];
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
struct metadata_area *mda = NULL;
metadata: Make the fid mda routines a little more resilient.
2013-11-18 21:00:49 +04:00
if (!fid->metadata_areas_index)
return_NULL;
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
FMT_INSTANCE_VG is redundant now
2012-02-13 03:01:19 +04:00
if (!_convert_key_to_string(key, key_len, sub_key,
full_key, sizeof(full_key)))
return_NULL;
metadata: Make the fid mda routines a little more resilient.
2013-11-18 21:00:49 +04:00
FMT_INSTANCE_VG is redundant now
2012-02-13 03:01:19 +04:00
mda = (struct metadata_area *) dm_hash_lookup(fid->metadata_areas_index,
full_key);
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
return mda;
}
int fid_remove_mda(struct format_instance *fid, struct metadata_area *mda,
const char *key, size_t key_len, const unsigned sub_key)
{
Replace dynamic buffer allocations for PATH_MAX Use static buffer instead of stack allocated buffer. This reduces stack size usage of lvm tool and the change is very simple. Since the whole library is not thread safe - it should not add any new problems - and if there will be some conversion it's easy to convert this to use some preallocated buffer.
2011-11-18 23:31:09 +04:00
static char full_key[PATH_MAX];
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
struct metadata_area *mda_indexed = NULL;
/* At least one of mda or key must be specified. */
if (!mda && !key)
return 1;
if (key) {
/*
* If both mda and key specified, check given mda
* with what we find using the index and return
* immediately if these two do not match.
*/
if (!(mda_indexed = fid_get_mda_indexed(fid, key, key_len, sub_key)) ||
(mda && mda != mda_indexed))
return 1;
mda = mda_indexed;
FMT_INSTANCE_VG is redundant now
2012-02-13 03:01:19 +04:00
if (!_convert_key_to_string(key, key_len, sub_key,
full_key, sizeof(full_key)))
return_0;
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
FMT_INSTANCE_VG is redundant now
2012-02-13 03:01:19 +04:00
dm_hash_remove(fid->metadata_areas_index, full_key);
Change create_instance to create PV-based as well as VG-based format instances. Add supporting functions to work with the format instance and metadata area structures stored within the format instance. Add support for simple indexing of metadata areas using PV id and mda order (for on-disk PV only for now, we can extend the indexing even for other mdas if needed - we only need to define a proper key for the index).
2011-02-21 15:05:49 +03:00
}
dm_list_del(&mda->list);
Add metadata_areas_ignored list and functions to manage ignored mdas. Add a second mda list, metadata_areas_ignored to fid, and a couple functions, fid_add_mda() and fid_add_mdas() to help manage the lists. These functions are needed to properly count the ignored mdas and manage the lists attached to the 'fid' and ultimately the 'vg'. Ensure metadata_areas_ignored is initialized in other formats, even if the list is never used. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:33:22 +04:00
return 1;
}
Add mda location specific mda_copy constructor. Because of the way mdas are handled internally, where a PV in a VG has mdas on both info->mdas and vg->fid->metadata_areas list, we need a location independent copy constructor for struct metadata_area. Break up the existing format-text specific copy constructor into a format independent piece and a format dependent piece. This function is necessary to properly implement pv_set_mda_ignored(). Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Reviewed-by: Alasdair G Kergon <agk@redhat.com>
2010-06-29 00:31:59 +04:00
/*
* Copy constructor for a metadata_area.
*/
struct metadata_area *mda_copy(struct dm_pool *mem,
struct metadata_area *mda)
{
struct metadata_area *mda_new;
if (!(mda_new = dm_pool_alloc(mem, sizeof(*mda_new)))) {
log_error("metadata_area allocation failed");
return NULL;
}
memcpy(mda_new, mda, sizeof(*mda));
if (mda->ops->mda_metadata_locn_copy && mda->metadata_locn) {
mda_new->metadata_locn =
mda->ops->mda_metadata_locn_copy(mem, mda->metadata_locn);
if (!mda_new->metadata_locn) {
dm_pool_free(mem, mda_new);
return NULL;
}
}
Init mda->list in mda_copy. This patch should be no functional change as all callers initialize mda->list.
2010-07-08 21:41:46 +04:00
dm_list_init(&mda_new->list);
Add mda location specific mda_copy constructor. Because of the way mdas are handled internally, where a PV in a VG has mdas on both info->mdas and vg->fid->metadata_areas list, we need a location independent copy constructor for struct metadata_area. Break up the existing format-text specific copy constructor into a format independent piece and a format dependent piece. This function is necessary to properly implement pv_set_mda_ignored(). Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Reviewed-by: Alasdair G Kergon <agk@redhat.com>
2010-06-29 00:31:59 +04:00
return mda_new;
}
Add mda_locns_match() internal library function for mapping pv/device to VG mda. A metadata_area is defined independent of the location. One downside is that there is no obvious mapping from a pv to an mda. For a PV in a VG, we need a way to start with a PV and end up with an MDA, if we are to manage mdas starting with a device/pv. This function provides us a way to go down the list of PVs on a VG, and identify which ones match a particular PV. I'm not entirely happy with this approach, but it does fit into the existing structures in a reasonable way. An alternative solution might be to refactor the VG - PV interface such that mdas are a list tied to a PV. However, this seemed a bit tricky since a PV does not come into existence until after the list of mdas is constructed (see _vg_read() - we create a 'fid' and attach mdas to it, then we go through them and attach pvs). Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Reviewed-by: Alasdair G Kergon <agk@redhat.com>
2010-06-29 00:31:38 +04:00
/*
* This function provides a way to answer the question on a format specific
* basis - does the format specfic context of these two metadata areas
* match?
*
* A metatdata_area is defined to be independent of the underlying context.
* This has the benefit that we can use the same abstraction to read disks
* (see _metadata_text_raw_ops) or files (see _metadata_text_file_ops).
* However, one downside is there is no format-independent way to determine
* whether a given metadata_area is attached to a specific device - in fact,
* it may not be attached to a device at all.
*
* Thus, LVM is structured such that an mda is not a member of struct
* physical_volume. The location of the mda depends on whether
* the PV is in a volume group. A PV not in a VG has an mda on the
* 'info->mda' list in lvmcache, while a PV in a VG has an mda on
Rename fid->metadata_areas to fid->metadata_areas_in_use. Rename the metadata_areas list to an 'in_use' list to prepare for future 'ignored' list.
2010-06-29 00:32:44 +04:00
* the vg->fid->metadata_areas_in_use list. For further details, see _vg_read(),
* and the sequence of creating the format_instance with fid->metadata_areas_in_use
Add mda_locns_match() internal library function for mapping pv/device to VG mda. A metadata_area is defined independent of the location. One downside is that there is no obvious mapping from a pv to an mda. For a PV in a VG, we need a way to start with a PV and end up with an MDA, if we are to manage mdas starting with a device/pv. This function provides us a way to go down the list of PVs on a VG, and identify which ones match a particular PV. I'm not entirely happy with this approach, but it does fit into the existing structures in a reasonable way. An alternative solution might be to refactor the VG - PV interface such that mdas are a list tied to a PV. However, this seemed a bit tricky since a PV does not come into existence until after the list of mdas is constructed (see _vg_read() - we create a 'fid' and attach mdas to it, then we go through them and attach pvs). Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Reviewed-by: Alasdair G Kergon <agk@redhat.com>
2010-06-29 00:31:38 +04:00
* list, as well as the construction of the VG, with list of PVs (comes
* after the construction of the fid and list of mdas).
*/
unsigned mda_locns_match(struct metadata_area *mda1, struct metadata_area *mda2)
{
if (!mda1->ops->mda_locns_match || !mda2->ops->mda_locns_match ||
mda1->ops->mda_locns_match != mda2->ops->mda_locns_match)
return 0;
return mda1->ops->mda_locns_match(mda1, mda2);
}
Add vg_status function and clean up vg->status in tools directory
2007-06-19 08:23:32 +04:00
Fix RHBZ 651590 (failure to lock LV results in failure to repair mirror after transient error), stemming from the following sequence of events: 1) devices fail IO, triggering repair 2) dmeventd starts fixing up the mirror 3) during the downconversion, a new metadata version is written --> the devices come back online here 4) the mirror device suspend/resume is called to update DM tables 5) during the suspend/resume cycle, *pre*-commit metadata is read; however, since the failed devices are now back online, we get back inconsistent set of precommit metadata and the whole operation fails The patch relaxes the check that fails in step 5 above, namely by ignoring inconsistencies coming from PVs that are marked MISSING.
2011-06-15 21:45:02 +04:00
struct device *mda_get_device(struct metadata_area *mda)
{
if (!mda->ops->mda_get_device)
return NULL;
return mda->ops->mda_get_device(mda);
}
Add location independent flag and functions to ignore mdas. First we add a 'flags' field to the location independent metadata_area structure, and a MDA_IGNORE flag. The mda_is_ignored and mda_set_ignored functions are added to manage the flag. Adding the flag and functions gives a library interface to ignore metadata areas independent of the underlying location (disk, file, etc). The location specific read/write functions must then handle the specifics of what this flag means to the location. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Reviewed-by: Alasdair G Kergon <agk@redhat.com>
2010-06-29 00:30:14 +04:00
unsigned mda_is_ignored(struct metadata_area *mda)
{
Rename 'flags' to 'status' for struct metadata_area. In other LVM memory structures such as volume_group, the field used to store flags is called "status", and on-disk fields are called 'flags', so rename the one inside metadata_area to be consistent. Not only is it more consistent with existing code but is cleaner to say "the status of this mda is ignored". Background for this patch - prajnoha pinged me on IRC this morning about a fix he was working on related to metadataignore when metadata/dirs was set. I was reviewing my patches from this year and realized the 'flags' field was probably not the best choice when I originally did the metadataignore patches.
2010-10-05 21:34:05 +04:00
return (mda->status & MDA_IGNORED);
Add location independent flag and functions to ignore mdas. First we add a 'flags' field to the location independent metadata_area structure, and a MDA_IGNORE flag. The mda_is_ignored and mda_set_ignored functions are added to manage the flag. Adding the flag and functions gives a library interface to ignore metadata areas independent of the underlying location (disk, file, etc). The location specific read/write functions must then handle the specifics of what this flag means to the location. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Reviewed-by: Alasdair G Kergon <agk@redhat.com>
2010-06-29 00:30:14 +04:00
}
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
void mda_set_ignored(struct metadata_area *mda, unsigned mda_ignored)
Add location independent flag and functions to ignore mdas. First we add a 'flags' field to the location independent metadata_area structure, and a MDA_IGNORE flag. The mda_is_ignored and mda_set_ignored functions are added to manage the flag. Adding the flag and functions gives a library interface to ignore metadata areas independent of the underlying location (disk, file, etc). The location specific read/write functions must then handle the specifics of what this flag means to the location. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Reviewed-by: Alasdair G Kergon <agk@redhat.com>
2010-06-29 00:30:14 +04:00
{
Improve logging for metadata ignore by printing device name. Print device name when setting or clearing metadata ignore bit. Example: label/label.c:160 /dev/loop2: lvm2 label detected cache/lvmcache.c:1136 lvmcache: /dev/loop2: now in VG #orphans_lvm2 (#orphans_lvm2) metadata/metadata.c:4142 Setting mda ignored flag for metadata_locn /dev/loop2. format_text/text_label.c:318 Skipping mda with ignored flag on device /dev/loop2 at offset 4096
2010-06-30 02:37:32 +04:00
void *locn = mda->metadata_locn;
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
unsigned old_mda_ignored = mda_is_ignored(mda);
Improve logging for metadata ignore by printing device name. Print device name when setting or clearing metadata ignore bit. Example: label/label.c:160 /dev/loop2: lvm2 label detected cache/lvmcache.c:1136 lvmcache: /dev/loop2: now in VG #orphans_lvm2 (#orphans_lvm2) metadata/metadata.c:4142 Setting mda ignored flag for metadata_locn /dev/loop2. format_text/text_label.c:318 Skipping mda with ignored flag on device /dev/loop2 at offset 4096
2010-06-30 02:37:32 +04:00
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
if (mda_ignored && !old_mda_ignored)
Rename 'flags' to 'status' for struct metadata_area. In other LVM memory structures such as volume_group, the field used to store flags is called "status", and on-disk fields are called 'flags', so rename the one inside metadata_area to be consistent. Not only is it more consistent with existing code but is cleaner to say "the status of this mda is ignored". Background for this patch - prajnoha pinged me on IRC this morning about a fix he was working on related to metadataignore when metadata/dirs was set. I was reviewing my patches from this year and realized the 'flags' field was probably not the best choice when I originally did the metadataignore patches.
2010-10-05 21:34:05 +04:00
mda->status |= MDA_IGNORED;
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
else if (!mda_ignored && old_mda_ignored)
Rename 'flags' to 'status' for struct metadata_area. In other LVM memory structures such as volume_group, the field used to store flags is called "status", and on-disk fields are called 'flags', so rename the one inside metadata_area to be consistent. Not only is it more consistent with existing code but is cleaner to say "the status of this mda is ignored". Background for this patch - prajnoha pinged me on IRC this morning about a fix he was working on related to metadataignore when metadata/dirs was set. I was reviewing my patches from this year and realized the 'flags' field was probably not the best choice when I originally did the metadataignore patches.
2010-10-05 21:34:05 +04:00
mda->status &= ~MDA_IGNORED;
Improve logging for metadata ignore by printing device name. Print device name when setting or clearing metadata ignore bit. Example: label/label.c:160 /dev/loop2: lvm2 label detected cache/lvmcache.c:1136 lvmcache: /dev/loop2: now in VG #orphans_lvm2 (#orphans_lvm2) metadata/metadata.c:4142 Setting mda ignored flag for metadata_locn /dev/loop2. format_text/text_label.c:318 Skipping mda with ignored flag on device /dev/loop2 at offset 4096
2010-06-30 02:37:32 +04:00
else
Improve various log messages.
2010-06-30 17:51:11 +04:00
return; /* No change */
lvm2app: Add VG/LV name validation C library portion for https://bugzilla.redhat.com/show_bug.cgi?id=883689 Signed-off-by: Tony Asleson <tasleson@redhat.com>
2013-09-26 20:37:40 +04:00
log_debug_metadata("%s ignored flag for mda %s at offset %" PRIu64 ".",
logging: classify log_debug messages Place most log_debug() messages into a class.
2013-01-08 02:30:29 +04:00
mda_ignored ? "Setting" : "Clearing",
mda->ops->mda_metadata_locn_name ? mda->ops->mda_metadata_locn_name(locn) : "",
mda->ops->mda_metadata_locn_offset ? mda->ops->mda_metadata_locn_offset(locn) : UINT64_C(0));
Add location independent flag and functions to ignore mdas. First we add a 'flags' field to the location independent metadata_area structure, and a MDA_IGNORE flag. The mda_is_ignored and mda_set_ignored functions are added to manage the flag. Adding the flag and functions gives a library interface to ignore metadata areas independent of the underlying location (disk, file, etc). The location specific read/write functions must then handle the specifics of what this flag means to the location. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Reviewed-by: Alasdair G Kergon <agk@redhat.com>
2010-06-29 00:30:14 +04:00
}
Add mdas_empty_or_ignored() helper function. Add a helper function to consolidate checking for an empty mdas list or ignored mdas. Ignored mdas should behave almost identically to an empty mda list - the metadata areas should not be read or written to. This function will make it easier to implement metadata balancing and easier to track pvs with an empty mda list or ignored mdas. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:34:40 +04:00
int mdas_empty_or_ignored(struct dm_list *mdas)
{
struct metadata_area *mda;
cleanup: use dm_list_empty Check for empty list directly.
2013-07-17 16:49:21 +04:00
if (dm_list_empty(mdas))
Add mdas_empty_or_ignored() helper function. Add a helper function to consolidate checking for an empty mdas list or ignored mdas. Ignored mdas should behave almost identically to an empty mda list - the metadata areas should not be read or written to. This function will make it easier to implement metadata balancing and easier to track pvs with an empty mda list or ignored mdas. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-29 00:34:40 +04:00
return 1;
dm_list_iterate_items(mda, mdas) {
if (mda_is_ignored(mda))
return 1;
}
return 0;
}
improve vgmetadatacopies unmanaged message
2010-07-01 00:03:52 +04:00
int pv_change_metadataignore(struct physical_volume *pv, uint32_t mda_ignored)
Move code into pv_change_metadataignore library function. In preparation to call this from both pvcreate as well as pvchange, move the guts of metadataignore into a library function. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-30 01:32:44 +04:00
{
const char *pv_name = pv_dev_name(pv);
improve vgmetadatacopies unmanaged message
2010-07-01 00:03:52 +04:00
if (mda_ignored && !pv_mda_used_count(pv)) {
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
log_error("Metadata areas on physical volume \"%s\" already "
"ignored.", pv_name);
Move code into pv_change_metadataignore library function. In preparation to call this from both pvcreate as well as pvchange, move the guts of metadataignore into a library function. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-30 01:32:44 +04:00
return 0;
}
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
improve vgmetadatacopies unmanaged message
2010-07-01 00:03:52 +04:00
if (!mda_ignored && (pv_mda_used_count(pv) == pv_mda_count(pv))) {
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
log_error("Metadata areas on physical volume \"%s\" already "
"marked as in-use.", pv_name);
Move code into pv_change_metadataignore library function. In preparation to call this from both pvcreate as well as pvchange, move the guts of metadataignore into a library function. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-30 01:32:44 +04:00
return 0;
}
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
Move code into pv_change_metadataignore library function. In preparation to call this from both pvcreate as well as pvchange, move the guts of metadataignore into a library function. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-30 01:32:44 +04:00
if (!pv_mda_count(pv)) {
log_error("Physical volume \"%s\" has no metadata "
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
"areas.", pv_name);
Move code into pv_change_metadataignore library function. In preparation to call this from both pvcreate as well as pvchange, move the guts of metadataignore into a library function. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-30 01:32:44 +04:00
return 0;
}
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
log_verbose("Marking metadata areas on physical volume \"%s\" "
improve vgmetadatacopies unmanaged message
2010-07-01 00:03:52 +04:00
"as %s.", pv_name, mda_ignored ? "ignored" : "in-use");
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
improve vgmetadatacopies unmanaged message
2010-07-01 00:03:52 +04:00
if (!pv_mda_set_ignored(pv, mda_ignored))
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
return_0;
Move code into pv_change_metadataignore library function. In preparation to call this from both pvcreate as well as pvchange, move the guts of metadataignore into a library function. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-30 01:32:44 +04:00
/*
* Update vg_mda_copies based on the mdas in this PV.
* This is most likely what the user would expect - if they
* specify a specific PV to be ignored/un-ignored, they will
* most likely not want LVM to turn around and change the
* ignore / un-ignore value when it writes the VG to disk.
* This does not guarantee this PV's ignore bits will be
* preserved in future operations.
*/
Add warning to vgextend and pvchange if metadataignore given on cmdline. Warn the user then change the value of vg_mda_copies. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-07-07 22:59:45 +04:00
if (!is_orphan(pv) &&
vg_mda_copies(pv->vg) != VGMETADATACOPIES_UNMANAGED) {
log_warn("WARNING: Changing preferred number of copies of VG %s "
"metadata from %"PRIu32" to %"PRIu32, pv_vg_name(pv),
vg_mda_copies(pv->vg), vg_mda_used_count(pv->vg));
Move code into pv_change_metadataignore library function. In preparation to call this from both pvcreate as well as pvchange, move the guts of metadataignore into a library function. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-30 01:32:44 +04:00
vg_set_mda_copies(pv->vg, vg_mda_used_count(pv->vg));
Add warning to vgextend and pvchange if metadataignore given on cmdline. Warn the user then change the value of vg_mda_copies. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-07-07 22:59:45 +04:00
}
more metadataignore message/code cleanup
2010-06-30 21:13:05 +04:00
Move code into pv_change_metadataignore library function. In preparation to call this from both pvcreate as well as pvchange, move the guts of metadataignore into a library function. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
2010-06-30 01:32:44 +04:00
return 1;
}
compilation: Rename tags variables to tagsl.
2014-01-31 01:09:28 +04:00
char *tags_format_and_copy(struct dm_pool *mem, const struct dm_list *tagsl)
Add tags_format_and_copy() common function and call from _tags_disp. Add a common function to allocate memory and format a string of tags. Call tags_format_and_copy() from _tags_disp().
2010-09-30 18:08:07 +04:00
{
report: select: refactor: move str_list to libdm The list of strings is used quite frequently and we'd like to reuse this simple structure for report selection support too. Make it part of libdevmapper for general reuse throughout the code. This also simplifies the LVM code a bit since we don't need to include and manage lvm-types.h anymore (the string list was the only structure defined there).
2014-05-29 11:41:03 +04:00
struct dm_str_list *sl;
Add tags_format_and_copy() common function and call from _tags_disp. Add a common function to allocate memory and format a string of tags. Call tags_format_and_copy() from _tags_disp().
2010-09-30 18:08:07 +04:00
if (!dm_pool_begin_object(mem, 256)) {
log_error("dm_pool_begin_object failed");
return NULL;
}
compilation: Rename tags variables to tagsl.
2014-01-31 01:09:28 +04:00
dm_list_iterate_items(sl, tagsl) {
Add tags_format_and_copy() common function and call from _tags_disp. Add a common function to allocate memory and format a string of tags. Call tags_format_and_copy() from _tags_disp().
2010-09-30 18:08:07 +04:00
if (!dm_pool_grow_object(mem, sl->str, strlen(sl->str)) ||
compilation: Rename tags variables to tagsl.
2014-01-31 01:09:28 +04:00
(sl->list.n != tagsl && !dm_pool_grow_object(mem, ",", 1))) {
Add tags_format_and_copy() common function and call from _tags_disp. Add a common function to allocate memory and format a string of tags. Call tags_format_and_copy() from _tags_disp().
2010-09-30 18:08:07 +04:00
log_error("dm_pool_grow_object failed");
return NULL;
}
}
if (!dm_pool_grow_object(mem, "\0", 1)) {
log_error("dm_pool_grow_object failed");
return NULL;
}
return dm_pool_end_object(mem);
}
Add and track an "ondisk" pointer to struct volume_group. This allows us to get the current on-disk version of the metadata whenever we have the current in-flight version, without a recourse to scanning or lvmcache.
2013-03-17 19:27:44 +04:00
cleanup: rename lv_ondisk to lv_committed Patch has no functional change.
2015-11-25 13:10:32 +03:00
const struct logical_volume *lv_committed(const struct logical_volume *lv)
Add and track an "ondisk" pointer to struct volume_group. This allows us to get the current on-disk version of the metadata whenever we have the current in-flight version, without a recourse to scanning or lvmcache.
2013-03-17 19:27:44 +04:00
{
struct volume_group *vg;
cleanup: move towards using direct LV pointers We do not won't to 'expose' internals of VG struct. ATM we use lists to keep all LVs - we may want to switch to better struct for quicker 'search'. Since we do not need 'lists' but always actual LV, switch find_lv_in_vg_by_lvid() to return LV, and replaces some use case of find_lv_in_vg() with 'better' working find_lv() which already returns LV.
2015-11-22 01:31:44 +03:00
struct logical_volume *found_lv;
Add and track an "ondisk" pointer to struct volume_group. This allows us to get the current on-disk version of the metadata whenever we have the current in-flight version, without a recourse to scanning or lvmcache.
2013-03-17 19:27:44 +04:00
if (!lv)
return NULL;
cleanup: rename vg_ondisk to vg_committed Unifying terminology. Since all the metadata in-use are ALWAYS on disk - switch to terminology committed and precommitted. Patch has no functional change inside.
2015-11-25 01:29:18 +03:00
if (!lv->vg->vg_committed)
lv_ondisk: option search for lv without vg_ondisk When vg_ondisk is NULL we do not need to search through the whole VG to find out the same LV. NOTE: as of now - VG locking is not enabled as some code parts are breaking memory locking rules (lvm2app). Once we enforce VG locking for read-only commands the effect will be much better for larger VGs.
2014-09-22 01:10:04 +04:00
return lv;
Add and track an "ondisk" pointer to struct volume_group. This allows us to get the current on-disk version of the metadata whenever we have the current in-flight version, without a recourse to scanning or lvmcache.
2013-03-17 19:27:44 +04:00
cleanup: rename vg_ondisk to vg_committed Unifying terminology. Since all the metadata in-use are ALWAYS on disk - switch to terminology committed and precommitted. Patch has no functional change inside.
2015-11-25 01:29:18 +03:00
vg = lv->vg->vg_committed;
Add and track an "ondisk" pointer to struct volume_group. This allows us to get the current on-disk version of the metadata whenever we have the current in-flight version, without a recourse to scanning or lvmcache.
2013-03-17 19:27:44 +04:00
cleanup: move towards using direct LV pointers We do not won't to 'expose' internals of VG struct. ATM we use lists to keep all LVs - we may want to switch to better struct for quicker 'search'. Since we do not need 'lists' but always actual LV, switch find_lv_in_vg_by_lvid() to return LV, and replaces some use case of find_lv_in_vg() with 'better' working find_lv() which already returns LV.
2015-11-22 01:31:44 +03:00
if (!(found_lv = find_lv_in_vg_by_lvid(vg, &lv->lvid))) {
cleanup: rename lv_ondisk to lv_committed Patch has no functional change.
2015-11-25 13:10:32 +03:00
log_error(INTERNAL_ERROR "LV %s (UUID %s) not found in committed metadata.",
cleanup: use find_lv_in_vg_by_lvid Replace another loop with existion function. TODO: consider using hash maps for lvname & lvid lookups.
2014-09-21 13:46:34 +04:00
display_lvname(lv), lv->lvid.s);
return NULL;
}
Add and track an "ondisk" pointer to struct volume_group. This allows us to get the current on-disk version of the metadata whenever we have the current in-flight version, without a recourse to scanning or lvmcache.
2013-03-17 19:27:44 +04:00
cleanup: move towards using direct LV pointers We do not won't to 'expose' internals of VG struct. ATM we use lists to keep all LVs - we may want to switch to better struct for quicker 'search'. Since we do not need 'lists' but always actual LV, switch find_lv_in_vg_by_lvid() to return LV, and replaces some use case of find_lv_in_vg() with 'better' working find_lv() which already returns LV.
2015-11-22 01:31:44 +03:00
return found_lv;
Add and track an "ondisk" pointer to struct volume_group. This allows us to get the current on-disk version of the metadata whenever we have the current in-flight version, without a recourse to scanning or lvmcache.
2013-03-17 19:27:44 +04:00
}
Add lvmlockd
2015-03-05 23:00:44 +03:00
/*
* Check if a lock_type uses lvmlockd.
* If not (none, clvm), return 0.
* If so (dlm, sanlock), return 1.
*/
int is_lockd_type(const char *lock_type)
{
if (!lock_type)
return 0;
if (!strcmp(lock_type, "dlm"))
return 1;
if (!strcmp(lock_type, "sanlock"))
return 1;
return 0;
}
vgs: add report field for shared equivalent to a non-empty -o locktype.
2018-05-31 18:23:03 +03:00
int vg_is_shared(const struct volume_group *vg)
{
return (vg->lock_type && is_lockd_type(vg->lock_type));
}
metadata: add vg_strip_outdated_historical_lvs fn and call it during VG read The vg_strip_outdated_historical_lvs iterates over the list of historical LVs we have and it shoots down the ones which are outdated. Configuration hook to set the timeout will be in subsequent patch.
2016-03-01 17:27:21 +03:00
int vg_strip_outdated_historical_lvs(struct volume_group *vg) {
struct glv_list *glvl, *tglvl;
time_t current_time = time(NULL);
conf: add metadata/lvs_history_timeout configuration setting
2016-03-01 17:29:27 +03:00
uint64_t threshold = find_config_tree_int(vg->cmd, metadata_lvs_history_retention_time_CFG, NULL);
metadata: add vg_strip_outdated_historical_lvs fn and call it during VG read The vg_strip_outdated_historical_lvs iterates over the list of historical LVs we have and it shoots down the ones which are outdated. Configuration hook to set the timeout will be in subsequent patch.
2016-03-01 17:27:21 +03:00
if (!threshold)
return 1;
dm_list_iterate_items_safe(glvl, tglvl, &vg->historical_lvs) {
/*
* Removal time in the future? Not likely,
* but skip this item in any case.
*/
cleanup: use matching signed number comparation
2017-02-12 20:18:54 +03:00
if (current_time < (time_t) glvl->glv->historical->timestamp_removed)
metadata: add vg_strip_outdated_historical_lvs fn and call it during VG read The vg_strip_outdated_historical_lvs iterates over the list of historical LVs we have and it shoots down the ones which are outdated. Configuration hook to set the timeout will be in subsequent patch.
2016-03-01 17:27:21 +03:00
continue;
if ((current_time - glvl->glv->historical->timestamp_removed) > threshold) {
if (!historical_glv_remove(glvl->glv)) {
log_error("Failed to destroy record about historical LV %s/%s.",
vg->name, glvl->glv->historical->name);
return 0;
}
log_verbose("Outdated record for historical logical volume \"%s\" "
"automatically destroyed.", glvl->glv->historical->name);
}
}
return 1;
}
Add dm-writecache support dm-writecache is used like dm-cache with a standard LV as the cache. $ lvcreate -n main -L 128M -an foo /dev/loop0 $ lvcreate -n fast -L 32M -an foo /dev/pmem0 $ lvconvert --type writecache --cachepool fast foo/main $ lvs -a foo -o+devices LV VG Attr LSize Origin Devices [fast] foo -wi------- 32.00m /dev/pmem0(0) main foo Cwi------- 128.00m [main_wcorig] main_wcorig(0) [main_wcorig] foo -wi------- 128.00m /dev/loop0(0) $ lvchange -ay foo/main $ dmsetup table foo-main_wcorig: 0 262144 linear 7:0 2048 foo-main: 0 262144 writecache p 253:4 253:3 4096 0 foo-fast: 0 65536 linear 259:0 2048 $ lvchange -an foo/main $ lvconvert --splitcache foo/main $ lvs -a foo -o+devices LV VG Attr LSize Devices fast foo -wi------- 32.00m /dev/pmem0(0) main foo -wi------- 128.00m /dev/loop0(0)
2018-08-27 22:53:09 +03:00
int lv_on_pmem(struct logical_volume *lv)
{
struct lv_segment *seg;
struct physical_volume *pv;
uint32_t s;
int pmem_devs = 0, other_devs = 0;
dm_list_iterate_items(seg, &lv->segments) {
for (s = 0; s < seg->area_count; s++) {
pv = seg_pv(seg, s);
if (dev_is_pmem(pv->dev)) {
log_debug("LV %s dev %s is pmem.", lv->name, dev_name(pv->dev));
pmem_devs++;
} else {
log_debug("LV %s dev %s not pmem.", lv->name, dev_name(pv->dev));
other_devs++;
}
}
}
if (pmem_devs && other_devs) {
log_error("Invalid mix of cache device types in %s.", display_lvname(lv));
return -1;
}
if (pmem_devs) {
log_debug("LV %s on pmem", lv->name);
return 1;
}
return 0;
}
pvscan: ignore online for shared and foreign PVs Activation would not be allowed anyway, but we can check for these cases early and avoid wasted time in pvscan managing online files an attempting activation.
2019-03-06 00:19:05 +03:00
int vg_is_foreign(struct volume_group *vg)
{
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
return vg->cmd->system_id && strcmp(vg->system_id, vg->cmd->system_id);
pvscan: ignore online for shared and foreign PVs Activation would not be allowed anyway, but we can check for these cases early and avoid wasted time in pvscan managing online files an attempting activation.
2019-03-06 00:19:05 +03:00
}
vgck --updatemetadata is a new command uses vg_write to correct more common or less severe issues, and also adds the ability to repair some metadata corruption that couldn't be handled previously.
2019-02-06 22:39:41 +03:00
void vg_write_commit_bad_mdas(struct cmd_context *cmd, struct volume_group *vg)
{
struct dm_list bad_mda_list;
struct mda_list *mdal;
struct metadata_area *mda;
struct device *dev;
dm_list_init(&bad_mda_list);
lvmcache_get_bad_mdas(cmd, vg->name, (const char *)&vg->id, &bad_mda_list);
dm_list_iterate_items(mdal, &bad_mda_list) {
mda = mdal->mda;
dev = mda_get_device(mda);
/*
* bad_fields:
*
* 0: shouldn't happen
*
* READ|INTERNAL: there's probably nothing wrong on disk
*
* MAGIC|START: there's a good chance that we were
* reading the mda_header from the wrong location; maybe
* the pv_header location was wrong. We don't want to
* write new metadata to the wrong location. To handle
* this we would want to do some further verification that
* we have the mda location correct.
*
* VERSION|CHECKSUM: when the others are correct these
* look safe to repair.
*
* HEADER: general error related to header, covered by fields
* above.
*
* TEXT: general error related to text metadata, we can repair.
vgck: let updatemetadata repair mismatched metadata Let vgck --updatemetadata repair cases where different mdas hold indepedently valid but unmatching copies of the metadata, i.e. different text metadata checksums or text metadata sizes.
2019-10-08 22:44:24 +03:00
*
* MISMATCH: different values between instances of metadata,
* can repair.
vgck --updatemetadata is a new command uses vg_write to correct more common or less severe issues, and also adds the ability to repair some metadata corruption that couldn't be handled previously.
2019-02-06 22:39:41 +03:00
*/
if (!mda->bad_fields ||
(mda->bad_fields & BAD_MDA_READ) ||
(mda->bad_fields & BAD_MDA_INTERNAL) ||
(mda->bad_fields & BAD_MDA_MAGIC) ||
(mda->bad_fields & BAD_MDA_START)) {
log_warn("WARNING: not repairing bad metadata (0x%x) for mda%d on %s",
mda->bad_fields, mda->mda_num, dev_name(dev));
continue;
}
/*
* vg_write/vg_commit reread the mda_header which checks the
* mda header fields and fails if any are bad, which stops
* vg_write/vg_commit from continuing. Suppress these header
* field checks when we know the field is bad and we are going
* to replace it. FIXME: do vg_write/vg_commit really need to
* reread and recheck the mda_header again (probably not)?
*/
if (mda->bad_fields & BAD_MDA_CHECKSUM)
mda->ignore_bad_fields |= BAD_MDA_CHECKSUM;
if (mda->bad_fields & BAD_MDA_VERSION)
mda->ignore_bad_fields |= BAD_MDA_VERSION;
log_warn("WARNING: repairing bad metadata (0x%x) in mda%d at %llu on %s.",
mda->bad_fields, mda->mda_num, (unsigned long long)mda->header_start, dev_name(dev));
if (!mda->ops->vg_write(vg->fid, vg, mda)) {
log_warn("WARNING: failed to write VG %s metadata to bad mda%d at %llu on %s.",
vg->name, mda->mda_num, (unsigned long long)mda->header_start, dev_name(dev));
continue;
}
if (!mda->ops->vg_precommit(vg->fid, vg, mda)) {
log_warn("WARNING: failed to precommit VG %s metadata to bad mda%d at %llu on %s.",
vg->name, mda->mda_num, (unsigned long long)mda->header_start, dev_name(dev));
continue;
}
if (!mda->ops->vg_commit(vg->fid, vg, mda)) {
log_warn("WARNING: failed to commit VG %s metadata to bad mda%d at %llu on %s.",
vg->name, mda->mda_num, (unsigned long long)mda->header_start, dev_name(dev));
continue;
}
}
}
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
scanning: optimize by checking text offset and checksum After the VG lock is taken for vg_read, reread the mda_header and compare the metadata text offset and checksum to what was seen during label scan. If it is unchanged, then the metadata has not changed since the label scan, and the metadata does not need to be reread under the lock for command processing. For commands that do not make changes (e.g. reporting), the mda_header is reread and checked on one mda to decide if the full metadata rereading can be skipped. For other commands (e.g. modifying the vg) the mda_header is reread and checked from all PVs. (These could probably just check one mda also.)
2019-11-26 20:56:51 +03:00
/*
* Reread an mda_header. If the text offset is the same as was seen and saved
* by label scan, it means the metadata is unchanged and we do not need to
* reread metadata.
*/
scanning: use bool type for _scan_text_mismatch
2019-11-27 18:26:04 +03:00
static bool _scan_text_mismatch(struct cmd_context *cmd, const char *vgname, const char *vgid)
scanning: optimize by checking text offset and checksum After the VG lock is taken for vg_read, reread the mda_header and compare the metadata text offset and checksum to what was seen during label scan. If it is unchanged, then the metadata has not changed since the label scan, and the metadata does not need to be reread under the lock for command processing. For commands that do not make changes (e.g. reporting), the mda_header is reread and checked on one mda to decide if the full metadata rereading can be skipped. For other commands (e.g. modifying the vg) the mda_header is reread and checked from all PVs. (These could probably just check one mda also.)
2019-11-26 20:56:51 +03:00
{
struct dm_list mda_list;
struct mda_list *mdal, *safe;
struct metadata_area *mda;
struct mda_context *mdac;
struct device_area *area;
struct mda_header *mdah;
struct raw_locn *rlocn;
struct device *dev;
uint32_t bad_fields;
scanning: use bool type for _scan_text_mismatch
2019-11-27 18:26:04 +03:00
bool ret = true;
scanning: optimize by checking text offset and checksum After the VG lock is taken for vg_read, reread the mda_header and compare the metadata text offset and checksum to what was seen during label scan. If it is unchanged, then the metadata has not changed since the label scan, and the metadata does not need to be reread under the lock for command processing. For commands that do not make changes (e.g. reporting), the mda_header is reread and checked on one mda to decide if the full metadata rereading can be skipped. For other commands (e.g. modifying the vg) the mda_header is reread and checked from all PVs. (These could probably just check one mda also.)
2019-11-26 20:56:51 +03:00
/*
* if cmd->can_use_one_scan, check one mda_header is unchanged,
* else check that all mda_headers are unchanged.
*/
dm_list_init(&mda_list);
lvmcache_get_mdas(cmd, vgname, vgid, &mda_list);
dm_list_iterate_items(mdal, &mda_list) {
mda = mdal->mda;
if (!mda->scan_text_offset)
continue;
if (mda->mda_num != 1)
continue;
if (!(dev = mda_get_device(mda))) {
log_debug("rescan for text mismatch - no mda dev");
goto out;
}
bad_fields = 0;
mdac = mda->metadata_locn;
area = &mdac->area;
/*
* Invalidate mda_header in bcache so it will be reread from disk.
*/
if (!dev_invalidate_bytes(dev, 4096, 512)) {
log_debug("rescan for text mismatch - cannot invalidate");
goto out;
}
if (!(mdah = raw_read_mda_header(cmd->fmt, area, 1, 0, &bad_fields))) {
log_debug("rescan for text mismatch - no mda header");
goto out;
}
rlocn = mdah->raw_locns;
if (bad_fields) {
log_debug("rescan for text mismatch - bad_fields");
} else if (rlocn->checksum != mda->scan_text_checksum) {
log_debug("rescan for text checksum mismatch - now %x prev %x",
rlocn->checksum, mda->scan_text_checksum);
} else if (rlocn->offset != mda->scan_text_offset) {
log_debug("rescan for text offset mismatch - now %llu prev %llu",
(unsigned long long)rlocn->offset,
(unsigned long long)mda->scan_text_offset);
} else {
scanning: use bool type for _scan_text_mismatch
2019-11-27 18:26:04 +03:00
/* the common case where fields match and no rescan needed */
ret = false;
scanning: optimize by checking text offset and checksum After the VG lock is taken for vg_read, reread the mda_header and compare the metadata text offset and checksum to what was seen during label scan. If it is unchanged, then the metadata has not changed since the label scan, and the metadata does not need to be reread under the lock for command processing. For commands that do not make changes (e.g. reporting), the mda_header is reread and checked on one mda to decide if the full metadata rereading can be skipped. For other commands (e.g. modifying the vg) the mda_header is reread and checked from all PVs. (These could probably just check one mda also.)
2019-11-26 20:56:51 +03:00
}
dm_pool_free(cmd->mem, mdah);
/* For can_use_one_scan commands, return result from checking one mda. */
if (cmd->can_use_one_scan)
goto out;
/* For other commands, return mismatch immediately. */
if (ret)
goto_out;
}
if (ret) {
/* shouldn't happen */
log_debug("rescan for text mismatch - no mdas");
goto out;
}
out:
if (!ret)
scanning: use bool type for _scan_text_mismatch
2019-11-27 18:26:04 +03:00
log_debug("rescan skipped - unchanged offset %llu checksum %x",
(unsigned long long)mda->scan_text_offset,
mda->scan_text_checksum);
scanning: optimize by checking text offset and checksum After the VG lock is taken for vg_read, reread the mda_header and compare the metadata text offset and checksum to what was seen during label scan. If it is unchanged, then the metadata has not changed since the label scan, and the metadata does not need to be reread under the lock for command processing. For commands that do not make changes (e.g. reporting), the mda_header is reread and checked on one mda to decide if the full metadata rereading can be skipped. For other commands (e.g. modifying the vg) the mda_header is reread and checked from all PVs. (These could probably just check one mda also.)
2019-11-26 20:56:51 +03:00
dm_list_iterate_items_safe(mdal, safe, &mda_list) {
dm_list_del(&mdal->list);
free(mdal);
}
return ret;
}
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
static struct volume_group *_vg_read(struct cmd_context *cmd,
const char *vgname,
const char *vgid,
scanning: open devs rw when rescanning for write When vg_read rescans devices with the intention of writing the VG, the label rescan can open the devs RW so they do not need to be closed and reopened RW in dev_write_bytes.
2019-06-12 00:17:24 +03:00
unsigned precommitted,
int writing)
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
{
const struct format_type *fmt = cmd->fmt;
struct format_instance *fid = NULL;
struct format_instance_ctx fic;
struct volume_group *vg, *vg_ret = NULL;
struct metadata_area *mda, *mda2;
unsigned use_precommitted = precommitted;
gcc: hide warn about possible uninitialized use of dev_ret Older gcc reports this fp problem.
2020-09-01 21:46:18 +03:00
struct device *mda_dev, *dev_ret = NULL, *dev;
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
struct cached_vg_fmtdata *vg_fmtdata = NULL; /* Additional format-specific data about the vg */
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
struct pv_list *pvl;
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
int found_old_metadata = 0;
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
int found_md_component = 0;
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
unsigned use_previous_vg;
log_debug_metadata("Reading VG %s %s", vgname ?: "<no name>", vgid ?: "<no vgid>");
metadata: open rw fd before closing ro fd lvm opens devices readonly to scan them, but needs to open then readwrite to update the metadata. Previously, the ro fd was closed before the rw fd was opened, leaving a small gap where the dev was not held open, and during which the dev could possibly change which storage it referred to. With the bcache_change_fd() interface, lvm opens a rw fd on a device to be written, tells bcache to change to the new rw fd, and closes the ro fd. . open dev ro . read dev with the ro fd (label_scan) . lock vg (ex for writing) . open dev rw . close ro fd . rescan dev to check if the metadata changed between the scan and the lock . if the metadata did change, reread in full . write the metadata
2020-09-18 22:42:23 +03:00
/*
* Devices are generally open readonly from scanning, and we need to
* reopen them rw to update metadata. We want to reopen them rw before
* before rescanning and/or writing. Reopening rw preserves the existing
* bcache blocks for the devs.
*/
if (writing)
lvmcache_label_reopen_vg_rw(cmd, vgname, vgid);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
/*
* Rescan the devices that are associated with this vg in lvmcache.
* This repeats what was done by the command's initial label scan,
* but only the devices associated with this VG.
*
* The lvmcache info about these devs is from the initial label scan
* performed by the command before the vg lock was held. Now the VG
* lock is held, so we rescan all the info from the devs in case
* something changed between the initial scan and now that the lock
* is held.
*
* Some commands (e.g. reporting) are fine reporting data read by
* the label scan. It doesn't matter if the devs changed between
* the label scan and here, we can report what was seen in the
* scan, even though it is the old state, since we will not be
* making any modifications. If the VG was being modified during
* the scan, and caused us to see inconsistent metadata on the
* different PVs in the VG, then we do want to rescan the devs
* here to get a consistent view of the VG. Note that we don't
* know if the scan found all the PVs in the VG at this point.
* We don't know that until vg_read looks at the list of PVs in
* the metadata and compares that to the devices found by the scan.
*
* It's possible that a change made to the VG during scan was
* adding or removing a PV from the VG. In this case, the list
* of devices associated with the VG in lvmcache would change
* due to the rescan.
*
* The devs in the VG may be persistently inconsistent due to some
* previous problem. In this case, rescanning the labels here will
* find the same inconsistency. The VG repair (mistakenly done by
* vg_read below) is supposed to fix that.
*
scanning: optimize by checking text offset and checksum After the VG lock is taken for vg_read, reread the mda_header and compare the metadata text offset and checksum to what was seen during label scan. If it is unchanged, then the metadata has not changed since the label scan, and the metadata does not need to be reread under the lock for command processing. For commands that do not make changes (e.g. reporting), the mda_header is reread and checked on one mda to decide if the full metadata rereading can be skipped. For other commands (e.g. modifying the vg) the mda_header is reread and checked from all PVs. (These could probably just check one mda also.)
2019-11-26 20:56:51 +03:00
* If the VG was not modified between the time we scanned the PVs
* and now, when we hold the lock, then we don't need to rescan.
* We can read the mda_header, and look at the text offset/checksum,
* and if the current text offset/checksum matches what was seen during
* label scan, we know that metadata is unchanged and doesn't need
* to be rescanned. For reporting/display commands (CAN_USE_ONE_SCAN/
* can_use_one_scan), we check that the text offset/checksum are unchanged
* in just one mda before deciding to skip rescanning. For other commands,
* we check that they are unchanged in all mdas. This added checking is
* probably unnecessary; all commands could likely just check a single mda.
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
*/
bcache: use indirection table for fd Add a "device index" (di) for each device, and use this in the bcache api to the rest of lvm. This replaces the file descriptor (fd) in the api. The rest of lvm uses new functions bcache_set_fd(), bcache_clear_fd(), and bcache_change_fd() to control which fd bcache uses for io to a particular device. . lvm opens a dev and gets and fd. fd = open(dev); . lvm passes fd to the bcache layer and gets a di to use in the bcache api for the dev. di = bcache_set_fd(fd); . lvm uses bcache functions, passing di for the dev. bcache_write_bytes(di, ...), etc. . bcache translates di to fd to do io. . lvm closes the device and clears the di/fd bcache state. close(fd); bcache_clear_fd(di); In the bcache layer, a di-to-fd translation table (int *_fd_table) is added. When bcache needs to perform io on a di, it uses _fd_table[di]. In the following commit, lvm will make use of the new bcache_change_fd() function to change the fd that bcache uses for the dev, without dropping cached blocks.
2020-09-17 17:40:18 +03:00
scanning: optimize by checking text offset and checksum After the VG lock is taken for vg_read, reread the mda_header and compare the metadata text offset and checksum to what was seen during label scan. If it is unchanged, then the metadata has not changed since the label scan, and the metadata does not need to be reread under the lock for command processing. For commands that do not make changes (e.g. reporting), the mda_header is reread and checked on one mda to decide if the full metadata rereading can be skipped. For other commands (e.g. modifying the vg) the mda_header is reread and checked from all PVs. (These could probably just check one mda also.)
2019-11-26 20:56:51 +03:00
if (lvmcache_scan_mismatch(cmd, vgname, vgid) || _scan_text_mismatch(cmd, vgname, vgid)) {
scanning: open devs rw when rescanning for write When vg_read rescans devices with the intention of writing the VG, the label rescan can open the devs RW so they do not need to be closed and reopened RW in dev_write_bytes.
2019-06-12 00:17:24 +03:00
log_debug_metadata("Rescanning devices for %s %s", vgname, writing ? "rw" : "");
if (writing)
lvmcache_label_rescan_vg_rw(cmd, vgname, vgid);
else
lvmcache_label_rescan_vg(cmd, vgname, vgid);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
}
/*
* A "format instance" is an abstraction for a VG location,
* i.e. where a VG's metadata exists on disk.
*
* An fic (format_instance_ctx) is a temporary struct used
* to create an fid (format_instance). The fid hangs around
* and is used to create a 'vg' to which it connected (vg->fid).
*
* The 'fic' describes a VG in terms of fmt/name/id.
*
* The 'fid' describes a VG in more detail than the fic,
* holding information about where to find the VG metadata.
*
* The 'vg' describes the VG in the most detail representing
* all the VG metadata.
*
* The fic and fid are set up by create_instance() to describe
* the VG location. This happens before the VG metadata is
* assembled into the more familiar struct volume_group "vg".
*
* The fid has one main purpose: to keep track of the metadata
* locations for a given VG. It does this by putting 'mda'
* structs on fid->metadata_areas_in_use, which specify where
* metadata is located on disk. It gets this information
* (metadata locations for a specific VG) from the command's
* initial label scan. The info is passed indirectly via
* lvmcache info/vginfo structs, which are created by the
* label scan and then copied into fid by create_instance().
*
* FIXME: just use the vginfo/info->mdas lists directly instead
* of copying them into the fid list.
*/
fic.type = FMT_INSTANCE_MDAS | FMT_INSTANCE_AUX_MDAS;
fic.context.vg_ref.vg_name = vgname;
fic.context.vg_ref.vg_id = vgid;
/*
* Sets up the metadata areas that we need to read below.
* For each info in vginfo->infos, for each mda in info->mdas,
* (found during label_scan), copy the mda to fid->metadata_areas_in_use
*/
if (!(fid = fmt->ops->create_instance(fmt, &fic))) {
log_error("Failed to create format instance");
return NULL;
}
/*
* We use the fid globally here so prevent the release_vg
* call to destroy the fid - we may want to reuse it!
*/
fid->ref_count++;
/*
* label_scan found PVs for this VG and set up lvmcache to describe the
* VG/PVs that we use here to read the VG. It created 'vginfo' for the
* VG, and created an 'info' attached to vginfo for each PV. It also
* added a metadata_area struct to info->mdas for each metadata area it
* found on the PV. The info->mdas structs are copied to
* fid->metadata_areas_in_use by create_instance above, and here we
* read VG metadata from each of those mdas.
*/
dm_list_iterate_items(mda, &fid->metadata_areas_in_use) {
mda_dev = mda_get_device(mda);
/* I don't think this can happen */
if (!mda_dev) {
log_warn("Ignoring metadata for VG %s from missing dev.", vgname);
continue;
}
use_previous_vg = 0;
if (use_precommitted) {
log_debug_metadata("Reading VG %s precommit metadata from %s %llu",
vgname, dev_name(mda_dev), (unsigned long long)mda->header_start);
pass cmd struct through more functions no functional change
2020-01-28 19:33:15 +03:00
vg = mda->ops->vg_read_precommit(cmd, fid, vgname, mda, &vg_fmtdata, &use_previous_vg);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
if (!vg && !use_previous_vg) {
log_warn("WARNING: Reading VG %s precommit on %s failed.", vgname, dev_name(mda_dev));
vg_fmtdata = NULL;
continue;
}
} else {
log_debug_metadata("Reading VG %s metadata from %s %llu",
vgname, dev_name(mda_dev), (unsigned long long)mda->header_start);
pass cmd struct through more functions no functional change
2020-01-28 19:33:15 +03:00
vg = mda->ops->vg_read(cmd, fid, vgname, mda, &vg_fmtdata, &use_previous_vg);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
if (!vg && !use_previous_vg) {
log_warn("WARNING: Reading VG %s on %s failed.", vgname, dev_name(mda_dev));
vg_fmtdata = NULL;
continue;
}
}
if (!vg)
continue;
if (vg && !vg_ret) {
vg_ret = vg;
dev_ret = mda_dev;
continue;
}
/*
* Use the newest copy of the metadata found on any mdas.
* Above, We could check if the scan found an old metadata
* seqno in this mda and just skip reading it again; then these
* seqno checks would just be sanity checks.
*/
if (vg->seqno == vg_ret->seqno) {
release_vg(vg);
vg_read: use else for 3 case Make it visible we check for ==, >, < of same var.
2019-11-28 17:09:27 +03:00
} else if (vg->seqno > vg_ret->seqno) {
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
log_warn("WARNING: ignoring metadata seqno %u on %s for seqno %u on %s for VG %s.",
vg_ret->seqno, dev_name(dev_ret),
vg->seqno, dev_name(mda_dev), vg->name);
found_old_metadata = 1;
release_vg(vg_ret);
vg_ret = vg;
dev_ret = mda_dev;
vg_fmtdata = NULL;
vg_read: use else for 3 case Make it visible we check for ==, >, < of same var.
2019-11-28 17:09:27 +03:00
} else { /* vg->seqno < vg_ret->seqno */
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
log_warn("WARNING: ignoring metadata seqno %u on %s for seqno %u on %s for VG %s.",
vg->seqno, dev_name(mda_dev),
vg_ret->seqno, dev_name(dev_ret), vg->name);
found_old_metadata = 1;
release_vg(vg);
vg_fmtdata = NULL;
}
}
improve info about vgck updatemetadata Add man page info about this option, and add log messages pointing to this option.
2020-06-03 20:38:27 +03:00
if (found_old_metadata) {
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
log_warn("WARNING: Inconsistent metadata found for VG %s.", vgname);
improve info about vgck updatemetadata Add man page info about this option, and add log messages pointing to this option.
2020-06-03 20:38:27 +03:00
log_warn("See vgck --updatemetadata to correct inconsistency.");
}
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
vg = NULL;
if (vg_ret)
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
set_pv_devices(fid, vg_ret, &found_md_component);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
fid->ref_count--;
if (!vg_ret) {
_destroy_fid(&fid);
goto_out;
}
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
/*
* Usually md components are eliminated during label scan, or duplicate
* resolution, but sometimes an md component can get through and be
* detected in set_pv_device() (which will do an md component check if
* the device/PV sizes don't match.) In this case we need to fix up
* lvmcache to drop the component dev and fix up metadata_areas_in_use
* to drop it also.
*/
if (found_md_component) {
dm_list_iterate_items(pvl, &vg_ret->pvs) {
if (!(dev = lvmcache_device_from_pvid(cmd, &pvl->pv->id, NULL)))
continue;
/* dev_is_md_component set this flag if it was found */
if (!(dev->flags & DEV_IS_MD_COMPONENT))
continue;
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
log_debug_metadata("Drop dev for MD component from cache %s.", dev_name(dev));
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
lvmcache_del_dev(dev);
clenaup: simpler form
2019-11-28 17:15:10 +03:00
dm_list_iterate_items(mda, &fid->metadata_areas_in_use)
if (mda_get_device(mda) == dev) {
log_debug_metadata("Drop mda from MD component from mda list %s.", dev_name(dev));
dm_list_del(&mda->list);
break;
}
md component detection addition in vg_read Usually md components are eliminated in label scan and/or duplicate resolution, but they could sometimes get into the vg_read stage, where set_pv_devices compares the device to the PV. If set_pv_devices runs an md component check and finds one, vg_read should eliminate the components. In set_pv_devices, run an md component check always if the PV is smaller than the device (this is not very common.) If the PV is larger than the device, (more common), do the component check when the config setting is "auto" (the default).
2019-08-01 22:43:19 +03:00
}
}
/*
* After dropping MD components there may be no remaining legitimate
* devices for this VG.
*/
if (!lvmcache_vginfo_from_vgid(vgid)) {
log_debug_metadata("VG %s not found on any remaining devices.", vgname);
release_vg(vg_ret);
vg_ret = NULL;
goto out;
}
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
/*
* Correct the lvmcache representation of the VG using the metadata
* that we have chosen above (vg_ret).
*
* The vginfo/info representation created by label_scan was not
* entirely correct since it did not use the full or final metadata.
*
* In lvmcache, PVs with no mdas were not attached to the vginfo during
* label_scan because label_scan didn't know where they should go. Now
* that we have the VG metadata we can tell, so use that to attach those
* info's to the vginfo.
*
* Also, outdated PVs that have been removed from the VG were incorrectly
* attached to the vginfo during label_scan, and now need to be detached.
*/
lvmcache_update_vg_from_read(vg_ret, vg_ret->status & PRECOMMITTED);
/*
* lvmcache_update_vg identified outdated mdas that we read above that
* are not actually part of the VG. Remove those outdated mdas from
* the fid's list of mdas.
*/
dm_list_iterate_items_safe(mda, mda2, &fid->metadata_areas_in_use) {
mda_dev = mda_get_device(mda);
if (lvmcache_is_outdated_dev(cmd, vg_ret->name, (const char *)&vg_ret->id, mda_dev)) {
log_debug_metadata("vg_read %s ignore mda for outdated dev %s",
vg_ret->name, dev_name(mda_dev));
dm_list_del(&mda->list);
}
}
out:
return vg_ret;
}
struct volume_group *vg_read(struct cmd_context *cmd, const char *vg_name, const char *vgid,
gcc: avoid warning: declaration of xxx shadows a global declaration Fix some gcc complaints again shadowing global declarations
2019-10-21 13:32:11 +03:00
uint32_t vg_read_flags, uint32_t lockd_state,
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
uint32_t *error_flags, struct volume_group **error_vg)
{
char uuidstr[64] __attribute__((aligned(8)));
struct volume_group *vg = NULL;
struct lv_list *lvl;
struct pv_list *pvl;
int missing_pv_dev = 0;
int missing_pv_flag = 0;
uint32_t failure = 0;
lvmcache: rework handling of VGs with duplicate vgnames The previous method of managing duplicate vgnames prevented vgreduce from working if a foreign vg with the same name existed.
2020-01-28 20:47:37 +03:00
int original_vgid_set = vgid ? 1 : 0;
gcc: avoid warning: declaration of xxx shadows a global declaration Fix some gcc complaints again shadowing global declarations
2019-10-21 13:32:11 +03:00
int writing = (vg_read_flags & READ_FOR_UPDATE);
int activating = (vg_read_flags & READ_FOR_ACTIVATE);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
if (is_orphan_vg(vg_name)) {
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
log_very_verbose("Reading orphan VG %s.", vg_name);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
vg = vg_read_orphans(cmd, vg_name);
*error_flags = 0;
*error_vg = NULL;
return vg;
}
if (!validate_name(vg_name)) {
log_error("Volume group name \"%s\" has invalid characters.", vg_name);
fix segfault for invalid characters in vg name Fixes a regression from commit ba7ff96faff0 "improve reading and repairing vg metadata" where the error path for a vg name with invalid charaters was missing an error flag, which led to the caller not recognizing an error occured. Previously, an error flag was hidden in the old _vg_make_handle function.
2019-08-29 19:35:46 +03:00
failure |= FAILED_NOTFOUND;
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
goto bad;
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
}
scanning: open devs rw when rescanning for write When vg_read rescans devices with the intention of writing the VG, the label rescan can open the devs RW so they do not need to be closed and reopened RW in dev_write_bytes.
2019-06-12 00:17:24 +03:00
/*
* When a command is reading the VG with the intention of eventually
* writing it, it passes the READ_FOR_UPDATE flag. This causes vg_read
* to acquire an exclusive VG lock, and causes vg_read to do some more
* checks, e.g. that the VG is writable and not exported. It also
* means that when the label scan is repeated on the VG's devices, the
* VG's PVs can be reopened read-write when rescanning in anticipation
* of needing to write to them.
*/
gcc: avoid warning: declaration of xxx shadows a global declaration Fix some gcc complaints again shadowing global declarations
2019-10-21 13:32:11 +03:00
if (!(vg_read_flags & READ_WITHOUT_LOCK) &&
pvscan: avoid full scan for activation When an online PV completed a VG, the standard activation functions were used to activate the VG. These functions use a full scan of all devs. When many pvscans are run during startup and need to activate many VGs, scanning all devs from all the pvscans can take a long time. Optimize VG activation in pvscan to scan only the devs in the VG being activated. This makes use of the online file info that was used to determine the VG was complete. The downside of this approach is that pvscan activation will not detect duplicate PVs and block activation, where a normal activation command (which scans all devices) would.
2019-08-27 01:07:18 +03:00
!lock_vol(cmd, vg_name, (writing || activating) ? LCK_VG_WRITE : LCK_VG_READ, NULL)) {
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
log_error("Can't get lock for %s.", vg_name);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
failure |= FAILED_LOCKING;
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
goto bad;
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
}
lvmcache: rework handling of VGs with duplicate vgnames The previous method of managing duplicate vgnames prevented vgreduce from working if a foreign vg with the same name existed.
2020-01-28 20:47:37 +03:00
/* I belive this is unused, the name is always set. */
if (!vg_name && !(vg_name = lvmcache_vgname_from_vgid(cmd->mem, vgid))) {
unlock_vg(cmd, NULL, vg_name);
log_error("VG name not found for vgid %s", vgid);
failure |= FAILED_NOTFOUND;
goto_bad;
}
/*
* If the command is process all vgs, process_each will get a list of vgname+vgid
* pairs, and then call vg_read() for each vgname+vigd. In this case we know
* which VG to read even if there are duplicate names, and we don't fail.
*
* If the user has requested one VG by name, process_each passes only the vgname
* to vg_read(), and we look up the vgid from lvmcache. lvmcache finds duplicate
* vgnames, doesn't know which is intended, returns a NULL vgid, and we fail.
*/
if (!vgid)
vgid = lvmcache_vgid_from_vgname(cmd, vg_name);
if (!vgid) {
unlock_vg(cmd, NULL, vg_name);
/* Some callers don't care if the VG doesn't exist and don't want an error message. */
if (!(vg_read_flags & READ_OK_NOTFOUND))
log_error("Volume group \"%s\" not found", vg_name);
failure |= FAILED_NOTFOUND;
goto_bad;
}
/*
* vgchange -ay (no vgname arg) will activate multiple local VGs with the same
* name, but if the vgs have the same lv name, activating those lvs will fail.
*/
if (activating && original_vgid_set && lvmcache_has_duplicate_local_vgname(vgid, vg_name))
log_warn("WARNING: activating multiple VGs with the same name is dangerous and may fail.");
scanning: open devs rw when rescanning for write When vg_read rescans devices with the intention of writing the VG, the label rescan can open the devs RW so they do not need to be closed and reopened RW in dev_write_bytes.
2019-06-12 00:17:24 +03:00
if (!(vg = _vg_read(cmd, vg_name, vgid, 0, writing))) {
lvmcache: rework handling of VGs with duplicate vgnames The previous method of managing duplicate vgnames prevented vgreduce from working if a foreign vg with the same name existed.
2020-01-28 20:47:37 +03:00
unlock_vg(cmd, NULL, vg_name);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
/* Some callers don't care if the VG doesn't exist and don't want an error message. */
gcc: avoid warning: declaration of xxx shadows a global declaration Fix some gcc complaints again shadowing global declarations
2019-10-21 13:32:11 +03:00
if (!(vg_read_flags & READ_OK_NOTFOUND))
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
log_error("Volume group \"%s\" not found.", vg_name);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
failure |= FAILED_NOTFOUND;
goto_bad;
}
/*
* Check and warn if PV ext info is not in sync with VG metadata
* (vg_write fixes.)
*/
_check_pv_ext(cmd, vg);
if (!vg_strip_outdated_historical_lvs(vg))
log_warn("WARNING: failed to strip outdated historical lvs.");
/*
* Check for missing devices in the VG. In most cases a VG cannot be
* changed while it's missing devices. This restriction is implemented
* here in vg_read. Below we return an error from vg_read if the
* vg_read flag indicates that the command is going to modify the VG.
* (We should probably implement this restriction elsewhere instead of
* returning an error from vg_read.)
*
* The PV's device may be present while the PV for the device has the
* MISSING_PV flag set in the metadata. This happened because the VG
* was written while this dev was missing, so the MISSING flag was
* written in the metadata for PV. Now the device has reappeared.
* However, the VG has changed since the device was last present, and
* if the device has outdated data it may not be safe to just start
* using it again.
*
* If there were no PE's used on the PV, we can just clear the MISSING
* flag, but if there were PE's used we need to continue to treat the
* PV as if the device is missing, limiting operations like the VG has
* a missing device, and requiring the user to remove the reappeared
* device from the VG, like a missing device, with vgreduce
* --removemissing.
*/
dm_list_iterate_items(pvl, &vg->pvs) {
if (!id_write_format(&pvl->pv->id, uuidstr, sizeof(uuidstr)))
uuidstr[0] = '\0';
if (!pvl->pv->dev) {
/* The obvious and common case of a missing device. */
drop warnings about missing pvs in foreign vgs When a foreign VG is ignored, don't print warnings that it is missing PVs.
2019-12-11 21:56:15 +03:00
if (vg_is_foreign(vg) && !cmd->include_foreign_vgs)
log_debug("VG %s is missing PV %s (last written to %s)", vg_name, uuidstr, pvl->pv->device_hint ?: "na");
else if (pvl->pv->device_hint)
metadata: import device name hint from metadata Start by using it in a comment for a missing PV.
2019-09-04 22:13:14 +03:00
log_warn("WARNING: VG %s is missing PV %s (last written to %s).", vg_name, uuidstr, pvl->pv->device_hint);
else
log_warn("WARNING: VG %s is missing PV %s.", vg_name, uuidstr);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
missing_pv_dev++;
} else if (pvl->pv->status & MISSING_PV) {
/* A device that was missing but has reappeared. */
if (pvl->pv->pe_alloc_count == 0) {
log_warn("WARNING: VG %s has unused reappeared PV %s %s.", vg_name, dev_name(pvl->pv->dev), uuidstr);
pvl->pv->status &= ~MISSING_PV;
/* tell vgextend restoremissing that MISSING flag was cleared here */
pvl->pv->unused_missing_cleared = 1;
} else {
log_warn("WARNING: VG %s was missing PV %s %s.", vg_name, dev_name(pvl->pv->dev), uuidstr);
missing_pv_flag++;
}
}
}
if (missing_pv_dev || missing_pv_flag)
vg_mark_partial_lvs(vg, 1);
if (!check_pv_segments(vg)) {
log_error(INTERNAL_ERROR "PV segments corrupted in %s.", vg->name);
failure |= FAILED_INTERNAL_ERROR;
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
goto bad;
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
}
dm_list_iterate_items(lvl, &vg->lvs) {
if (!check_lv_segments(lvl->lv, 0)) {
log_error(INTERNAL_ERROR "LV segments corrupted in %s.", lvl->lv->name);
failure |= FAILED_INTERNAL_ERROR;
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
goto bad;
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
}
}
dm_list_iterate_items(lvl, &vg->lvs) {
/* Checks that cross-reference other LVs. */
if (!check_lv_segments(lvl->lv, 1)) {
log_error(INTERNAL_ERROR "LV segments corrupted in %s.", lvl->lv->name);
failure |= FAILED_INTERNAL_ERROR;
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
goto bad;
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
}
}
if (!check_pv_dev_sizes(vg))
log_warn("WARNING: One or more devices used as PVs in VG %s have changed sizes.", vg->name);
_check_devs_used_correspond_with_vg(vg);
if (!_access_vg_lock_type(cmd, vg, lockd_state, &failure)) {
/* Either FAILED_LOCK_TYPE or FAILED_LOCK_MODE were set. */
goto_bad;
}
if (!_access_vg_systemid(cmd, vg)) {
failure |= FAILED_SYSTEMID;
goto_bad;
}
if (!_access_vg_clustered(cmd, vg)) {
failure |= FAILED_CLUSTERED;
goto_bad;
}
exported vg handling The exported VG checking/enforcement was scattered and inconsistent. This centralizes it and makes it consistent, following the existing approach for foreign and shared VGs/PVs, which are very similar to exported VGs/PVs. The access policy that now applies to foreign/shared/exported VGs/PVs, is that if a foreign/shared/exported VG/PV is named on the command line (i.e. explicitly requested by the user), and the command is not permitted to operate on it because it is foreign/shared/exported, then an access error is reported and the command exits with an error. But, if the command is processing all VGs/PVs, and happens to come across a foreign/shared/exported VG/PV (that is not explicitly named on the command line), then the command silently skips it and does not produce an error. A command using tags or --select handles inaccessible VGs/PVs the same way as a command processing all VGs/PVs, and will not report/return errors if these inaccessible VGs/PVs exist. The new policy fixes the exit codes on a somewhat random set of commands that previously exited with an error if they were looking at all VGs/PVs and an exported VG existed on the system. There should be no change to which commands are allowed/disallowed on exported VGs/PVs. Certain LV commands (lvs/lvdisplay/lvscan) would previously not display LVs from an exported VG (for unknown reasons). This has not changed. The lvm fullreport command would previously report info about an exported VG but not about the LVs in it. This has changed to include all info from the exported VG.
2019-06-21 21:37:11 +03:00
if (!_access_vg_exported(cmd, vg)) {
failure |= FAILED_EXPORTED;
goto_bad;
}
scanning: open devs rw when rescanning for write When vg_read rescans devices with the intention of writing the VG, the label rescan can open the devs RW so they do not need to be closed and reopened RW in dev_write_bytes.
2019-06-12 00:17:24 +03:00
/*
* If the command intends to write or activate the VG, there are
* additional restrictions. FIXME: These restrictions should
* probably be checked/applied after vg_read returns.
*/
if (writing || activating) {
if (!(vg->status & LVM_WRITE)) {
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
log_error("Volume group %s is read-only.", vg->name);
scanning: open devs rw when rescanning for write When vg_read rescans devices with the intention of writing the VG, the label rescan can open the devs RW so they do not need to be closed and reopened RW in dev_write_bytes.
2019-06-12 00:17:24 +03:00
failure |= FAILED_READ_ONLY;
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
goto bad;
scanning: open devs rw when rescanning for write When vg_read rescans devices with the intention of writing the VG, the label rescan can open the devs RW so they do not need to be closed and reopened RW in dev_write_bytes.
2019-06-12 00:17:24 +03:00
}
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
scanning: open devs rw when rescanning for write When vg_read rescans devices with the intention of writing the VG, the label rescan can open the devs RW so they do not need to be closed and reopened RW in dev_write_bytes.
2019-06-12 00:17:24 +03:00
if (!cmd->handles_missing_pvs && (missing_pv_dev || missing_pv_flag)) {
log_error("Cannot change VG %s while PVs are missing.", vg->name);
log_error("See vgreduce --removemissing and vgextend --restoremissing.");
failure |= FAILED_NOT_ENABLED;
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
goto bad;
scanning: open devs rw when rescanning for write When vg_read rescans devices with the intention of writing the VG, the label rescan can open the devs RW so they do not need to be closed and reopened RW in dev_write_bytes.
2019-06-12 00:17:24 +03:00
}
allow activating known LVs when other LVs have unknown segtypes When a VG contains some LVs with unknown segtypes, the user should still be allowed to activate other LVs in the VG that are understood. $ lvs foo WARNING: Unrecognised flag CACHE_USES_CACHEVOL in segment type cache+CACHE_USES_CACHEVOL. WARNING: Unrecognised segment type cache+CACHE_USES_CACHEVOL LV VG Attr LSize lvol0 foo -wi------- 4.00m other foo vwi---u--- 48.00m $ lvcreate -l1 foo WARNING: Unrecognised flag CACHE_USES_CACHEVOL in segment type cache+CACHE_USES_CACHEVOL. WARNING: Unrecognised segment type cache+CACHE_USES_CACHEVOL Cannot change VG foo with unknown segments in it! Cannot process volume group foo $ lvchange -ay foo/lvol0 WARNING: Unrecognised flag CACHE_USES_CACHEVOL in segment type cache+CACHE_USES_CACHEVOL. WARNING: Unrecognised segment type cache+CACHE_USES_CACHEVOL $ lvchange -ay foo/other WARNING: Unrecognised flag CACHE_USES_CACHEVOL in segment type cache+CACHE_USES_CACHEVOL. WARNING: Unrecognised segment type cache+CACHE_USES_CACHEVOL Refusing activation of LV foo/other containing an unrecognised segment. $ lvs foo WARNING: Unrecognised flag CACHE_USES_CACHEVOL in segment type cache+CACHE_USES_CACHEVOL. WARNING: Unrecognised segment type cache+CACHE_USES_CACHEVOL LV VG Attr LSize lvol0 foo -wi-a----- 4.00m other foo vwi---u--- 48.00m
2019-10-14 23:51:35 +03:00
}
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
allow activating known LVs when other LVs have unknown segtypes When a VG contains some LVs with unknown segtypes, the user should still be allowed to activate other LVs in the VG that are understood. $ lvs foo WARNING: Unrecognised flag CACHE_USES_CACHEVOL in segment type cache+CACHE_USES_CACHEVOL. WARNING: Unrecognised segment type cache+CACHE_USES_CACHEVOL LV VG Attr LSize lvol0 foo -wi------- 4.00m other foo vwi---u--- 48.00m $ lvcreate -l1 foo WARNING: Unrecognised flag CACHE_USES_CACHEVOL in segment type cache+CACHE_USES_CACHEVOL. WARNING: Unrecognised segment type cache+CACHE_USES_CACHEVOL Cannot change VG foo with unknown segments in it! Cannot process volume group foo $ lvchange -ay foo/lvol0 WARNING: Unrecognised flag CACHE_USES_CACHEVOL in segment type cache+CACHE_USES_CACHEVOL. WARNING: Unrecognised segment type cache+CACHE_USES_CACHEVOL $ lvchange -ay foo/other WARNING: Unrecognised flag CACHE_USES_CACHEVOL in segment type cache+CACHE_USES_CACHEVOL. WARNING: Unrecognised segment type cache+CACHE_USES_CACHEVOL Refusing activation of LV foo/other containing an unrecognised segment. $ lvs foo WARNING: Unrecognised flag CACHE_USES_CACHEVOL in segment type cache+CACHE_USES_CACHEVOL. WARNING: Unrecognised segment type cache+CACHE_USES_CACHEVOL LV VG Attr LSize lvol0 foo -wi-a----- 4.00m other foo vwi---u--- 48.00m
2019-10-14 23:51:35 +03:00
if (writing && !cmd->handles_unknown_segments && vg_has_unknown_segments(vg)) {
log_error("Cannot change VG %s with unknown segments in it!", vg->name);
failure |= FAILED_NOT_ENABLED; /* FIXME new failure code here? */
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
goto bad;
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
}
/*
* When we are reading the VG with the intention of writing it,
* we save a second copy of the VG in vg->vg_committed. This
* copy remains unmodified by the command operation, and is used
* later if there is an error and we want to reactivate LVs.
* FIXME: be specific about exactly when this works correctly.
*/
if (writing) {
struct dm_config_tree *cft;
if (dm_pool_locked(vg->vgmem)) {
/* FIXME: can this happen? */
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
log_warn("WARNING: vg_read no vg copy: pool locked.");
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
goto out;
}
if (vg->vg_committed) {
/* FIXME: can this happen? */
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
log_warn("WARNING: vg_read no vg copy: copy exists.");
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
release_vg(vg->vg_committed);
vg->vg_committed = NULL;
}
if (vg->vg_precommitted) {
/* FIXME: can this happen? */
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
log_warn("WARNING: vg_read no vg copy: pre copy exists.");
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
release_vg(vg->vg_precommitted);
vg->vg_precommitted = NULL;
}
if (!(cft = export_vg_to_config_tree(vg))) {
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
log_warn("WARNING: vg_read no vg copy: copy export failed.");
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
goto out;
}
metadata: add vg_from_config_tree Add cmd/fmt args to import functions so that they can be used without the fid arg which.
2019-10-29 22:31:37 +03:00
if (!(vg->vg_committed = import_vg_from_config_tree(cmd, vg->fid, cft)))
debug: enhance messages Drop 'extra' stack trace where errors are already logged from function. Add some missing dots in messages.
2019-11-28 15:19:44 +03:00
log_warn("WARNING: vg_read no vg copy: copy import failed.");
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
dm_config_destroy(cft);
} else {
if (vg->vg_precommitted)
gcc: use apropriate type for reading and printing values
2020-08-28 20:35:25 +03:00
log_error(INTERNAL_ERROR "vg_read vg %p vg_precommitted %p", (void *)vg, (void *)vg->vg_precommitted);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
if (vg->vg_committed)
gcc: use apropriate type for reading and printing values
2020-08-28 20:35:25 +03:00
log_error(INTERNAL_ERROR "vg_read vg %p vg_committed %p", (void *)vg, (void *)vg->vg_committed);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
}
out:
/* We return with the VG lock held when read is successful. */
*error_flags = SUCCESS;
if (error_vg)
*error_vg = NULL;
return vg;
bad:
*error_flags = failure;
/*
* FIXME: get rid of this case so we don't have to return the vg when
* there's an error. It is here for process_each_pv() which wants to
* eliminate the VG's devs from the list of devs it is processing, even
* when it can't access the VG because of wrong system id or similar.
* This could be done by looking at lvmcache info structs intead of 'vg'.
* It's also used by process_each_vg/process_each_lv which want to
* include error_vg values (like system_id) in error messages.
* These values could also be found from lvmcache vginfo.
*/
if (error_vg && vg) {
if (vg->vg_precommitted)
gcc: use apropriate type for reading and printing values
2020-08-28 20:35:25 +03:00
log_error(INTERNAL_ERROR "vg_read vg %p vg_precommitted %p", (void *)vg, (void *)vg->vg_precommitted);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
if (vg->vg_committed)
gcc: use apropriate type for reading and printing values
2020-08-28 20:35:25 +03:00
log_error(INTERNAL_ERROR "vg_read vg %p vg_committed %p", (void *)vg, (void *)vg->vg_committed);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
/* caller must unlock_vg and release_vg */
*error_vg = vg;
return_NULL;
}
if (vg) {
unlock_vg(cmd, vg, vg_name);
release_vg(vg);
}
if (error_vg)
*error_vg = NULL;
return_NULL;
}
/*
* Simply a version of vg_read() that automatically sets the READ_FOR_UPDATE
* flag, which means the caller intends to write the VG after reading it,
* so vg_read should acquire an exclusive file lock on the vg.
*/
struct volume_group *vg_read_for_update(struct cmd_context *cmd, const char *vg_name,
gcc: avoid warning: declaration of xxx shadows a global declaration Fix some gcc complaints again shadowing global declarations
2019-10-21 13:32:11 +03:00
const char *vgid, uint32_t vg_read_flags, uint32_t lockd_state)
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
{
struct volume_group *vg;
uint32_t error_flags = 0;
gcc: avoid warning: declaration of xxx shadows a global declaration Fix some gcc complaints again shadowing global declarations
2019-10-21 13:32:11 +03:00
vg = vg_read(cmd, vg_name, vgid, vg_read_flags | READ_FOR_UPDATE, lockd_state, &error_flags, NULL);
improve reading and repairing vg metadata The fact that vg repair is implemented as a part of vg read has led to a messy and complicated implementation of vg_read, and limited and uncontrolled repair capability. This splits read and repair apart. Summary ------- - take all kinds of various repairs out of vg_read - vg_read no longer writes anything - vg_read now simply reads and returns vg metadata - vg_read ignores bad or old copies of metadata - vg_read proceeds with a single good copy of metadata - improve error checks and handling when reading - keep track of bad (corrupt) copies of metadata in lvmcache - keep track of old (seqno) copies of metadata in lvmcache - keep track of outdated PVs in lvmcache - vg_write will do basic repairs - new command vgck --updatemetdata will do all repairs Details ------- - In scan, do not delete dev from lvmcache if reading/processing fails; the dev is still present, and removing it makes it look like the dev is not there. Records are now kept about the problems with each PV so they be fixed/repaired in the appropriate places. - In scan, record a bad mda on failure, and delete the mda from mda in use list so it will not be used by vg_read or vg_write, only by repair. - In scan, succeed if any good mda on a device is found, instead of failing if any is bad. The bad/old copies of metadata should not interfere with normal usage while good copies can be used. - In scan, add a record of old mdas in lvmcache for later, do not repair them while reading, and do not let them prevent us from finding and using a good copy of metadata from elsewhere. One result is that "inconsistent metadata" is no longer a read error, but instead a record in lvmcache that can be addressed separate from the read. - Treat a dev with no good mdas like a dev with no mdas, which is an existing case we already handle. - Don't use a fake vg "handle" for returning an error from vg_read, or the vg_read_error function for getting that error number; just return null if the vg cannot be read or used, and an error_flags arg with flags set for the specific kind of error (which can be used later for determining the kind of repair.) - Saving an original copy of the vg metadata, for purposes of reverting a write, is now done explicitly in vg_read instead of being hidden in the vg_make_handle function. - When a vg is not accessible due to "access restrictions" but is otherwise fine, return the vg through the new error_vg arg so that process_each_pv can skip the PVs in the VG while processing. (This is a temporary accomodation for the way process_each_pv tracks which devs have been looked at, and can be dropped later when process_each_pv implementation dev tracking is changed.) - vg_read does not try to fix or recover a vg, but now just reads the metadata, checks access restrictions and returns it. (Checking access restrictions might be better done outside of vg_read, but this is a later improvement.) - _vg_read now simply makes one attempt to read metadata from each mda, and uses the most recent copy to return to the caller in the form of a 'vg' struct. (bad mdas were excluded during the scan and are not retried) (old mdas were not excluded during scan and are retried here) - vg_read uses _vg_read to get the latest copy of metadata from mdas, and then makes various checks against it to produce warnings, and to check if VG access is allowed (access restrictions include: writable, foreign, shared, clustered, missing pvs). - Things that were previously silently/automatically written by vg_read that are now done by vg_write, based on the records made in lvmcache during the scan and read: . clearing the missing flag . updating old copies of metadata . clearing outdated pvs . updating pv header flags - Bad/corrupt metadata are now repaired; they were not before. Test changes ------------ - A read command no longer writes the VG to repair it, so add a write command to do a repair. (inconsistent-metadata, unlost-pv) - When a missing PV is removed from a VG, and then the device is enabled again, vgck --updatemetadata is needed to clear the outdated PV before it can be used again, where it wasn't before. (lvconvert-repair-policy, lvconvert-repair-raid, lvconvert-repair, mirror-vgreduce-removemissing, pv-ext-flags, unlost-pv) Reading bad/old metadata ------------------------ - "bad metadata": the mda_header or metadata text has invalid fields or can't be parsed by lvm. This is a form of corruption that would not be caused by known failure scenarios. A checksum error is typically included among the errors reported. - "old metadata": a valid copy of the metadata that has a smaller seqno than other copies of the metadata. This can happen if the device failed, or io failed, or lvm failed while commiting new metadata to all the metadata areas. Old metadata on a PV that has been removed from the VG is the "outdated" case below. When a VG has some PVs with bad/old metadata, lvm can simply ignore the bad/old copies, and use a good copy. This is why there are multiple copies of the metadata -- so it's available even when some of the copies cannot be used. The bad/old copies do not have to be repaired before the VG can be used (the repair can happen later.) A PV with no good copies of the metadata simply falls back to being treated like a PV with no mdas; a common and harmless configuration. When bad/old metadata exists, lvm warns the user about it, and suggests repairing it using a new metadata repair command. Bad metadata in particular is something that users will want to investigate and repair themselves, since it should not happen and may indicate some other problem that needs to be fixed. PVs with bad/old metadata are not the same as missing devices. Missing devices will block various kinds of VG modification or activation, but bad/old metadata will not. Previously, lvm would attempt to repair bad/old metadata whenever it was read. This was unnecessary since lvm does not require every copy of the metadata to be used. It would also hide potential problems that should be investigated by the user. It was also dangerous in cases where the VG was on shared storage. The user is now allowed to investigate potential problems and decide how and when to repair them. Repairing bad/old metadata -------------------------- When label scan sees bad metadata in an mda, that mda is removed from the lvmcache info->mdas list. This means that vg_read will skip it, and not attempt to read/process it again. If it was the only in-use mda on a PV, that PV is treated like a PV with no mdas. It also means that vg_write will also skip the bad mda, and not attempt to write new metadata to it. The only way to repair bad metadata is with the metadata repair command. When label scan sees old metadata in an mda, that mda is kept in the lvmcache info->mdas list. This means that vg_read will read/process it again, and likely see the same mismatch with the other copies of the metadata. Like the label_scan, the vg_read will simply ignore the old copy of the metadata and use the latest copy. If the command is modifying the vg (e.g. lvcreate), then vg_write, which writes new metadata to every mda on info->mdas, will write the new metadata to the mda that had the old version. If successful, this will resolve the old metadata problem (without needing to run a metadata repair command.) Outdated PVs ------------ An outdated PV is a PV that has an old copy of VG metadata that shows it is a member of the VG, but the latest copy of the VG metadata does not include this PV. This happens if the PV is disconnected, vgreduce --removemissing is run to remove the PV from the VG, then the PV is reconnected. In this case, the outdated PV needs have its outdated metadata removed and the PV used flag needs to be cleared. This repair will be done by the subsequent repair command. It is also done if vgremove is run on the VG. MISSING PVs ----------- When a device is missing, most commands will refuse to modify the VG. This is the simple case. More complicated is when a command is allowed to modify the VG while it is missing a device. When a VG is written while a device is missing for one of it's PVs, the VG metadata is written to disk with the MISSING flag on the PV with the missing device. When the VG is next used, it is treated as if the PV with the MISSING flag still has a missing device, even if that device has reappeared. If all LVs that were using a PV with the MISSING flag are removed or repaired so that the MISSING PV is no longer used, then the next time the VG metadata is written, the MISSING flag will be dropped. Alternative methods of clearing the MISSING flag are: vgreduce --removemissing will remove PVs with missing devices, or PVs with the MISSING flag where the device has reappeared. vgextend --restoremissing will clear the MISSING flag on PVs where the device has reappeared, allowing the VG to be used normally. This must be done with caution since the reappeared device may have old data that is inconsistent with data on other PVs. Bad mda repair -------------- The new command: vgck --updatemetadata VG first uses vg_write to repair old metadata, and other basic issues mentioned above (old metadata, outdated PVs, pv_header flags, MISSING_PV flags). It will also go further and repair bad metadata: . text metadata that has a bad checksum . text metadata that is not parsable . corrupt mda_header checksum and version fields (To keep a clean diff, #if 0 is added around functions that are replaced by new code. These commented functions are removed by the following commit.)
2019-05-24 20:04:37 +03:00
return vg;
}
Copy Permalink