1
0
mirror of git://sourceware.org/git/lvm2.git synced 2024-12-30 17:18:21 +03:00
Commit Graph

782 Commits

Author SHA1 Message Date
Zdenek Kabelac
c451d7925d cleanup: correcting some log_print
Correcting to log_print_unless_silent(),
so -qq can do some work.
2023-05-06 20:38:49 +02:00
Zdenek Kabelac
7bc5c8ac3d cov: avoid using strcpy
Coverity is complaining about unchecked strcpy here, which is
irelevant as we preallocate buffer to fit in copied string,
however we could actually reuse these size and use just memcpy().
So lets make some simple conversions.
2023-02-17 00:00:04 +01:00
Zdenek Kabelac
f89c369a01 archive: update message
Better suggesting message as suggested by RHBZ 2123151.
2023-02-01 11:47:30 +01:00
Wu Guanghao
ce58e9d5b3 _vg_read_raw_area: fix segfault caused by using null pointer
When we tested lvm2, the kernel injected various random faults.

(gdb) bt
...
(gdb) p vg
$1 = (struct volume_group *) 0x0
(gdb) p use_previous_vg
$2 = (unsigned int *) 0x0

Signed-off-by: Wu Guanghao <wuguanghao3@huawei.com>
2022-08-15 09:39:02 -05:00
Zdenek Kabelac
f83b3962c1 asan: fix some reports from libasan
When compiled and used with:

CFLAGS="-fsanitize=address -g -O0"
ASAN_OPTIONS=strict_string_checks=1:detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1

we have few reported issue - they where not normally spotted, since
we were still accessing our own memory - but ouf of buffer-range.

TODO: there is still something to enhance with handling of #orphan vgids
2022-02-07 20:02:11 +01:00
David Teigland
114e1cfee5 fix spelling of pruning 2021-11-19 12:02:35 -06:00
Zdenek Kabelac
2779830a06 cov: avoid using NULL info
Check lvmcache info exists before calling lvmcache_del_save_bad_mda().
2021-10-15 23:36:22 +02:00
Zdenek Kabelac
14cf579263 gcc: archive reoder struct vars
Easier for automatic struct initialization = { 0 } without warns
from older gcc.
2021-10-15 00:09:07 +02:00
Zdenek Kabelac
8784211958 debug: reduce some debugging noise
Resolve event_activation configure option just once.

Do not print debug_devs about 'bad' filtering, when
actually filter already printed reason for skipping

Do not trace more then once about backup being disabled.

No debug when unlinked file does not exists in pvscan.
2021-10-14 23:34:11 +02:00
Zdenek Kabelac
af0ce203e9 debug: better error message
Add more details about size difference.
2021-10-14 23:34:11 +02:00
Zdenek Kabelac
7e346ee2a5 archiving: update refactoring
Commit 5ea426e656 handled case with
file path without '/' incorrectly - there is valid use-case so
switch 'log_error()' to valid code branch.
2021-10-14 23:33:26 +02:00
Zdenek Kabelac
197f6aaad7 cleanup: skip zeroing buffer overwritten by memcpy
Since the buffer is going to be overwritten by memcpy, reset only last
char to 0.

Also more use of DM_LIST_INIT list.h macro.
2021-10-06 15:43:31 +02:00
Zdenek Kabelac
5ea426e656 archiving: refactor code to allocate less memory
Do not store full path with each archived name reduces memory usage if
the directory has thousands of entries and just add 'dir' path when
needed.

Also emit info print message to a user if the total size of archived
files for a VG is more then 128MiB or 8192 files.

TODO: Consider wheather adding a new 'lvm.conf  archive{option}' to support
trimming these wild archive sizes can make situation better.
We already support retain_min && retain_days  - but if user is
generating too many and too large archives with minutes - maybe archiving
should be disabled by a user - as it's not producing anything largely usable
and just slows-down command ??
If we add 'retain_max & retain_max_size' the condition will go against
each other and we need to chose priorities.

mm
2021-10-06 15:42:53 +02:00
David Teigland
939b4bc587 handle bad metadata text in vg_read path
Corrupt metadata text (with good mda header) was being handled
in the label_scan phase, but not in the vg_read phase.  This
was sufficient because metadata areas would always be read and
checksummed during label_scan (metadata parsing was skipped
previously as an optimization.)

This changed with the optimization in
commit 61a6f9905e
"metadata: optimize reading metadata copies in scan"

Now, some metadata areas will not be read and checksummed
at all during the label_scan phase, only during the vg_read
phase.  This means that bad metadata text may first be detected
in the vg_read phase.  So, add equivalent bad metadata handling
to the vg_read path to match the label_scan path.
2021-09-28 15:17:43 -05:00
Zdenek Kabelac
e3b4c365a4 debug: shorter code 2021-09-27 18:56:14 +02:00
Zdenek Kabelac
a8a958cc04 cleanup: reuse existing code for free path 2021-09-27 18:56:14 +02:00
Zdenek Kabelac
dab4acb8f8 make: build without versionsort
Reported-by: adamboardman of gemian
2021-09-27 18:56:14 +02:00
Zdenek Kabelac
61a6f9905e metadata: optimize reading metadata copies in scan
While being in lockless scanning phase, we can avoid reading and checking
matching metadata copies if we already know them from other PV
and just rely on matching metadata header information.

These copies will be examined later during locked metadata read/write
access.

This patch may postpone discovering some read failures to locked phase.
2021-09-27 18:56:13 +02:00
Zdenek Kabelac
24e90f9594 metadata: remember parsing size of VG metadata
When creating lvm2 metadata for VG, lvm2 allocate some buffer,
and if buffer is not big enough, the buffer is 'reallocated' bigger,
and whole metadata creation is repeated until metadata fits.
We can try to use 'previous' metadata size as hint to reduce looping
here.
2021-09-27 18:49:41 +02:00
Zdenek Kabelac
a42ea349f6 metadata: remember crc32 when writing vg metadata
Preserve computed crc32 check from first written PV, just like we
preserve generated metadata.

Also there is no need to call crc32 twice on wrapping buffer with 2 calcs,
result must be always the same as with single crc32 checking.
2021-09-27 18:49:41 +02:00
Zdenek Kabelac
becffe4567 gcc-fanalyzer: drop zeroing of pointers
This explicit pointer zeroing is not needed since
we use 'zalloced' buffers.
2021-09-20 14:18:13 +02:00
Zdenek Kabelac
63930f576a cov: add some initializers 2021-09-13 12:34:41 +02:00
David Teigland
9e79a022ea logging: messages for found metadata 2021-08-20 14:51:09 -05:00
David Teigland
96b777167c cov: clean up pvid and vgid usage
pvid and vgid are sometimes a null-terminated string, and
other times a 'struct id', and the two types were often
cast between each other.  When a struct id was cast to a char
pointer, the resulting string would not necessarily be null
terminated.  Casting a null-terminated string id to a
struct id is fine, but is still avoided when possible.

A struct id is:  int8_t uuid[ID_LEN]
A string id is:  char pvid[ID_LEN + 1]

A convention is introduced to help distinguish them:

- variables and struct fields named "pvid" or "vgid"
  should be null-terminated strings.

- variables and struct fields named "pv_id" or "vg_id"
  should be struct id's.

- examples:
  char pvid[ID_LEN + 1];
  char vgid[ID_LEN + 1];
  struct id pv_id;
  struct id vg_id;

Function names also attempt to follow this convention.

Avoid casting between the two types as much as possible,
with limited exceptions when known to be safe and clearly
commented.

Avoid using variations of strcpy and strcmp, and instead
use memcpy/memcmp with ID_LEN (with similar limited
exceptions possible.)
2021-08-16 11:31:15 -05:00
Zdenek Kabelac
a4eb378fe4 cov: add explicit check for vg existence
Since we check !vg - add check before accessing ->status.
2021-07-28 00:49:28 +02:00
David Teigland
b876dbfc24 scan: move metadata vgname check
There have been two separate checks for metadata
validity: first that the metadata text begins with
a valid VG name, and second the checksum of the
metadata text.  These happen in different places,
which means there have been two separate error paths
for invalid metadata.  This also causes large metadata
to be read in multiple parts, the first part is read
just to check the vgname, and then remaining parts are
read later when the full metadata is needed.

This patch moves the vg name verification so it's
done just before the checksum verification, which
results in a single error path for invalid metadata,
and causes the entire metadata to be read together
rather that in parts from different parts of the code.
2021-07-06 10:10:23 -05:00
David Teigland
e035e32350 scan: retry reading metadata on error
If label_scan encounters bad vg metadata, invalidate
bcache data for the device and reread the mda_header
and metadata text back to back.  With concurrent commands
modifying large metadata, it's possible that the entire
metadata area can be rewritten in the time between a
command reading the mda_header and reading the metadata
text that the header points to.  Since the label_scan
is just assembling an initial overview of devices, it
doesn't use locking to serialize with other commands
that may be modifying the vg metadata at the same time.
2021-07-06 10:10:23 -05:00
Zdenek Kabelac
bb45e33518 backup: automatically store data on vg_unlock
Previously there have been necessary explicit call of backup (often
either forgotten or over-used). With this patch the necessity to
store backup is remember at vg_commit and once the VG is unlocked,
the committed metadata are automatically store in backup file.

This may possibly alter some printed messages from command when the
backup is now taken later.
2021-06-09 14:56:13 +02:00
Zdenek Kabelac
25b672417e cov: use correct enum type
Correct function declaration.
2021-04-23 23:00:55 +02:00
David Teigland
0a28e3c44b Add metadata-based autoactivation property for VG and LV
The autoactivation property can be specified in lvcreate
or vgcreate for new LVs/VGs, and the property can be changed
by lvchange or vgchange for existing LVs/VGs.

 --setautoactivation y|n
 enables|disables autoactivation of a VG or LV.

Autoactivation is enabled by default, which is consistent with
past behavior.  The disabled state is stored as a new flag
in the VG metadata, and the absence of the flag allows
autoactivation.

If autoactivation is disabled for the VG, then no LVs in the VG
will be autoactivated (the LV autoactivation property will have
no effect.)  When autoactivation is enabled for the VG, then
autoactivation can be controlled on individual LVs.

The state of this property can be reported for LVs/VGs using
the "-o autoactivation" option in lvs/vgs commands, which will
report "enabled", or "" for the disabled state.

Previous versions of lvm do not recognize this property.  Since
autoactivation is enabled by default, the disabled setting will
have no effect in older lvm versions.  If the VG is modified by
older lvm versions, the disabled state will also be dropped from
the metadata.

The autoactivation property is an alternative to using the lvm.conf
auto_activation_volume_list, which is still applied to to VGs/LVs
in addition to the new property.

If VG or LV autoactivation is disabled either in metadata or in
auto_activation_volume_list, it will not be autoactivated.

An autoactivation command will silently skip activating an LV
when the autoactivation property is disabled.

To determine the effective autoactivation behavior for a specific
LV, multiple settings would need to be checked:
the VG autoactivation property, the LV autoactivation property,
the auto_activation_volume_list.  The "activation skip" property
would also be relevant, since it applies to both normal and auto
activation.
2021-04-07 15:32:49 -05:00
Zdenek Kabelac
8a03675241 cov: variable initialization 2021-03-10 01:34:58 +01:00
Zdenek Kabelac
c05077f4d4 cleanup: easier print of historical lv
Drop unneeded macros.
2021-03-10 01:11:52 +01:00
Zdenek Kabelac
d2a3bfe6ca vg_write: shift check for exported metadata
Since we now immediatelly use created buffer for CFT creation,
the result needs to be validate before such use.
2021-03-10 01:00:18 +01:00
Zdenek Kabelac
e5456c259f cleanup: simpler checks first
Minor optimizatoins...
2021-03-08 15:43:27 +01:00
Zdenek Kabelac
f4200acac2 cleanup: on stack structure instead of allocation 2021-03-08 15:43:27 +01:00
Zdenek Kabelac
dac990ae03 cleanup: avoid allocating memory for hash key
Hash always allocates its own copy of the key so avoid
this extra uuid copy.
2021-03-08 15:43:27 +01:00
Zdenek Kabelac
2d64ffaee5 hash: use individual hint sizes
Use different 'hint' size for dm_hash_create() call - so
when debug info about hash is printed we can recognize which
hash was in use.

This patch doesn't change actual used size since that is always
rounded to be power of 2 and >=16 - so as such is only a
help to developer.

We could eventually use 'name' arg, but since this would have changed
API and this patchset will be routed to libdm & stable - we will
just use this small trick.
2021-03-08 15:33:15 +01:00
Zdenek Kabelac
936c7b5104 vg_read: reuse already parsed config tree
When parsing VG metadata we can create from a single config tree
also 'vg_committed' that is always created for writable VG.

This avoids extra uncessary step of serializing and deserilizing
just parsed VG.
2021-03-08 15:30:18 +01:00
Zdenek Kabelac
bc0cb66304 vg_write: optimize caching of precommitted VG
Every vg_write stores new 'metadata' into precommitted slot.
For this step we use 'serialized buffer' to ascii metadata.

Instead of recreating this buffer after whole 'vg_write()' we
use this buffer instantly for creating of precommitted VG.

This has also the advantage of catching any problems with
reparsing of ascii metadata back to VG early before any write.
2021-03-08 15:30:18 +01:00
Zdenek Kabelac
eb3dcc72eb cleanup: free already checks for NULL 2021-03-02 22:57:35 +01:00
Zdenek Kabelac
2a9a3346e7 archive: support interruption
When lvm2 calls archive() or backup() it can be useful to allow handling
break signal so the command can be interrupted at some consistent point.

Signal is accepted during processing these calls - and can be evaluated
later during even lengthy processing loops.

So now user can interrupt lengthy lvremove().
2021-03-02 22:54:40 +01:00
Zdenek Kabelac
eb1160ee42 lvremove: backup at the end of loop
Taking backup with each removed LV is slowing down the process
considerable and is largerly uneeded. We are supposed to take
backup only on significant points and making sure the backup
is correct when the command is finished.

TODO: check how many other commands can be improved.
2021-03-02 22:54:40 +01:00
David Teigland
83fe6e720f device usage based on devices file
The LVM devices file lists devices that lvm can use.  The default
file is /etc/lvm/devices/system.devices, and the lvmdevices(8)
command is used to add or remove device entries.  If the file
does not exist, or if lvm.conf includes use_devicesfile=0, then
lvm will not use a devices file.  When the devices file is in use,
the regex filter is not used, and the filter settings in lvm.conf
or on the command line are ignored.

LVM records devices in the devices file using hardware-specific
IDs, such as the WWID, and attempts to use subsystem-specific
IDs for virtual device types.  These device IDs are also written
in the VG metadata.  When no hardware or virtual ID is available,
lvm falls back using the unstable device name as the device ID.
When devnames are used, lvm performs extra scanning to find
devices if their devname changes, e.g. after reboot.

When proper device IDs are used, an lvm command will not look
at devices outside the devices file, but when devnames are used
as a fallback, lvm will scan devices outside the devices file
to locate PVs on renamed devices.  A config setting
search_for_devnames can be used to control the scanning for
renamed devname entries.

Related to the devices file, the new command option
--devices <devnames> allows a list of devices to be specified for
the command to use, overriding the devices file.  The listed
devices act as a sort of devices file in terms of limiting which
devices lvm will see and use.  Devices that are not listed will
appear to be missing to the lvm command.

Multiple devices files can be kept in /etc/lvm/devices, which
allows lvm to be used with different sets of devices, e.g.
system devices do not need to be exposed to a specific application,
and the application can use lvm on its own set of devices that are
not exposed to the system.  The option --devicesfile <filename> is
used to select the devices file to use with the command.  Without
the option set, the default system devices file is used.

Setting --devicesfile "" causes lvm to not use a devices file.

An existing, empty devices file means lvm will see no devices.

The new command vgimportdevices adds PVs from a VG to the devices
file and updates the VG metadata to include the device IDs.
vgimportdevices -a will import all VGs into the system devices file.

LVM commands run by dmeventd not use a devices file by default,
and will look at all devices on the system.  A devices file can
be created for dmeventd (/etc/lvm/devices/dmeventd.devices)  If
this file exists, lvm commands run by dmeventd will use it.

Internal implementaion:

- device_ids_read - read the devices file
  . add struct dev_use (du) to cmd->use_devices for each devices file entry
- dev_cache_scan - get /dev entries
  . add struct device (dev) to dev_cache for each device on the system
- device_ids_match - match devices file entries to /dev entries
  . match each du on cmd->use_devices to a dev in dev_cache, using device ID
  . on match, set du->dev, dev->id, dev->flags MATCHED_USE_ID
- label_scan - read lvm headers and metadata from devices
  . filters are applied, those that do not need data from the device
  . filter-deviceid skips devs without MATCHED_USE_ID, i.e.
    skips /dev entries that are not listed in the devices file
  . read lvm label from dev
  . filters are applied, those that use data from the device
  . read lvm metadata from dev
  . add info/vginfo structs for PVs/VGs (info is "lvmcache")
- device_ids_find_renamed_devs - handle devices with unstable devname ID
  where devname changed
  . this step only needed when devs do not have proper device IDs,
    and their dev names change, e.g. after reboot sdb becomes sdc.
  . detect incorrect match because PVID in the devices file entry
    does not match the PVID found when the device was read above
  . undo incorrect match between du and dev above
  . search system devices for new location of PVID
  . update devices file with new devnames for PVIDs on renamed devices
  . label_scan the renamed devs
- continue with command processing
2021-02-23 16:43:32 -06:00
David Teigland
87ee401eea md component detection changes
Move extra md component detection into the label scan phase.
It had been in set_pv_devices which was deep within the vg_read
phase, which wasn't a good place (better to detect that earlier.)
Now that pv metadata info is available in the scan phase, the pv
details (size and device_hint) can be used for extra md checking.

Use the device_hint from the pv metadata to trigger a full md
component check if the device_hint begins with /dev/md.

Stop triggering full md component checks based on missing
udev info for a dev.

Changes to tests to reflect that the code is now detecting
md components in some test case that it wasn't before.
2021-02-05 16:23:51 -06:00
Zdenek Kabelac
b4212be2e7 thin: improve 16g support for thin pool metadata
Initial support for thin-pool used slightly smaller max size 15.81GiB
for thin-pool metadata. However the real limit later settled at 15.88GiB
(difference is ~64MiB - 16448 4K blocks).

lvm2 could not simply increase the size as it has been using hard cropping
of the loaded metadata device to avoid warnings printing warning of kernel
when the size was bigger (i.e. due to bigger extent_size).

This patch adds the new lvm.conf configurable setting:
allocation/thin_pool_crop_metadata
which defaults to 0 -> no crop of metadata beyond 15.81GiB.
Only user with these sizes of metadata will be affected.

Without cropping lvm2 now limits metadata allocation size to 15.88GiB.
Any space beyond is currently not used by thin-pool target.
Even if i.e. bigger LV is used for metadata via lvconvert,
or allocated bigger because of to large extent size.

With cropping enabled (=1) lvm2 preserves the old limitation
15.81GiB and should allow to work in the evironement with
older lvm2 tools (i.e. older distribution).

Thin-pool metadata with size bigger then 15.81G is now using CROP_METADATA
flag within lvm2 metadata, so older lvm2 recognizes an
incompatible thin-pool and cannot activate such pool!

Users should use uncropped version as it is not suffering
from various issues between thin_repair results and allocated
metadata LV as thin_repair limit is 15.88GiB
Users should use cropping only when really needed!

Patch also better handles resize of thin-pool metadata and prevents resize
beoyond usable size 15.88GiB. Resize beyond 15.81GiB automatically
switches pool to no-crop version. Even with existing bigger thin-pool
metadata command 'lvextend -l+1 vg/pool_tmeta' does the change.

Patch gives better controls 'coverted' metadata LV and
reports less confusing message during conversion.

Patch set also moves the code for updating min/max into pool_manip.c
for better sharing with cache_pool code.
2021-02-01 12:06:13 +01:00
Zdenek Kabelac
1b8c6f09bc debug: show actually reason for taking this code path
Instead of not so useful backtrace, report what was the reason.
2020-10-02 21:04:16 +02:00
Zdenek Kabelac
e1af80c81c debug: drop FD from error message
Since now the error path already has device close and set -1,
there is not much in printing this info - actually shouldn't be
there at all..
2020-10-02 21:04:16 +02:00
Zdenek Kabelac
fd96f1014b gcc: zero-sized array to fexlible array C99
Switch remaining zero sized struct to flexible arrays to be C99
complient.

These simple rules should apply:

- The incomplete array type must be the last element within the structure.
- There cannot be an array of structures that contain a flexible array member.
- Structures that contain a flexible array member cannot be used as a member of another structure.
- The structure must contain at least one named member in addition to the flexible array member.

Although some of the code pieces should be still improved.
2020-09-01 17:57:50 +02:00
David Teigland
cc4051eec0 pass cmd struct through more functions
no functional change
2020-04-21 10:58:05 -05:00
David Teigland
d9e8895a96 Allow dm-integrity to be used for raid images
dm-integrity stores checksums of the data written to an
LV, and returns an error if data read from the LV does
not match the previously saved checksum.  When used on
raid images, dm-raid will correct the error by reading
the block from another image, and the device user sees
no error.  The integrity metadata (checksums) are stored
on an internal LV allocated by lvm for each linear image.
The internal LV is allocated on the same PV as the image.

Create a raid LV with an integrity layer over each
raid image (for raid levels 1,4,5,6,10):

lvcreate --type raidN --raidintegrity y [options]

Add an integrity layer to images of an existing raid LV:

lvconvert --raidintegrity y LV

Remove the integrity layer from images of a raid LV:

lvconvert --raidintegrity n LV

Settings

Use --raidintegritymode journal|bitmap (journal is default)
to configure the method used by dm-integrity to ensure
crash consistency.

Initialization

When integrity is added to an LV, the kernel needs to
initialize the integrity metadata/checksums for all blocks
in the LV.  The data corruption checking performed by
dm-integrity will only operate on areas of the LV that
are already initialized.  The progress of integrity
initialization is reported by the "syncpercent" LV
reporting field (and under the Cpy%Sync lvs column.)

Example: create a raid1 LV with integrity:

$ lvcreate --type raid1 -m1 --raidintegrity y -n rr -L1G foo
  Creating integrity metadata LV rr_rimage_0_imeta with size 12.00 MiB.
  Logical volume "rr_rimage_0_imeta" created.
  Creating integrity metadata LV rr_rimage_1_imeta with size 12.00 MiB.
  Logical volume "rr_rimage_1_imeta" created.
  Logical volume "rr" created.
$ lvs -a foo
  LV                  VG  Attr       LSize  Origin              Cpy%Sync
  rr                  foo rwi-a-r---  1.00g                     4.93
  [rr_rimage_0]       foo gwi-aor---  1.00g [rr_rimage_0_iorig] 41.02
  [rr_rimage_0_imeta] foo ewi-ao---- 12.00m
  [rr_rimage_0_iorig] foo -wi-ao----  1.00g
  [rr_rimage_1]       foo gwi-aor---  1.00g [rr_rimage_1_iorig] 39.45
  [rr_rimage_1_imeta] foo ewi-ao---- 12.00m
  [rr_rimage_1_iorig] foo -wi-ao----  1.00g
  [rr_rmeta_0]        foo ewi-aor---  4.00m
  [rr_rmeta_1]        foo ewi-aor---  4.00m
2020-04-15 12:10:32 -05:00