IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
The process of using persistent reservations for recovery:
host A owns a lock
host A fails
host B requests the lock
host B request fails because A owns the lock
host A enters the FAIL state in sanlock
host B retries the lock, and sees owner A is failed
host B runs lvmpersist to remove the PR key of host A
host B tells sanlock that host A is dead
host B retries the lock, which is now granted by sanlock
The new option: --setlockargs persist,notimeout
adds "persist" and "notimeout" components to the
lock_args string in the VG metadata, viewable with
vgs -o+lockargs. The option is available in vgcreate
or vgchange.
. "persist" tells lvmlockd to remove the PR key of
a failed host, by running lvmpersist, and set the
sanlock host state to "dead".
. "notimeout" tells lvmlockd to configure sanlock
leases to not time out. sanlock does not use the
watchdog to protect leases that do not time out.
With this combination, PR removal replaces the watchdog
for fencing hosts with expired leases that are blocking
other hosts.
The lock_args version component is "2.0.0" when
these new settings are used, otherwise remains
"1.0.0". Previous lvm versions will not start a
VG with lockargs version 2.
Requires sanlock version 4.2.0 or later.
Sometimes sg_persist/mpathpersist commands will fail if the
device returns a Unit Attention, e.g. if the host's last
registration was cleared. Run sg_turs on each scsi/mpath
device at the start of each command to clear any UA errors.
This is simpler than adding code to retry commands throughout
the script if they happen to hit a UA error.
Add comprehensive validation to ensure args.h option definitions follow
the established sorting rules:
1. Options without short options - sorted alphabetically
2. Alias options (without descriptions) - sorted alphabetically
3. Options with short options - sorted by uppercase symbol with
lowercase variants first, then by long option name
Changes include:
- Enhanced validation logic in _find_lvm_command_enum() to check args.h
ordering during MAN_PAGE_GENERATOR builds
- Improved error messages showing which entries violate sorting rules
- Added _sort_opt_args() function to sort optional/required option args
for consistent man page and help generation output
- Fixed validation flag handling to properly report multiple errors
This ensures the args.h reorganization from the previous commit is
maintained and prevents future ordering regressions that could affect
code maintainability and generated documentation consistency.
tools: reorganize args.h argument definitions by sorting rules
Reorganize argument definitions in args.h to follow consistent sorting:
1. Options without short options - sorted alphabetically
2. Alias options (unadvertised, no description)
3. Options with short options - sorted by uppercase symbol
with lowercase variants first, then by long option name
This improves code organization and maintainability by establishing
a clear, documented ordering for LVM command line argument definitions.
No functional changes - only reordering of existing argument definitions.
Also note there should never be a command using 2 options with the same
short option symbol
Previously, setpersist was only supported in vgchange
on an existing VG. The PR is acquired exclusively before
the devices are modified, and in the case of a shared
VG the PR is subsequently changed to a shared mode.
adding a new helper function to create key file path.
Including vgid protects against cases of reading a
stale key file that was left over from a previous VG
with the same name.
vgremove should check for keys registered by other hosts
before removing the VG to avoid leaving dangling PR keys
on devices. This involves refactoring the related commit
ca6fe99162 "lvmpersist: fix vgremove when another key is registered"
that separated persist_stop into before and after parts.
When an XFS file system was previously mounted with quota mount options
(combination of -o uquota,gquota,pgquota) and then we are mounting the
file system as part of the lvresize/lvextend operation (through the fs
resize helper script), we need to preserve the quota mount options.
Otherwise, the XFS would need to recheck quotas - in that case the kernel
log contains:
XFS (<device>): Quotacheck needed: Please wait.
This may take a long time, depending on the file system size.
Related issue: https://github.com/lvmteam/lvm2/issues/182
Previously this was hard-coded to: "Autoactivation commands use a number
of temp files in /run/lvm (with the expectation that /run is cleared
between boots.)"
Since c1bfc8737f it was made more generic,
but on some systems this logic leads to "Autoactivation commands use a
number of temp files in /run/lvm (with the expectation that /var/run
is cleared between boots)." which I'd say adds more confusion than it
solves.
As man pages now tend to print options as man macro reference,
we may still need some local 'specialization'.
ATM --persist option is known expection where command may accept only
certain argument to be recognized/allowed - so generic String printed
in Italic can be replaced with just specific argument printed in Bold.
TODO: recognize only cases where the 'generic' String is NOT used and
for rest of them use the common O_persist macro reference.
The ls struct was being freed after ls thread exit for common stop,
but was missing when the ls thread was stopped for drop and rename.
Also free ls->actions structs in case any still exist.
Replace %4095c sscanf format specifier with position-based copying.
The _stats_parse_list_region() function previously used %4095c to read
the remaining part of the input buffer. Recent glibc versions
(2.42.9000-3.fc44) have changed the behavior of this format
specifier to align with C standard specifications, which affects
parsing when the buffer contains fewer than 4095 characters.
To ensure compatibility across different glibc versions, switch to
using %n to track the parsing position and manually copy the string
data using dm_strncpy().
Note: Binaries compiled with the previous implementation may experience
compatibility issues with newer glibc versions.
Remove per-thread grace_mutex and use existing global_mutex
with per-thread condition variables.
We separate take of grace mutex there was slight race chance
of hitting wait condition while 2nd. thread already signaled wakeup.
With the use of global mutex (which must be held before and after
anyway), there is no such race chance even possible.
With old systems we cannot always check /dev/mapper/vg-lv name,
as those might be symlinks managed by udev. However monitoring
is not 'synchronized' with udev as dmventd API is major:minor
based and does not needed symlinks. So to 'compensate' this
and make just 'old system' working - just check also for
presence of /dev/dm-X device.
Improve _get_device_inode() function with kernel version check
For kernel >=3: Use sysfs path format (/sys/dev/block/major:minor).
For kernel <3: Use /dev/mapper device path format.
Add explicit handling for very old kernels (<3.x) that don't change
inode numbers for /sys device-mapper devices.
For vgremove, split persist stop steps into:
1. prepare, which gets the pr key and VG devices,
and happens before the normal vgremove.
2. run, which removes pr key from the VG devices,
and happens after the normal vgremove.
This is necessary if another PR key is registered on the device
(which likely doesn't happen in the normal usage pattern, but is
still possible.)
When another key exists, removing the local PR key will cause the
machine to lose the ability to write VG metadata to the device.
So, the removal of the local PR key must happen after the VG metadata
is written for vgremove. The prepare step must remain at the beginning
of the command, while the list of PVs in the VG still exists.
Unlock devices file at the end of cmd processing, after all the PVs are
processed, not after the first processed PV.
Before this patch:
❯ pvchange -u /dev/sda /dev/sdb
Physical volume "/dev/sda" changed
WARNING: Devices file unlock no fd.
Physical volume "/dev/sdb" changed
2 physical volumes changed / 0 physical volumes not changed
With this patch applied:
❯ pvchange -u /dev/sda /dev/sdb
Physical volume "/dev/sda" changed
Physical volume "/dev/sdb" changed
2 physical volumes changed / 0 physical volumes not changed
The lvm_run_command/devices_file_exit/unlock_devices_file will
do the unlock at the very end of cmd processing.
Fix several problems in recent dmeventd grace period enhancement:
1. **Timer reset bug**: Threads exiting grace period were starting
time period with rather random time 'offset' causing change
in behavior and firing monitor action in non-deterministic time.
Reset timer (next_time = current_time + timeout) AFTER grace
period wait.
2. **Device identity verification**: Add inode tracking to prevent
incorrect thread reuse when device UUID is recycled. Grace period
thread lookup now verifies device inode matches to ensure same
physical device. This avoid reusing of monitored values for
a different device.
'diskseq' was also considered as unique identifier, but inode
check seems to be easier to check and is not 'so new'.
Also another solution could have been to change dmeventd protocol,
and introduce supend/resume calls - but this could be another source
of problem.
3. **Signal handling order**: Move SIGALRM reset to occur BEFORE
entering grace period instead of after. This prevents potential
error path trouble that may have shutdown reused thread causing
the actual monitoring to be silently lost.
4. **Event state cleanup**: Clear current_events before grace period
to prevent stale event processing after thread reuse.
Note: wondering if there can be any problem with inode check....
vgchange --persist stop requires locking to first be stopped
(vgchange --lockstop). Allow setting --lockopt force to bypass
this check so that PR can be forcibly stopped.
Fix commit b2bc06caf8
"lvmpersist: check devices support PR before certain commands"
which broke clear and read commands. Change to skip individual
devices in clear/read if they don't support PR.
The pv/vg/lvdisplay (without -C|--columns) does not use the reporting
mode for the output. However, we can still allow the --reportformat
option for these commands, but it will affect only the log report,
like we have for other non-reporting lvm commands
(e.g. lvchange,vgchange...).
Just like process_each_vg and process_each_lv, the top-level process_each_pv
should also initialize the processing handle (if the handle not already passed
as argument for use). This is mainly important, among others, for proper
log_report tracking and proper output formatting.
For example:
Before this patch - there are two "log" sections as the processing handle is
initialized dynamically more times if not passed explicitly from the top-level
process_each_pv function:
❯ pvdisplay --config 'log/report_command_log=1 report/output_format=json'
{
"log": [
...
]
}
{
"log": [
...
]
}
❯ pvdisplay --config 'log/report_command_log=1 report/output_format=json' | jsonlint
<stdin>:17:2: Error: Unexpected text after end of JSON value
| At line 17, column 2, offset 4463
<stdin>: has errors
With this patch applied:
❯ pvdisplay --config 'log/report_command_log=1 report/output_format=json'
{
"log": [
...
]
}
❯ pvdisplay --config 'log/report_command_log=1 report/output_format=json' | jsonlint
<stdin>: ok
--configreport, --logonly, --reportformat and --select options can be
used only if -C|--columns is used at the same time. These options are
only to control the reporting infrastructure. We switch to reporting
mode using that -C|--columns switch.
Using the options for reporting mode if reporting is not initiated
(using the -C|--columns switch) can cause various issues and
inconsistencies in the output.
See also https://github.com/lvmteam/lvm2/issues/144.
When cachevol was used for the cache LV, the check for not splitting an
LV between two VGs was incorrect, not allowing to split the VG in cases
it should have been possible.
For example, splitting a VG which contains the cache LV with cachevol
only on two devices (sda and sdb here) and leaving third device completely
unused (sdc here) should clearly allow us to split it into a new VG:
❯ vgcreate vg /dev/sd{a..c}
Physical volume "/dev/sda" successfully created.
Physical volume "/dev/sdb" successfully created.
Physical volume "/dev/sdc" successfully created.
Volume group "vg" successfully created
❯ lvcreate -l2 -n main vg /dev/sda
Logical volume "main" created.
❯ lvcreate -l2 -n fast vg /dev/sdb
Logical volume "fast" created.
❯ lvconvert -y --type cache --cachevol fast vg/main
Logical volume vg/main is now cached.
❯ lvs -a -o name,devices vg
lv_name devices
[fast_cvol] /dev/sdb(0)
main main_corig(0)
[main_corig] /dev/sda(0)
❯ lsblk -o name /dev/sd{a..c}
NAME
sda
└─vg-main_corig
└─vg-main
sdb
└─vg-fast_cvol
├─vg-fast_cvol-cdata
│ └─vg-main
└─vg-fast_cvol-cmeta
└─vg-main
sdc
Before this patch:
❯ vgsplit vg vg2 /dev/sdc
Logical volume vg/main must be inactive.
❯ vgchange -an vg
0 logical volume(s) in volume group "vg" now active
❯ vgsplit vg vg2 /dev/sdc
Can't split LV main between two Volume Groups
With this patch applied:
❯ vgsplit vg vg2 /dev/sdc
New volume group "vg2" successfully split from "vg"
The test shell/integrity.sh creates raid arrays, corrupts one of the
legs, then reads the array and verifies that the corruption was
corrected. Finally, the test tests that the number of mismatches on the
corrupted leg is non-zero.
The problem is that the raid1 implementation may freely choose which leg
to read from. If it chooses to read the non-corrupted leg, the corruption
is not detected, the number of mismatches is not incremented and the test
reports this as failure.
Fix this failure by marking the non-corrupted leg as "writemostly", so
that the kernel doesn't try to read it (it reads it only if it finds
corruption on the other leg).
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
The test shell/integrity.sh creates raid arrays, corrupts one of the
legs, then reads the array and verifies that the corruption was
corrected. Finally, the test tests that the number of mismatches on the
corrupted leg is non-zero.
The problem is that the raid1 implementation may freely choose which leg
to read from. If it chooses to read from the non-corrupted leg, the
corruption is not detected, the number of mismatches is not incremented
and the test reports this as a failure.
Fix the test by not checking the number of integrity mismatches for
raid1.
Fix check for not splitting an LV between two VGs in case
where the LVs contains an internal layer.
For example, integrity layer for RAIDs and splitting a PV that
is not part of the RAID LV at all (sdc here):
❯ vgcreate vg /dev/sda /dev/sdb /dev/sdc
Volume group "vg" successfully created
❯ lvcreate -l1 -m1 --type raid1 --raidintegrity y vg /dev/sda /dev/sdb
Logical volume "lvol0" created.
Before this patch:
❯ vgsplit vg vg2 /dev/sdc
Logical volume vg/lvol0_rimage_0 (part of lvol0) must be inactive.
❯ vgchange -an vg
0 logical volume(s) in volume group "vg" now active
❯ vgsplit vg vg2 /dev/sdc
Can't split LV lvol0_rimage_0 between two Volume Groups
With this patch applied:
❯ vgsplit vg vg2 /dev/sdc
New volume group "vg2" successfully split from "vg"
For our tests we expect very fast reaction.
So to keep i.e. reporting for the test 'lvextend-thin-metadata-dmeventd'
fast use just 2 seconds grace period (thread reuse).
Add thread reuse mechanism to reduce overhead when devices are frequently
registered/unregistered by introducing a configurable grace period where
monitoring threads wait for potential reuse before termination.
Key changes:
- Add -g option to configure grace period (0-300 seconds, default: 10s)
- Introduce DM_THREAD_GRACE_PERIOD state for threads awaiting reuse
- Implement thread reuse for matching device/dso combinations
- Add pthread condition variables and mutexes for grace period synchronization
- Add thread usage counter and enhanced debug logging
- Add _reset_pending_signal() to handle SIGALRM cleanup on thread reuse
- Refactor _monitor_thread() to support grace period workflow
- Update timeout thread to skip non-running threads
- Simplify thin plugin logging
This optimization significantly reduces thread creation/destruction overhead
in scenarios with rapid device registration/unregistration cycles, such as
creating snapshots of thin volumes, while maintaining proper cleanup and
thread safety.
No longer returning EINVAL when the dmeventd is not monitoring
any device (there are no registered devices).
This makes usable i.e. 'dmeventd -R' for the case, the dmeventd
was not monnitoring anything during this call.
Previously this actually cause refuse of 'restart' - since
restarting 'dmeventd' has seen failing resutl of a call:
DM_EVENT_CMD_GET_STATUS
Common vgextend commands were logging a message about
dev_read_reservation when PR was not being used.
Only attempt PR work from vgextend when either the
PR require or autostart settings are enabled on the VG.
Add device existence check after successful wait ioctl to prevent
processing events on removed devices.
When dm_task_run() succeeds on a WAITEVENT ioctl, the success might
actually indicate that the device was removed rather than a genuine
device event. Without this check, dmeventd would attempt to process
ERROR events on non-existent devices, leading to unnecessary error
commands and log noise.
This fix adds a _fill_device_data() call immediately after successful
wait completion to verify the device still exists. If the device is
gone, execution jumps to the existing ENXIO error path which properly
handles device disappearance.
Note: Future kernel versions may return ENXIO directly from the wait
ioctl when devices are removed, making this workaround unnecessary.
Until then, this extra INFO ioctl provides the needed verification.
Check first whether the monitored device actually really exists,
before resolving its device name.
In the case device is not present in DM table, fail _fill_device_data().
Break down complex one-liner directory check into explicit steps with
clear error messages and comments.
The original code used a dense subshell expression combining multiple
tests with logical operators, making it difficult to understand and
maintain. This refactoring separates the logic into:
1. Check directory accessibility (read/write/execute permissions)
2. Check if directory is empty (including hidden files)
Each check now has a specific error message, making it easier for users
to understand what went wrong. The functionality remains identical, but
the code is now more maintainable and debuggable.
Changes:
- Split accessibility and emptiness checks into separate if statements
- Add descriptive comments explaining each validation step
- Provide specific error messages for different failure conditions
- Maintain original behavior including dotglob/nullglob handling
Replace size_t with uint64_t in struct vdo_header to ensure consistent
64-bit size field across all architectures.
On 32-bit platforms, size_t is only 4 bytes, which causes incorrect
structure layout when reading VDO metadata that expects a 64-bit size
field. This fix ensures the VDO header structure maintains the same
binary layout regardless of target architecture.
While 32-bit architectures are not officially supported for VDO,
this change improves code correctness and prevents potential issues
during cross-compilation or when building on 32-bit development
environments.
If we have LV->crypt->FS stack, check that the adjustment for the crypt
data offset will left space for the crypt data itself. Fix possible underflow.
Example:
❯ lvs -o name,size vg/lvol0
lv_name lv_size
lvol0 124.00m
❯ lsblk /dev/vg/lvol0
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
vg-lvol0 252:2 0 124M 0 lvm
└─a 252:4 0 108M 0 crypt
Before this patch (the incorrect resulting underflowed file system size after adjustment
for crypt data offset, crypt data then severed by the LV resize without proper
crypt resize beforehand):
❯ lvreduce --yes --resizefs -L -123M vg/lvol0
Rounding size to boundary between physical extents: 120.00 MiB.
Checking crypt device /dev/dm-4 on LV vg/lvol0.
File system size 18446744073696968704b is adjusted for crypt data offset 16777216b.
File system ext4+crypto_LUKS found on vg/lvol0.
File system size (108.00 MiB) is smaller than the requested size (<16.00 EiB).
File system reduce is not needed, skipping.
crypt device is already reduced to 113246208 bytes.
Size of logical volume vg/lvol0 changed from 124.00 MiB (31 extents) to 4.00 MiB (1 extents).
Logical volume vg/lvol0 successfully resized.
With this patch applied:
❯ lvreduce --yes --resizefs -L -123M vg/lvol0
Rounding size to boundary between physical extents: 120.00 MiB.
Checking crypt device /dev/dm-4 on LV vg/lvol0.
Crypt header requires 16.00 MiB, not enough space left for crypt data.
Use correct path - if the system does not have installed lvm udev rules
these /dev/vgname/lvname symlinks are not created - but test
running in test's dev dir gets symlinks created by lvm2 itself.
Use slightly bigger PVs, so the raid takes a bit more time to fully
reshape arrays - so we avoid conversion to be finished to fast even
with slowered devices.
Grab blockdev size before calling 'lvs' command.
We have different behavior for lvreduce (contains the fs size
pre-check before calling the external script) and lvextend
(does not ave the pre-check).
Also, there's different behavior with "lvreduce --fs resize"
(contains the pre-check) and "lvreduce --fs resize_fsadm"
(does not have the pre-check).
Obvously, only "lvreduce --fs resize" does have the pre-check.
To make it consitent all across, we should remove that one
and everybody will be happy.
Kernel without commit kernel commit 9f346f7d4ea73692b82f5102ca8698e4040469ea
cannot reasize raid LV without failing the raid itself.
So skip some tests for kernels 6.13 6.13 6.15.
TODO: maybe we want to print some warning message to the users
with affected kernels ??
With the use of --noudevsync we actually need to also ensure
we verify link create - as without synchronization we are not
able to properly wait - so lvm2 needs to create links itself.
It's hack where it would be better to not use it - but so far
there is no easy fix.
Make skip variables easily overridable by setting make vars.
Also further reduce used PV sizes.
Fix the usage of delay_dev so the test is properly executed
and not skiped. For major mirror slowdown use smalled region
sizes that are cause way more frequent commits so we can go
with significantly smaller delays.
Also check repair to work for failing mirror leg and mirror log.
Improve the RAID reshape size test script with the following changes:
**Code Quality Improvements:**
- Add proper shell quoting throughout the script to prevent word splitting issues
- Replace manual arithmetic with cleaner shell arithmetic syntax
- Improve variable handling and remove unnecessary local variable assignments
- Fix typo: "hilesystem" -> "filesystem"
**Test Reliability Enhancements:**
- Add EXTENSIVE_FSCK environment variable for optional additional filesystem validation
- Reduce delay times from 40ms/25ms to 20ms for all RAID types to speed up testing
- Add helper functions _delay_dev() and _restore_dev() for cleaner device delay management
- Use --noudevsync flag in lvconvert to avoid udev-related timing issues
- Remove unnecessary sleep calls and udevadm settle commands
**Functionality Improvements:**
- Improve _check_size() function to return proper exit codes instead of echo statements
- Better error handling in conditional statements using proper test syntax
- Cleaner parameter passing using "$@" instead of manual argument handling
- More robust device path handling using $DM_DEV_DIR consistently
**Code Structure:**
- Extract device delay logic into reusable helper functions
- Improve readability with better variable naming and consistent formatting
- Add explanatory comments for complex operations
_check_size_timeout function added compensating the block layer bdev update race
thus avoiding other explicit sleeps spread in the test code.
"udevadm settle" approach gone.
No --noudevsync mandatory.
Size check for stripe size reshape was bogus.
Whilst on it, optimize delay on test device to
avoid delying LVM2 MDA and RAID rmeta SubLV.
Also update some comments.
Adapting the test for the changed behavior of lvconvert
where the mirror leg and log count is not changing.
Properly test for this condition - checking only leg count is
not enough to expect error return code.
And finaly fixing invalid bash scripting logic since:
test && TRUE || FALSE
is not bash equivalent of:
if test ; then
TRUE
else
FALSE
fi
Improve the consistency and readability of warning messages.
Capitalize the first word of all warning messages.
Add periods at the end of warning messages.
The changes are purely cosmetic and do not affect functionality.
Some a/an fixups along with a couple more odds and ends.
Hopefully this is useful like this as plain diff output
or let me know if something else will work better.
Simplify printf format strings by removing unnecessary parameters
in libdm-report and toollib components to resolve coverity warnings
about format string mismatches.
Add coverity annotations to suppress false positive warnings for
unimportant results across multiple files including dmeventd,
lvmlockd, display, logging, dmsetup, and man-generator components.
Replace all uses of Linux kernel-style endian conversion macros/functions
(e.g., le32_to_cpu, cpu_to_le32, xlate32, xlate64, etc.) with the standard
POSIX/glibc macros (e.g., le32toh, htole32, htobe32, be32toh, etc.) from
<endian.h>.
- Update all code to use le16toh, le32toh, le64toh, htole16, htole32, htole64,
htobe16, htobe32, htobe64, be16toh, be32toh, be64toh as appropriate.
- Provide fallback macro definitions in xlate.h for systems lacking these
standard macros, ensuring backward compatibility with older glibc and non-glibc
systems.
- Remove or replace all project-specific xlateXX and cpu_to_leXX/cpu_to_beXX
macros.
- No functional change intended; this is a mechanical, treewide modernization
for clarity, portability, and future maintainability.
To have the very same matching logic to raid1 commit:
c901528053
we add similar check for mirror where we check if mirror leg and
log count is not changing which will now return also an error.
Set LVM_DID_EXEC to "1" instead of using the command name string,
avoiding potential issues with unusual command names and improving
consistency in environment variable handling.
Add arg_force_value() function that returns the correct force_t enum
type, replacing direct string comparisons. Update lvconvert and
pvremove to use this new function for better type safety.
This is cleaner solution over just plain cast to force_t as we can
validate force level in use.
Replace custom binary search implementation with the standard library's
bsearch() function for better maintainability.
Also convert command_name from pointer to char array and simplify name
ordering validation logic.
Fix inconsistent spacing in command help output by introducing
_print_opt_with_align() function that properly handles alignment
for options with and without short forms. This resolves the extra
spaces that were being printed in --longhelp output.
Fixes regression introduced in commit:
491c6652ae.
Replace direct structure assignment with explicit copying to prevent
potential undefined behavior from structure aliasing. This ensures
proper memory handling when working with sanlock structures.
This possibly fixes regression introduced with commit: e9640e5178
as unintended side effect.
We already check loop devices for LVM_LOOP_PV_ACTIVATED="1" in udev
rules to see if have executed pvscan before. If that is the case, we
don't want to execute it again to avoid VG reactivation.
However, the rules missed the IMPORT{db}="LVM_LOOP_PV_ACTIVATED" rule
to actually get the value already stored in udev db from previous event.
As a result, the pvscan executed on each CHANGE udev event, hence the
VG autoactivation triggered each time as well.
Fix this by adding the missing IMPORT{db}="LVM_LOOP_PV_ACTIVATED" rule
(just like we already do for MD devices).
Note: Keep the behavior for ADD events. That is, we still want the
autoactivation to trigger each time, otherwise coldplug will not work
(again, we have the same principle used for MD devices).
In case a Raid1LV has e.g. 3 images already, running "lvconvert --mirrors 2 $Raid1LV"
results in success even though the image count didn't change.
Make it fail in such case.
Related issue: https://issues.redhat.com/browse/RHEL-82138
Add explicit type casts to resolve GCC signness comparison warnings:
- Cast dest_size to int in lvmlockctl.c
- Cast NVME_IDENTIFY_DATA_SIZE to int in nvme.c
- Cast bitwise AND results to int in nvme.c and persist.c
- Cast DM_STATS_GROUP_NOT_PRESENT to int in libdm-stats.c
This commit fixes C90 compatibility issues by ensuring variable declarations
are placed at the beginning of code blocks before any executable statements.
Change kernel_send() function signature from accepting struct dm_ulog_request*
to void* to properly handle variable-sized data access.
The dm_ulog_request structure contains a flexible array member (data[0])
that allows variable-sized payloads to be appended. When accessing data
beyond the base structure size, Coverity correctly flags this as a
potential buffer overrun since the structure size doesn't account for
the variable data.
By using void* as the parameter type, we make it explicit that we're
working with a memory region that may extend beyond the base structure,
eliminating the false positive while maintaining type safety through
proper casting within the function.
Replace malloc() with calloc() when allocating the string pointer array
for regex pattern matching in _create_field_selection(). This ensures
all array elements are initialized to NULL pointers, preventing potential
use of uninitialized memory objected by Coverity.
Existing code sets all elements in the follow up loop.
Add explicit (void) cast to ignore the return value of _lv_types_match()
in _check_lv_rules() function. This addresses Coverity warning about
unused return values and maintains consistency with other similar
function calls in the codebase that properly handle return values.
Add Coverity annotations to suppress false positive warnings in several
files where the static analysis tool incorrectly flags potential issues
that are actually safe due to proper validation or intentional behavior.
The annotations address the following false positives:
- daemons/dmeventd/dmeventd.c: overflow_sink warnings for 'current' variable
that is validated to be positive before use in buffer operations
- daemons/lvmlockd/lvmlockd-core.c: overflow_sink warning for 'ret' variable
that is validated to be positive before use
- lib/config/config.c: overflow_sink warning for 'sz' variable that is
validated to be positive before use in read operations
- libdm/dm-tools/dmsetup.c: overflow_sink, overflow, and deref_overflow
warnings for 'n' variable that is validated to be positive before use
in buffer operations and string termination
- libdm/libdm-stats.c: overflow_sink warning for 'i & j' variables that are
validated to be positive before use in array indexing
Introduce _mda_is_ignored() and _rlocn_is_ignored() wrapper functions
with warn_unused_result attribute to enforce return value checking.
This follows the established pattern used by dm_strncpy() and _dm_strncpy():
- Functions without underscore prefix can be used without checking return values
- Functions with underscore prefix must have their return values checked
The change improves static analysis coverage by ensuring that critical
metadata area and raw location ignored state checks are properly validated,
reducing the risk of unhandled error conditions in metadata processing.
- Fix potential resource leak by ensuring va_end() is called on error path
- Set matched = -1 and break from while loop instead of immediate return
- This ensures proper cleanup of va_list when unsupported format
specifiers are encountered in lvmlockctl.
The szscanf function is a custom string scanner used for parsing lvmlockd
status information. Previously, encountering an unsupported format
specifier would cause an immediate return without calling va_end(),
leading to undefined behavior and potential resource leaks.
This fix ensures proper cleanup of variable argument lists in all code paths.
Replace potentially unsafe strcpy() with bounds-checked memcpy()
and proper null termination in the _sysfs_get_dm_name() function.
This prevents buffer overflow vulnerabilities when copying device
mapper names from sysfs.
Also replace another strcpy() with _dm_strncpy() which also checks
whether copied string with into a given buffer in the function
_sysfs_get_kernel_name().
Changes:
- Replace strcpy(buf, temp_buf) with memcpy(buf, temp_buf, len) + buf[len] = '\0'
- Use stack-allocated buffers instead of malloc/dm_malloc for better performance
- Improve error handling logic for ENOENT vs other errors
- Add proper newline stripping with bounds checking
- Remove memory allocation failure paths and cleanup code
This addresses potential security issues identified by static analysis
tools while also improving performance by avoiding dynamic memory allocation.
- Add bounds checking for sysconf(_SC_PAGESIZE) return value
- Validate page size is positive and reasonable (< 16MB)
- Use size_t for page_size variable to match usage context
- Simplify page calculation with ceiling division
- Fix potential integer overflow in page count calculation
This improves robustness when handling edge cases in clustered
mirror log disk operations.
Use signed int instead of size_t for loop counter to prevent reporting
unsigned underflow when decrementing from 0 in Coverity.
Loops itself worked just fine.
The _count variable was declared as uint64_t but used in arithmetic operations
that could result in underflow when subtracting from smaller values. This
could cause issues when calculating interval numbers or handling count-based
reporting.
Changes:
- Change _count variable type from uint64_t to int64_t
- Update _interval_num() to use proper casting for arithmetic
- Change UINT64_MAX to INT64_MAX for default count value
- Remove unnecessary casting in count assignment
This prevents potential underflow issues when _count is decremented or used
in subtraction operations, ensuring proper behavior for interval-based
reporting and count tracking in dmsetup commands.
The fix maintains compatibility while providing safer integer arithmetic
for the reporting loop logic.
The _stats_map_extents() function processes file extents returned
by FIEMAP ioctl calls. When handling the case where a file has
only a single extent, the code accesses fm_ext[i - 1] to check
if the logical offset is 0.
However, when i is 0 (no extents processed yet), this results in a negative
array index access which can cause undefined behavior or crashes.
So check early whether there are fm_mapped_extents to process.
This avoids using negative index array.
Existing code already checks fm_mapped_extents == 0 before calling
this function so the patch is not fixing any real bug.
Store vg->lock_type in a local variable to avoid repeated null checks
and string comparisons throughout the function. This improves code
readability and eliminates redundant conditional evaluations.
Change return value semantics:
- Previously returned -1 for non-mirrored segments (error condition)
- Now returns 0 for non-mirrored segments (no failed mirrors) and
reports INTERNAL_ERROR for such case.
- This makes the function more consistent as it counts failures, not errors
Simplify recursive calls:
- Remove unnecessary intermediate variable 'r' in both functions
- Directly add recursive call results to return value
- Eliminates redundant error checking for recursive calls
Code cleanup:
- Remove unused variable declarations
- Improve code readability and reduce compiler warnings
The refactoring makes the functions more robust and easier to understand
while preserving their core behavior of counting failed mirror images
and logs across logical volume segments.
The stats implementation was using uint64_t for group_id and loop variables
while calling dm_bit_* functions that return int types. This created a
potential issue where large values could be incorrectly handled due to
implicit casting between signed and unsigned types.
Changes:
- Change loop variables from uint64_t to int in _stats_group_tag_len()
- Change loop variable from uint64_t to int in _stats_clear_group_regions()
- Change loop variables from uint64_t to int in dm_stats_get_counter()
- Change loop variable from uint64_t to int in dm_stats_get_region_len()
- Fix _stats_create_group() to properly handle dm_bit_get_first() return value
This ensures consistent type usage and prevents potential issues with
values exceeding INT_MAX (2^31 - 1). Also this limitats group_id to 31 bits
as a constraint that may need addressing in the future.
FIXME: Maybe consider implementing 64-bit variants of dm_bit functions or
documenting this limitation more prominently is this ever become an
issue...
Function dm_strncpy() ensures the last character is \0,
so pass whole array size for buffer size.
This give 1 extra character for use to store owner state and name.
lvcreate accepts --setautoactivation for thick logical volumes, but
not for e.g. thin volumes (see linked bug report). Currently, in such
cases, a subsequent lvchange invocation is necessary to change the
autoactivation flag.
To fix this, make lvcreate accept the flag for all volumes by adding
it to LVCREATE_ARGS.
Fixes: https://gitlab.com/lvmteam/lvm2/-/issues/32
Fixes: 0a28e3c44 ("Add metadata-based autoactivation property for VG and LV")
Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
Update test infrastructure to use the lvresize_fs_helper script
from the LVM2 codebase instead of relying on the system-installed
version. This ensures consistent testing behavior and avoids
issues when the system version doesn't exist or differs from
the expected implementation.
- Add lvresize_fs_helper to LVM_SCRIPTS in test/Makefile.in
for installation
- Configure global/lvresize_fs_helper in test/lib/aux.sh
to point to the test version
- Add global_lvresize_fs_helper_executable_CFG config option to allow
specifying the path to the lvresize_fs_helper script.
- Add DEFAULT_LVRESIZE_FS_HELPER_PATH macro for default value.
- Update _get_lvresize_fs_helper_path() in lib/device/filesystem.c
to read the path from configuration using find_config_tree_str,
similar to _fsadm_cmd().
- This allows users to override the helper path via configuration,
improving flexibility and consistency with fsadm_executable handling.
- Avoid using static variable to get helper path just once, since
it may change between commands via lvm.conf in lvm2 shell.
alphasort was replaced with versionsort in 09e508cd43 commit.
This commit fixes the following compiler error.
device/device_id.c:1608:65: error: use of undeclared identifier 'versionsort'
1608 | sort_count = scandir(dirpath, &namelist, _filter_backup_files, versionsort);
| ^
= and == are equivalent in Bash for strings, but = is the only portable
operator for compatibility with other shells. Before this change,
running ./configure with Dash as /bin/sh resulted in:
./configure: 14558: test: yes: unexpected operator
and the test did not work (i.e. --enable-cmdlib --disable-shared allowed
a failed build to continue). Now, the test works in Bash and Dash.
Enable lvm to use persistent reservations on a VG, which
are applied to each PV in the VG.
. lvmpersist is a low level script, which uses commands
sg_persist, mpathpersist, and nvme to do PR operations
on devices. This script is used by higher level lvm
commands, and would not often be run by users.
. vgchange --setpersist is a VG metadata configuration command
that specifies how PR should be started and enforced for a VG
relative to other lvm commands.
. vgchange --persist is a command to change PR state of PVs in
the VG, e.g. start PR to register and reserve.
The lvmpersist man page contains a complete description.
LVM objects (like pv_device_id) can sometimes contain literal control characters. The JSON spec disallows control characters. So we need to escape them for output.
Closes https://gitlab.com/lvmteam/lvm2/-/issues/35
Update test scripts to use new command-line options for skipping tests,
replacing the previous use of environment variables. The following options
are now supported:
--skip-root-dm-check
--skip-with-devices-file
--skip-with-lvmpolld
--skip-with-lvmlockd
- Add command-line option parsing to lib/inittest.sh.
- Replace SKIP_ROOT_DM_CHECK=1 with --skip-root-dm-check.
- Replace SKIP_WITH_DEVICES_FILE=1 with --skip-with-devices-file.
- Replace SKIP_WITH_LVMPOLLD=1 with --skip-with-lvmpolld.
- Replace SKIP_WITH_LVMLOCKD=1 with --skip-with-lvmlockd.
- Update 410 test files to use the new syntax.
This change provides a cleaner, more consistent, and maintainable
way to handle test skipping across the test suite.
Also eliminates number of shellcheck errors.
TODO: convert couple more remaining.
Fix include paths in cmd_enum.h and command.c by removing the "include/"
prefix from #include directives. This resolves build failures when
building from a directory different from the source directory.
The include paths were incorrectly referencing "include/cmds.h" instead
of "cmds.h", causing compilation errors in out-of-source builds.
Remove unnecessary quotes from section headers in man pages:
- lvm_import_vdo.8_main: .SH "NAME" -> .SH NAME
- fsadm.8_main: .SH "NAME" -> .SH NAME
This improves consistency and follows man page conventions.
Originated-by: Cursor AI
Fix company name format in .TH headers by adding missing comma:
- dmeventd.8_main: 'Red Hat Inc' -> 'Red Hat, Inc.'
- cmirrord.8_main: 'Red Hat Inc' -> 'Red Hat, Inc.'
- lvmpolld.8_main: 'Red Hat Inc' -> 'Red Hat, Inc.'
- lvmdbusd.8_main: 'Red Hat Inc' -> 'Red Hat, Inc.'
- lvm_import_vdo.8_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- lvmlockctl.8_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- lvmcache.7_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- lvmvdo.7_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- blkdeactivate.8_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- lvmthin.7_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- lvmreport.7_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- lvmsadc.8_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- lvmautoactivation.7_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- lvmlockd.8_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- fsadm.8_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- lvmraid.7_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- lvmsystemid.7_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
- lvmsar.8_main: 'Red Hat, Inc' -> 'Red Hat, Inc.'
This ensures consistent company name formatting across all man pages.
Originated-by: Cursor AI
- cmirrord.8_main: Remove unnecessary article 'the' before 'corosync'
- dmsetup.8_main: Fix multiple grammar and punctuation issues
* Add missing 'to' in 'Set this to zero to continue'
* Fix 'eg,' -> 'e.g.,' and add missing comma
* Add missing 'the' in 'for the live device'
* Fix 'customised by following options' -> 'customised by the following options'
* Fix 'comma-separate' -> 'comma-separated'
- fsadm.8_main: Improve sentence structure for dm-crypt description
- lvm.8_main: Fix capitalization 'volume Groups' -> 'Volume Groups'
- lvm_import_vdo.8_main: Fix header title and multiple grammar issues
* Fix header title 'FSADM' -> 'LVM_IMPORT_VDO'
* Fix 'LV a backend device' -> 'LV as a backend device'
* Fix 'the of volume group' -> 'the name of the volume group'
* Remove extra 'with' in 'with within volume group'
- lvmsystemid.7_main: Fix punctuation 'e.g.' -> 'e.g.,' and 'i.e.' -> 'i.e.,'
These changes improve grammatical correctness, consistency, and readability.
Originated-by: Cursor AI
Override LC_NUMERIC part of the locale to "C" if we detect that the
radix character interferes with JSON_STD format. If that's the case,
override LC_NUMERIC locale to "C" in report_format_init, that is,
before any reporting is executed (including log reporting). Restore
it back in report_format_destroy, that is, once we're sure that all
reporting is finished.
Related: https://gitlab.com/lvmteam/lvm2/-/issues/33
When a report is under DM_REPORT_GROUP_JSON_STD, the formatting of the
report follows more standard form of the JSON output. This includes
unquoted numbers (as opposed to the DM_REPORT_GROUP_JSON).
The JSON standard dictates the radix character (decimal point) must
be '.' only (https://www.rfc-editor.org/rfc/rfc7158#section-6).
However, some locales may use other character for the radix delimiter,
like ','. This character also interferes with ',' used as delimiter for
json items.
Therefore, we need to check whether current locale is not posing an
issue when using DM_REPORT_GROUP_JSON_STD. If that's the case, simply
error out from dm_report_group_create as we don't want to override
current locale in libdm or do anything else at this level. The libdm
caller is responsible here for setting the proper locale.
Related: https://gitlab.com/lvmteam/lvm2/-/issues/33
This commit adds lvresize/lvextend/lvreduce support for btrfs.
'btrfs filesystem resize [devid:][+/-]<newsize>[kKmMgGtTpPeE]|[devid:]max <path>'
is used to resize one device only when it's mounted.
The code pattern is like xfs but it supports shrink.
For multi-devices btrfs, There is one difficulty to be handled:
If `lvreduce --fs resize` is given, lvm2 will check newsize vs current fs size
to judge if it's need to shrink fs or not.
For one device btrfs, fslastblock * fsblocksize/FSSIZE is the correct value like
ext* and xfs. But for multi-devices btrfs, the two values are whole fs size.
There is no other way without relying btrfs superblock parse. It's too
complicated and inproper to implemnt the logic in lvm2.
So here just sets fs_last_byte to 0 for btrfs and skips boundary check in
_fs_reduce_allow(). It's safe as btrfs will handle it well.
The another complicated part is how to get mount point info if multi-devices.
There is only one mnt entry per mounted fs in /etc/mtab even it's a
multi-devices btrfs. So we first get uuid from lv device then traverse devices
under /sys/fs/btrfs/$uuid/devices and compare them to the mnt entry to get the
mount point.
Signed-off-by: Su Yue <glass.su@suse.com>
Add new field fs_info::uuid to record device uuid when calling
fs_get_blkid() for further use.
No functional change.
Signed-off-by: Su Yue <glass.su@suse.com>
As we always require and check for version 3.7,
avoid extra CHECK_EXIST and go for CHECK_MODULE.
LOCKDSANLOCK_SUPPORT is not defined is the build
is not enabled.
When build for sanlock is enabled, and
CHEKC_MODULE does not detect at least version 3.7
then whole configure process errors out.
Count or clear transiently failed devices as of dm-raid superblocks.
Updated debuging.
Use lvconvert --repair to repair transiently failed legs.
Activating all 'meta' LVs with single sync_local_dev_names().
Using proper DM path for meta LV.
Modified-by: zkabelac@redhat.com
Use the new sanlock_acquire2() which returns info about the owner
of a lease. Pass this info back to the lvm command, where it's
initially used to print the host_id of a host holding a lock
when it cannot be acquired.
Add szscanf() to use in place of sscanf. It takes a buffer size for
strings, so avoids needing to use max field width, which is hard to
read when implemented with stringify macros.
Manually enhance pages for thin, cache, raid, vdo.
Replace usage of .HP with .TP when it makes sense (but keep .HP
where so far we don't have replacement giving same visual results).
Use .CMS, .CME macros in dmsetup/dmstats so it easy to switch
But keep using .HP when the rendering looks simply better in terminal
although HTML output does not looks that well - so we may eventually
switch here to .TP.
For basic command synopsis use .NSY macro that will
use .SY for graphical rendering (postscript/pdf) but keep
using .TP for ASCII terminal output, as here many HTML renderers
are emitting unreadable pages.
For options use '\ ' (non-breakable space) between option and
its argument and option are not across line.
Reformat lines to fit in 80 columns.
Use .EX .. .EE for example output - this improves
character alignment for poststrict/pdf rendering as
it use monospace fonts (unlike .nf .. .fi).
Also with Example section never let the line being with space
and use '\' for such line.
If the Example line should being with '.' it needs to be
prefix with \&.
Add .nf/.fi macros for cases where .EX & .EE are undefined
(this happens i.e. with man2html)
Avoid use of tables (.TS .. .TE) as HTML renderers often use
images (.png) files for such tables and this does not scale well
when user changes font size in browser.
(and the large/long table were split into 2 pieces
so it does fit to 80 columns).
Use .MT .ME for emails.
Use .UR .UE. for URL.
Use .\|.\|.\& as sequence of 3 dots and add \& to not create end of
sentence and possibly wrong alignment.
On lines starting with .BR & .IR avoid using \fB \fI as this
can cause problem when i.e. html rendered may keep use
italic bold when just italic was really wanted.
When using series of .TP/.IP - set the size only with the
first tag - and let renderer align others to match the column.
Correct some small typographical rendering issues.
Rendering was evaluated for readable results with:
- mandoc -T html (-O style=mandoc.css)
- groff -Thtml -mman
- man2html
- man -Thtml
- man -Tps
- man
Unfortunatelly various g/troff troubles are with each of them,
so we need to select usage of macros in a way, that is not mangling
results for above engines.
Improve generated output for better compliance with '-T lint' checker
(mandoc -T lint and groff --mandoc).
Try to properly place .P sections and also correctly use .TP rendering
where we need to place '.na .ad' after the first rendered keyword,
otherwise it has not the desired impact.
Also use .nh .na .. .ad .hy around whole command USAGE description
so we avoid unwanted alignment,spacing,hyphenation there.
Make sure we are emitting properly order paragraphs and avoid
i.e. submit of .br after .RS/.in that has this implicit
Also more frequently emit '\n' so there are not too long lines as
rendering engine will format line breaking according to its rules.
To keep generated page better controllable emit more empty line
and use such lines for every .SH, .TP.
Use 'short_ops' loop to avoid duplicating code.
Emit .\|.\|. for 3dot sequence for proper graphical rendering
Emit \\0\\0\\0 (3 white space of width of letter '0')
for better alignment of options with graphical rendering.
There is ATM prepare '#define TABBED' - enabling this make
initial option list nicely aligned in graphical rendering
and doesn't seem to have bad side effect on text rendering.
man: generator use macros for options
Predefine all options used by the command into list of '.de O_name'
macros that are pregenerated in the front of man page.
(interestingly usage of groff strings (.ds) seems to have some
non-trivial issues across rendering engines)
This allows to use '\t' without producing warnings with
'mandoc -T lint' - as normally tabs are allowed only within
'.nf ... .fi' section, but than there is not working line-breaking.
While we could use purely 'tab' base version, for some 'html' (ascii)
redering its producing not so well indented option list.
For this reason use tabs only with graphical renderers .ie t / .el
and use only spaces for ascii rendering.
Ensure the ... (3dots continuation is properly renderer with
a single space after repeatable option/argument and not adding extra
space before i.e. closing bracket.
Using .ta for graphical rendering - allows to keep option aligned
with proportional font.
Couple commands (lvcreate,lvconvert,vgcreate,lvchange,vgchange)
has the 'specific' property that within them the option --profile
behaves like --metadataprofile, while for all other commands this
option should be simply an alias for --commandprofile.
We may eventually drop this rather confusing behavior in the future
version and there will be only one use as --[command]profile
It should be noted this --commandprofile can be often used
instead of --config option for preconfiguring setting
for some group of commands - we should possibly more propagate
this usage.
Recent patch set for select enhancement missed to initialize
ssl struct element regex to NULL and this code might have
crashed on this code path evaluation.
Actually mirror were never supposed to be usable with any newer
target as they are very problematic with any stacked usage.
So now it's going to be properly checked and prohibited.
Users are always supposed to use 'raid1' --type instead.
We can print list of supported LV types for
options like --cachepool,--thinpool,--vdopool when
they are specifying particular LV.
TODO: while we nicely document them, the parser engine ATM
is not capable to validate and enforce these properties,
so the code needs to match them on its own.
Although our command-line description file describes
supported types for conversion with some rules,
these are technically not yet fully implemented in
the code, thus we need explicit functionality to
validate passed LVs for conversion.
Add some explicit warning for commands that are destroying content
of converted volume.
Add thinpooldata to the list of allow LVs for caching
(as the code already support this).
See Linux source Documentation/admin-guide/blockdev/zram.rst .
zram devices offer a good performance and efficient resource utilization
through the use of compression.
Signed-off-by: David Disseldorp <ddiss@suse.de>
If lvresize is given a size > the maximum COW size for a given origin
the command will fail with an internal error and no error message:
# lvresize --size 1.6g fedora/snaptest-snap
Rounding size to boundary between physical extents: <1.59 GiB.
Reached maximum COW size <1.01 GiB (258 extents).
Command failed with status code 5.
With -vvv:
Found snapshot target v1.16.0.
Getting target version for snapshot-origin
dm versions [ opencount flush ] [2048] (*1)
Found snapshot-origin target v1.9.0.
Reached maximum COW size <1.01 GiB (258 extents). <<<
Unlock: Memlock counters: prioritized:0 locked:0 critical:0 daemon:0 suspended:0
Syncing device names
Unlocking /run/lock/lvm/V_fedora
_undo_flock /run/lock/lvm/V_fedora
Freeing VG fedora at 0x55781b142890.
Freeing VG fedora at 0x55781b136860.
global/notify_dbus not found in config: defaulting to 1
Destroy lvmcache content
Completed: lvresize -vvv --debug --size 1706243072b fedora/snaptest-snap
Internal error: Failed command did not use log_error
This happens because in this case _lvresize_adjust_extents() returns
early without setting lp->resize to either LV_EXTEND or LV_REDUCE after
capping lp->extents to the maximum COW size.
Fix this by just capping lp->extents and relying on the existing code in
_lvresize_adjust_extents() to fixup lp->resize in the case that
lp->extents == existing_logical_extents. This is consistent with the
no-op case where -l is given as the existing size:
root@localhost:~/src/git/lvm2# LD_LIBRARY_PATH="$PWD/tools" ./tools/lvm lvresize -L 1.6g fedora/snaptest-snap
Rounding size to boundary between physical extents: 1.60 GiB.
Reached maximum COW size <1.01 GiB (258 extents).
New size (258 extents) matches existing size (258 extents).
No size change.
The c065b407cb77a7a14d7c7c3c94e09fcca2fcff09..872e085030ae8039f18908f6e45bad7ba99250a7
was for device_mapper/libdm-report.c. Do the same for libdm/libdm-report.c
Recognize regex in string list selection criterion, including grouping
items by using {} and [] together with && (or ",") and || (or "#")
logical operators:
- [ <regex> && <regex> ... ]
- [ <regex> || <regex> ... ]
- { <regex> && <regex> ... ]
- { <regex> || <regex> ... ]
Also recognize simple "<regex>" (without any grouping operators)
as a shortcut for "{ <regex> }".
Regex remembers the mempool it was given during dm_regex_create and
then it uses it for further allocation during dm_regex_match. This
could be dangerous in case we used the same mempool for any other
allocations/frees in between dm_regex_create and dm_regex_match calls
in the outer code. This patch adds separate regex mempool for the
report/select to avoid the possible issues.
Previous patch made a proper difference between [...||...] and
[...&&...]. If the criterion for a string list does not use any [] or
{}, we need to make sure that proper matching function is called -
in this case not using {} or [] is the same as if {} was used
(matching subset).
Matching a string list criterion which had [... || ... ] was not
correctly implemented - it was the same as [ ... && ... ]. This patch
makes a difference between the two:
- [ ... || ... ] matches if all items from string list value are
matched by ANY item from selection string list (that is, not
all the selection string list items need to match)
- [ ... && ... ] matches if all items from string list value are
matched by an item from selection string list 1:1 (that is,
all the selection string list items need to match)
Remove superfluous struct reserved_value_wrapper param for
_tok_value_regex function. The only thing that _tok_value did was
zeroing the reserve field within the struct. But this one is already
zero-initialied in outer _parse_selection function.
Describe missed --segments opt for lvdisplay (matches lvs).
Describe lvm-fullreport --all option - show text for lvs,vgs,pvs.
Missed '.' for --separator.
Initialization of union is somewhat tricky as it initialize only
the first member + padding, but in our case this does not clear
the whole size of union so explicitly set \0 after 2 'struct id'
and make sure DM uuid is not using random characters from stack.
Also add explicit .id designators (c99).
After detecting that a VG has wrongly claimed a PV, unpair
the pv->dev setting. This will cause the usual "missing PV"
message to appear for that VG. Make this message, and some
others, clearer by using the VGID rather than the VG name
when there are multiple VGs with the same name.
_vg_read() calls lvmcache_update_vg_from_read() which detects
that the VG metadata is incorrectly claiming the PV. Flag this
condition in the PV status as WRONG_VG. Later, vg_read() can
simply check the WRONG_VG flag rather than repeating the same
PV/VG checks that were already done in lvmcache_update_vg_from_read.
Outdated VG metadata that appears when an old device is attached
to the system can result in PVs appearing to belong to the
old/wrong VG, and commands are allowed to use (corrupt) the PVs.
- vgcreate old /dev/sda /dev/sdb /dev/sdc
- offline /dev/sda
- vgreduce --removemissing old
- vgremove old
- vgcreate new /dev/sdb /dev/sdc
- online /dev/sda
When sda is reattached, sdb and sdc will appear to be
in VG old again. An attempt to correct the problem,
e.g. with vgremove old or vgreduce old, would modify
sdb and sdc, removing them from the new VG.
To fix this, check that sdb and sdc contain metadata for
VG old before allowing VG old to claim ownership of them.
With the fix, sdb and sdc are not displayed as part of
VG old, and commands to change VG old will fail as long
as it references incorrect PVs.
To fix VG old (sda), remove the incorrect PVs from VG old
while limiting the command to see only the correct PVs:
vgreduce --removemissing --devices /dev/sda old
If an inactive LV is being cached in writeback mode, then
removing the cache does a temporary activation to flush
the cache back to the main LV. However, it forgot to
deactivate the LV again, so the temporary activation
was left in place.
Drop 'TEST WARNING' from this case - the code works the same
were for 'pvmove -b' polling as for lvmpolld.
Just keep there 'TODO' notice we want to eventually reduce amount
of 'worked' pvmove monitoring process - ATM there will be 1 per LV.
The current manpage is unclear in the example of a raid10 type LV RAID
with --mirrors 1 --stripes 4. The number of devices in each raid1 mirror
is NumberStripes/(NumberMirrors+1). Thus the example should read:
e.g. mirrors 1 and stripes 4 will stripe data across two raid1
mirrors, where each mirror contains two devices.
Fixes: https://gitlab.com/lvmteam/lvm2/-/issues/26
Signed-off-by: Salvatore Bonaccorso <carnil@debian.org>
Previous commit 874a8ab4d0 missed 'IGNORE' mode.
Fix it by adding rather 'explicit' test for this value,
so the code is better readable.
Also unlock memory earlier and drop unneeded <backtrace>
from return since we already logged error in this function.
Since kernel patch 3d9a9e9a77c5ebecda43b514f2b9659644b904d0 (6.14)
it seem device size is no limitted to <8Exa bytes (so LLONG loff_t
works across whole device).
So reduce our tested size to 8191 Peta ~ <8 Exa.
Increase the used mirror size for longer processing.
Add more -vvvv traces to pvmove to better chase test error.
Add extra code to wait for finish of polling pvmove after it's
been aborted - so it doesn't break 'next' test.
Use "groff -e ' $' -e '\\~$' <file>" to find obvious trailing spaces.
Use "mandoc -T lint dmstats.8"
Use "test-groff -mandoc -t -ww -z dmstats.8"
-.-.
Lines containing '\c' (' \c' does not make sense):
503:.B \-\-units \c
-.-
Change '-' (\-) to '\(en' (en-dash) for a (numeric) range.
GNU gnulib has recently (2023-06-18) updated its
"build_aux/update-copyright" to recognize "\(en" in man pages.
dmstats.8:470:expressed as a hyphen separated range, for example: '1\-10'.
-.-.
Add a (no-break, "\ " or "\~") space between a number and a unit,
as these are not one entity.
1114:Create a 32M region 1G into device d0
-.-.
Add a "\&" after "e.g." and "i.e.", or use English words
(man-pages(7)).
Abbreviation points should be protected against being interpreted as
an end of sentence, if they are not, and that independent of the
current place on the line.
511:Can also specify custom units e.g. \fB\-\-units\ 3M\fP.
-.-.
Wrong distance between sentences in the input file.
Separate the sentences and subordinate clauses; each begins on a new
line. See man-pages(7) ("Conventions for source file layout") and
"info groff" ("Input Conventions").
The best procedure is to always start a new sentence on a new line,
at least, if you are typing on a computer.
Remember coding: Only one command ("sentence") on each (logical) line.
-.-.
The name of a man page is typeset in bold and the section in roman
(see man-pages(7)).
798:extents). This currently includes \fBxfs(5)\fP and \fBext4(5)\fP.
801:group, and the group alias is set to the \fBbasename(3)\fP of the
-.-.
Use thousand markers to make large numbers easy to read
560:is equivalent to 10000000. Latency values with a precision of less than
-.-.
Remove quotes when there is a printable
but no space character between them
and the quotes are not for emphasis (markup),
for example as an argument to a macro.
1:.TH DMSTATS 8 "Jun 23 2016" "Linux" "MAINTENANCE COMMANDS"
-.-.
Output from "test-groff -mandoc -t -K utf8 -rF0 -rHY=0 -rCHECKSTYLE=10 -ww -z ":
.-.
Additionally:
Fix some arguments for '.TP'. A single-font macro does not work with
'\c', so use a two-font macro.
-.-
Use the pair ".na / .ad" to set no-adjustment (same result as '.ad l')
and '.ad' to restore previous adjustment.
[Replacing ".ad l" ... ".ad b"]
Set singular '.ad b' to '.ad \*(AD' as the user should have the choice to
control the adjustment from the command line.
Add an empty string to string 'AD' with '.as AD "\"' to avoid a warning
about an undefined string.
-.-
Generally:
Split (sometimes) lines after a punctuation mark; before a conjunction.
Updated-by: zkabelac@redhat.com
Checking for defects with a new version
Use test-[g|n]roff -mandoc -t -K utf8 -rF0 -rHY=0 -rCHECKSTYLE=10 -ww -z < "man page"
Use "groff -e ' $' -e '\\~$' <file>" to find obvious trailing spaces.
Use "mandoc -T lint dmsetup.8":
Use "test-groff -mandoc -t -ww -z dmsetup.8":
-.-.
Add a (no-break, "\ " or "\~") space between a number and a unit,
as these are not one entity.
-.-.
Use "\e" to print the escape character instead of "\\" (which gets
interpreted in copy mode).
487:with its hex value (two digits) prefixed by \\x.
-.-.
Strings longer than 3/4 of a standard line length (80)
Use "\:" to split the string at the end of an output line, for example a
long URL (web address)
1030 <name>,<uuid>,\:<minor>,<flags>,\:<table>\:[,<table>+]\:[;<name>,<uuid>,\:<minor>,<flags>,<table>\:[,<table>+]]
-.-.
Add a "\&" after "e.g." and "i.e.", or use English words
(man-pages(7)).
Abbreviation points should be protected against being interpreted as
an end of sentence, if they are not, and that independent of the
current place on the line.
581:Note: Same cookie should be used for same type of operations i.e. creation of
767:Attempts to remove all device definitions i.e. reset the driver. This also runs
946:e.g. striped 2 32 /dev/hda1 0 /dev/hdb1 0
-.-.
Wrong distance between sentences in the input file.
Separate the sentences and subordinate clauses; each begins on a new
line. See man-pages(7) ("Conventions for source file layout") and
"info groff" ("Input Conventions").
The best procedure is to always start a new sentence on a new line,
at least, if you are typing on a computer.
Remember coding: Only one command ("sentence") on each (logical) line.
Mark a final abbreviation point as such by suffixing it with "\&".
Some sentences (etc.) do not begin on a new line.
Split (sometimes) lines after a punctuation mark; before a conjunction.
-.-.
Use \(en (en-dash) for a dash at the beginning (en) of a line,
or between space characters,
not a minus (\-) or a hyphen (-), except in the NAME section.
-.-.
Remove quotes when there is a printable
but no space character between them
and the quotes are not for emphasis (markup),
for example as an argument to a macro.
1:.TH DMSETUP 8 "Apr 06 2006" "Linux" "MAINTENANCE COMMANDS"
-.-.
Output from "test-groff -mandoc -t -K utf8 -rF0 -rHY=0 -rCHECKSTYLE=10 -ww -z ":
Updated-by: <zkabelac@redhat.com>
Handle interruption caught in sleep between polling and
abort() tool execution in such case.
(Although ATM we are not normally signalling the tool this way).
Improve error handling in polling functions where errors
were previously ignored. These errors result from serious
failures (e.g., allocation errors) and should lead to a full
command exit, as the tool cannot function in such a state.
FIXME:
However, there is a fundamental design issue worth considering:
when a command like pvmove --abort cancels an ongoing operation,
the existing polling command continues running and only terminates
once it detects that there is nothing left to poll.
Next issue is perment reopening of a VG when 'monitoring' progress.
And the last is big trouble with '--interval 0' which is able to
wait in DM ioctl() and hold the VG lock, and there is not good way
to about such operation (other then sending a signal to such process).
Remove any older lcov generated files ('*.gcda|gcno') then the currently
generated 'make.file' before creating a new lcov report.
Otherwise we may hit the problem of using some older generated files
possibly with different format.
Restore missing symbols to the libdevmapper.so library.
These symbols:
dm_bitset_parse_list@@DM_1_02_138
and dm_tree_node_size_changed@Base
become 'lost' with commit: 40b277ae17
which supposedly cleaned local 'symbols' from visibility,
however these missing symbols were impproperly exported.
Signed-off-by: Jianqi Zeng <zengjianqi@kylinos.cn>
It looks like occasionally supports_json() in cmdhandler.py
for some reason does not find 'fullreport' in err output
of lvm help... let's see more traces...
The simple common case of locking the LV to remove with a
persistent lock would usually be fine, but there are a number
of special cases that were not addressed:
- no locking was done for removing cow snapshot
- direct locking to vdo pool
- dm-cache uncache using lvremove was not handled
Use the same lockd_lvcreate_lock() for all cases in which
creating a new LV first requires locking another associated
LV, e.g. locking the pool or origin for the new LV.
Refactor _get_split_name(). code to simplify detection of memory leak
in _destroy_split_name(). Now there are always just 2 pointers
instead of conditional pointer free() which is hard to follow.
In most cases header should be self-compilable, so the
do not expect other 'header' files to be used upfront
so the header would be compilable.
No functional change.
These internal header were using misleading variable names
in function prototypes, but correct names were used in
function definition. Noticed with:
clang-tidy --checks=readability-inconsistent-declaration-parameter-name
No functional change.
Bit flags likely should never have been 'enum' but since
we have this in a public header - it might be hard to
replace this. So at least add missing 'enum' element
we use.
Although the code was exiting only for (update == 0),
the later code actually used requires 'extents' to exist
also for (update != 0).
TODO: The logic here is not very clear, more testing needed.
After reorganizing elements in `possible_takeover_reshape_type`
(in commit 5b92ce741f),
it became apparent that the code relied on struct overlap,
which is somewhat unsafe. This commit removes it and ensures
proper `const` qualification for the struct usage.
While building lcov files - ignore errors from 'negative' counter
(perhaps we can use -fprofile-update=atomic - but it would be another
slowdown of test runs)
Also ignore unexecuted blocks warnings with 'gcov'.
Failure of lcov goal is not supposed to error whole make build.
When test runs with lvmpolld - we cannot check
messages from pvmove - as those are visible through
output of lvmpolld - so just skip this and
only check LVs are in expected state.
Since long_opt was changed to char[], we were only comparing pointers
that always exist, whereas the original intention of the test was
to verify the presence of a string
(i.e., checking that the first byte is not \0).
vgreduce --removemissing --force replaces a partial image
with an error target. When that image includes an integrity
layer, that layer needs to first be removed.
Fix logging of VDO configuration info message which has acutally
printed " and," using next element..
Increase the array element size so it can store >=5 bytes
for " and" + \0.
Similar to the pvmove update, enhance error path handling
for scenarios where legs or logs remain open and cannot be
closed during the splitting of a mirror image.
Remove the now obsolete _delete_lv() function,
as it will no longer be needed.
When the pvmove operation is completing, it attempts to deactivate
the temporary mirror and remove its mirror legs. However,
if an external tool holds these volumes open, the operation would
previously abort entirely, leaving the LVM2 metadata in a partially
unusable state that required manual administrative fixes.
To improve this, the code has been enhanced to handle such scenarios
more gracefully. It will now complete the pvmove operation even
if some volumes cannot be deactivated, marking them in the metadata
with an error segment. While the command will report errors,
the metadata will remain in a usable state. The administrator
can then remove the orphaned volumes when they are no longer in use.
Fix the code that limited the total number of backup files.
It failed, and left excess files, when the file version
number was greated than 9999, exceeding the four digit suffix.
Now, after version 9999, the suffix intentionally grows beyond
four digits as needed, and is not a fixed width, or zero padded.
Fix for recent commit "lvmlockd: free resource structs for LVs"
When a vg_write() fails in lvcreate, lvmlockd sees init_lv()
followed by free_lv(). The LV lock is not acquired prior to
free_lv, and no prior resource struct exists. This wasn't being
handled.
Previous commit 5f71cebcbe was not
correct. 4k requirement cannot be put on attribute_offset - where
it is valid to have this only 512b aligned.
The rule might get more complicated to recognized invalid values.
For this moment however add more easier requirement - we
impose 4K restriction on minimal and optimal io size if they
are bigger then 1 sector (512B).
Sscan may automatically add 0 after field width mark,
and since it's not exactlu trivial to do a macro calculation
for PATH_MAX - 1, rather make buffer for sscanf results bigger.
Also use matching FSTYPE_MAX as field width specifier.
lvm2 is caching DM nodes with the use of DM_LIST_DEVICES ioctl().
And tried to preserve the cached structure for the same list,
however there was 1 case where cache was empty, and new LIST ioctl
returned some elements - if this DM table change has happened
in the moment of 'scanning' and locking - lvm2 has then continued
to use 'invalid' empty cache.
Fix by capturing this missed case and update cache properly.
TODO: we could possibly use plain memcmp() with previous ioctl result.
Since we started to use DM cache now also for basic checks
whether the DM devices is present in DM table, this cache
now needs to be actually refreshed when the LOCK is taken.
This hiddenly happend if there was enabled 'scan_lvs' however
still not at the right place.
Move this explicit cache update call right after the moment
vg_read grabs the lock.
TODO: in the optimal case, we should mark the 'cache invalid'
and later refresh this cache, when the first reader appears,
but since this would be large patch, do this little fix step patch
first and improve performance later.
Once created, resource structs for LVs were never being freed.
If LVs are activated, then later removed or never used again,
the unused structs waste memory and cause the resource list
to grow.
There was a lot of messy and inefficient locking calls sent from
a command to lvmlockd when working with thin volumes, e.g.
- requesting a lock numerous times that was already held
- releasing a lock numerous times that was already unlocked
- repeating lock/unlock/lock/unlock rather than holding the
lock until it was no longer needed
Mistakes in the locking could easily hide among all the noise.
The mess was largely because thin-related commands involve a
lot of internal LV manipulations, and lvmlockd calls were done
at the lower level of LV activation/deactivation. This change
adds locking code that is more specific to the thin command
being run, so it can be more intelligent in acquiring and
releasing locks where needed.
Since we now support disabling memory locking by setting
reserved memory or stack to 0 - it would be useful if this would
work also with cmdline --config option.
TODO: rework creation and usage of cmdtool context so we avoid
several places in the code which do try to initialized something...
Fix regression introduced with commit:
964012fdb9
that effectively disabled memory locking before suspending volumes.
From merging/testing there remained wrong condition
as we really want to check for 0 memory reservation value
for both checked settings.
Because now, we are doing the fsinfo check before extending an LV and if
that check fails, we do not proceed to the LV extension itself and the
lvextend command bails out immediatelly.
It seems we need new_size_bytes in places where struct fs_info is also
passed. Store the new_size_bytes inside the struct fs_info so we
can just pass that one to all the functions we call and hence make
the code a bit cleaner and easier to follow.
This avoids a situation where we would extend an LV and then we would
not do anything to the FS on it because the FS info check failed for some
reason, like the type was not supported (e.g. swap) or we could not resize
the FS unless being in some supported state (e.g. XFS to be mounted for
the xfs_growfs to work).
Before this patch (LV resized, FS not resized):
❯ lvextend --fs resize -L+4M vg/swap
Size of logical volume vg/swap changed from 32.00 MiB (8 extents) to 36.00 MiB (9 extents).
File system extend is not supported (swap).
File system extend error.
Logical volume vg/swap successfully resized.
With this patch (LV not resized, FS not resized):
❯ lvextend --fs resize -L+4M vg/swap
File system extend is not supported (swap).
Deactivate converted volume to pool early, so the conversion
exits early and does not leave some already created metadata
volumes that needed manual cleanup by user after command
aborted its conversion operation when the converted volume
was actually in-use (i.e. when user tried to convert
a mounted LV into a thin-pool, 2 extra volumes needed removal).
Device quirks may cause sysfs wwid file to change what it
displays, from a bogus eui... string to an nvme... string.
The old wwid may be saved in system.devices, so recognizing
the device requires finding the old value from libnvme.
After matching the old bogus value using libnvme, system.devices
is updated with the current sysfs wwid value.
All block devices have a disk sequence number assigned (an ever-increasing 64 bit
sequence number) since kernel v5.15 (February 2021). The number is exported through
/sys/block/<disk>/diskseq property and also as DISKSEQ udev event variable.
The diskseq helps with referencing a device throughout its existence in
race-free way.
By default, the /usr/lib/udev/rules.d/60-persistent-storage.rules set
/dev/disk/by-diskseq/<diskseq> symlink for each block device. However,
these rules do not apply for DM devices because we manage the symlinks
ourselves in 13-dm-disk.rules where it properly follows the
DM_UDEV_DISABLE_DISK_RULES flag, among other things.
Add a rule to 13-dm-disk.rules to create the /dev/disk/by-diskseq/<diskseq>.
Since commit d106ac04ab ("configure.ac: use LIBSYSTEMD"),
lvmlockd is not built with SD_NOTIFY by default but depending
on LIBSYSTEMD_LIBS. There are three prerequisites of
nonempty LIBSYSTEMD_LIBS:
NOTIFYDBUS_SUPPORT, SYSTEMD_JOURNAL_SUPPORT and SYSTEMD_JOURNAL_SUPPORT.
If ./configure is called with options ' --disable-systemd-journal
--disable-app-machineid --enable-lvmlockd-sanlock
--disable-notify-dbus', the lvmlockd built is without sd_notify
support which causes hang of start lvmlockd service in notify type.
This commit adds options disable-sd-notify and enable-sd-notify.
The default value is autodetected and when the lvm2 is build with
systemd then sd-notify is enabled.
If systemd/sd-daemon.h is existed, call PKG_CHECK_MODULES libsystemd.
Signed-off-by: Su Yue <glass.su@suse.com>
Modified-by: Zdenek Kabelac <zkabelac@redhat.com>
Also fix the use of --all that was mistakenly included
as an accepted option for vgdisplay and two cases of pvdisplay
in commit "tools: enhance lvdisplay vgdisplay pvdisplay"
Remove --noudevsync option - as this breaks synchronization with
udev which is necessary when trying to i.e. create _rmeta_3
and wipe it - as the symlinks must be present for wiping.
So if there was some other issue (behind the comment) - we need to
check for the problem elsewhere instead of disabling udev sync.
Automatically use --enable-notify-dbus when building lvmlockd
if not configured otherwise by a configure user -
as the lvmlockd.service is notify based.
Split description for display commands so we can better describe
it's usage and combination of individual options in man page.
Now we can separately describe:
lvdisplay, lvdisplay -c, lvdisplay -C
vgdisplay, vgdisplay -c, vgdisplay -C
pvdisplay, pvdisplay -c, pvdisplay -C
TODO: Drop validation from command code itself.
Update the _print_man_option_desc() to also handle common parts
as the initial text without any specified section and also
add support for '#\n' to be able to revert to common part.
Avoid printing lvm2 command trace, if the test finds the 'dmeventd'
was started unxpectedly during testing as the last command is hardly
ever responsible for this
Also reorder some messages when doing teardown of devices.
Do not print 'help' message from hostname command, when it does
not support option '-I'.
Since we now keep lv names valid all the time (as they are part
of radix_tree) - there is a problem with this renaming code, that
for a moment used duplicated name in vg struct.
Fix it by interating LVs backwared - which avoids breaking consitency
and also actually makes code more simple.
Add check for 'leaked' symlinks after test and trap
the case when some 'danglink' links are present.
This might be some problem with udev synchronization
or some other strange race.
All such symlinks will be also removed so they will not
influence following tests.
When command prints warning about suppressing query
for inactive table, because this is not supported
by kernel - 1 printed message is just enough, no
reason to 'spam' command output all the time, message
will remain only in debug log.
Also drop 'WARNING:' from real 'error' message.
WARNIGS are supposed to be just warning and command
then exists with 'success'.
When specifying minimum_io_size with --vdosettings,
command assumed wrong unit (sectors).
So '--vdosettings minimum_io_size=512|4096' resulted into
an error that only 512 or 4096 values are allowed, but
at the same time values 1 or 8 were accepted.
So fix by converting any number >= 512 to 'sectors' and
keep input of 1 or 8 still valid if anyone has been using
this before.
So now we take 512 or 4096 and still also 1 or 8 with the
same effect.
Also correct the 'error' message when invalid minimum_io_size
is specified.
The df -a looks at whole system and it returns an error code in case
there's an inaccessible fs which is not even part of the testing environment.
The -a for df is not actually needed here in the lvresize-xfs test, so remove it.
Fix stripe count and size parameter validation for RAID LVs and
include existing automatic setting of these parameters based
on current shape of the RAID LV in case these are not set
on command line fully.
Previously, this was done only to a certain subset given by this
condition (where the 'stripes' is the '-i|--stripes' cmd line arg
and the 'stripe_size' is actually the '-I|--stripesize' cmd line arg):
!(stripes == 1 || (stripes > 1 && stripe_size))
This condition is a bit harder to follow at first sight and there
are no comments around with explanation for why this one is used,
so let's analyze it a bit more.
First, let's convert this to an equivalent condition (De Morgan law)
so it's easier to read for humans:
stripes != 1 && !(stripes > 1 && stripe_size)
Note: Both stripe and stripesize are unsigned integers, so they can't be negative.
Now, based on that condition, we were running the code to deduce the
stripe/stripesize and do the checks ("the code") only if both of these
are true:
- stripes is different from 1
- we don't have stripes > 1 and stripe_size defined at the same time
But this is not correct in all cases, because:
A) if someone uses stripes = 0, then "the code" is executed
(correct)
B) if someone uses stripes = 1, then "the code" is not executed
(wrong: we still need to be able to check the args against
existing RAID LV stripes whether it matches)
- if someone uses stripes > 1, then "the code" is:
C) if stripe_size = 0, executed
(correct)
D) if stripe_size > 0, not executed
(wrong: we still want to check against existing RAID LV stripes)
Current issues with this condition:
The B) ends up with segfault.
❯ lvextend -i 1 -l+1 vg/lvol0
Rounding size 4.00 MiB (1 extents) up to stripe boundary size 8.00 MiB (2 extents).
Segmentation fault (core dumped)
The D) ends up with errors like:
❯ lvextend -i 3 -l+1 -I128k vg/lvol0
Rounding size 4.00 MiB (1 extents) up to stripe boundary size 8.00 MiB (2 extents).
Rounding size (4 extents) up to stripe boundary size for segment (5 extents).
Size of logical volume vg/lvol0 changed from 8.00 MiB (2 extents) to 20.00 MiB (5 extents).
LV lvol0: segment 1 with len=5 has inconsistent area_len 3
Couldn't read all logical volumes for volume group vg.
Failed to write VG vg.
Conclusion:
The condition needs to be removed so we always run "the code" to check
given striping args given on command line against existing RAID LV
striping. The reason is that we don't want to allow changing stripe
count for RAID LVs through lvextend and we need to end up with the
error:
"Unable to extend <RAID segment type> segment type with different number of stripes"
(We do support changing the striping by lvconvert's reshaping functionality only).
With commit acbeaa7a8d we started
to use symlinks to link test suite shell scripts, however
they remained within CLEAN_TARGETS.
So when running 'make clean' within non-srcdir build dir, we
were cleaning actuall shell script in this dir.
So remove list of this script from CLEAN_TARGETS in this case.
get_sizes_lockspace() may not always initilize all passed values
in case the bitfield would not trigger if() path.
So just in case keep the path initilized.
TODO: maybe add INTERNAL_ERROR to get_sizes_lockspace().
When converting a VG to locktype sanlock, a new
lease is allocated for each existing lv. Finding
a new lease location involved searching the lvmlock
LV from the start for an unused location, which
would be very slow with many LVs. Improve this by
starting each search from the last used location.
Fix regression from commit 7f29afdb06
"lvmlockd: configurable sanlock lease sizes on 4K disks"
That change failed to recognize that a running lockspace will not
exist in lvmlockd when converting a local VG to a sanlock VG, i.e.
vgchange --locktype sanlock vgname. When the vgchange attempted
to initialize new lv leases for existing LVs, lvmlockd would
return an error when it found no lockspace.
When searching for committed LV by uuid, this search can
be expensive for commands like 'vgremove' - so for
this part introduce 'lv_uuids' radix_tree that is
build with first access to lv_committed().
Since there is a group of commands that need to access 'lv_list'
while still need to search for LV by its name, make the whole
struct lv_list a member of logical_volume structure.
This makes it easy to return also 'lv_list' this list this LV
within VG.
Also the patch should not use more memory, since we were allocating
lv_list for each LV anyway when linkin LV to VG.
Since find_lv_by_name() is now using radix_tree(),
use the same 'search for /' in LV in name for both
find_lv() & find_lv_in_vg().
TODO: Possibly refactor code and use only dm_list
instead of lv_list and dereference LV with container_of()
(thus saving pointer within struct logical_volume) - but
we use 'lv_list' currently in many places...
Add lvm.conf config/validate_metadata configurable setting.
Allows to disable validation of volume_group structure before
writing to disk.
Call of vg_validate() is supposed to catch any inconsistency
of in-memory volume group structure and possibly early aborting
commnand before making any more 'damage' in case the VG struct
is found insistent after some metadata manipulation.
This is almost always useful for devel - and also for normal user
as for small metadata size this doesn't add too much overhead.
However if the volume_group size is large and operations are just
adding removing simple LVs - this validation time may add noticable
to final command running time.
So if the user seeks the highest perfomance of command and does
not do any 'complex' metadata manipulation - it's reasonably safe
to disable validation (with the use of setting "none") here.
With presence of uniq_insert, use this function also
here for extra protection and check for duplicate lv_name
when inserting a new name into radix_tree.
Avoid config 'grep' with actual 'randomly' generated path name
which may eventually contain 'cc' as part the path and
causing a mismatch of the grep test.
When 'lvresize -r' is used to resize the volume, it's valid to
resize even to the same size of an LV, as the command then runs
fs-resize utility to eventually upsize the fs to the current
volume size.
Return code of such command then reflects the return value
of this fs-resize tool.
This fixes the regression introduced when the support
for option --fs was added (2.03.17).
Use proper function names in annotation
There are no fuction named print_common_options_cmd()
and print_common_options_lvm(). So, rename them to the
real function named print_usage_common_cmd() and
print_usage_common_lvm().
Signed-off-by: YunJian Long
Tricky one - as the pipe exit codes may result into whole
test failure depending on how quick/slow command exits
are within pipeline.
So get the len without piping.
Replace usage of dm_hash with radix_tree to quickly find LV name
with a vg and also index PV names with set of available PVs.
This PV index is only needed during the import, but instead
of passing 'radix_tree *' everywhere, just keep this within
a VG struct as well and once the parsing is finished, release
this PV index radix_tree.
This also makes it easier to replace this structure
in the future if needed.
lv_set_name now uses radix_tree remove+insert to keep lv_names
tree in-sync and usable for find_lv queries.
Enhance usage with uniq_insert and also try to better
utilize CPU cache and do a smaller loop for individual
hashing of lvname and separately lvid.
Also correcting usage of 'continue' within validation of
historical names as it should report as much errors
as it can within a loop.
When using radix_tree to identify duplicate entries we may
avoid to call an extra 'lookup()' prior the insert() operation
add radix_tree_uniq_insert/_ptr() that is able to report -1 if
there was already set a value for the given key.
Avoid finding problems in vg_validate when restoring
invalid VG metadata as that would lead to internal error.
i.e. adding unsupported METADATA_FLAG to zero segtype
can trigger such thing.
Previous update needed to add handling segtype within flag.c
which somewhat breaks API separition and also had bug in hanlding
actual flags.
So instead keep segtype code in _read_segtype_and_lvflags() within
import_vsn1.c and handle purly flags in read_lvflags() from const
string.
Instead of duplicating whole segtype string with flags and
using 2 calls read_segtype_lvflags() + get_segtype_from_string(),
merge the functionality into a single read_segtype_and_lvflags().
This allows to make only a local string copy (no allocs) and eventually
to not copy segtype string at all, when there are no flags.
As the 'emit_to_buffer' uses relatively complex
vsnprintf() call inside, try to reduce number
of unnecessary calls and try replace some more
complex string build with a single call instead.
With existing code, the cache was working only to the 2nd. locking.
So i.e. when 'lvs' scans system with more then one VG, the caching
was effectively not working.
Update the code, so the label invalidate code is able to update DM
cache - so whenever we take a new lock - we will refresh the cache.
TODO: the refresh ATM does a very simple compare of old a new list
of cached DM device, and with the first spotted difference, it just
fallback to the full rebuild of DM cache - with large amount of active
devices this might not the most efficient way....
Since we detect 'debug' level after calling 'log_debug()' - all
the arguments are evaluated, so in this case display_lvname() was
preparing a string that is not used in case debugging is not enabled.
So since these string are on 'hot-path' and it's already known
which VG is being worked on, in these few cases just use lv->name.
When processing LVs for a command we stored '*object_id' & '*group_id'
as printable string that was however only used with json reporting.
Refactor code so we simply store there 'struct id*' that is just
converted into printable string when json reporting is really used.
Also check for 'sigint()' right before loop processing begins which
is primary purpose of this test.
This function call is able to setup config parser so it stops
parsing 'subsection' nodes after parsing named section node.
Only nodes at 'level' 0 will be still processed. And this nodes
are found by searching for last \n}\n sequence from the end of
buffer (instead of trying to analyze all the text in buffer).
Replace use of dm_hash with radix_tree when making PV index names.
Store just the index number itself and use pv%d for outf() string.
For lookup up a PV - use just the PV pointer itself, it's faster then
converint for it's ID to UUID format.
Split single check_lv_segments() into 2 separate
versions so they can be called independently.
This allow to 'skip' already checked segment
check after it's been imported to VG and also
avoid another repeated checking when validating
segment with complete vg.
**
check_lv_segments_incomplete_vg()
this check just basic LV segment properties and does not
validate those requiring full VG.
**
check_lv_segments_complete_vg()
Remaining check that expects complete VG is present.
ATM this rather save a lot of unncessary log entries as it grabs
the global autoextend_threshold (profile == NULL) just once instead
of revealing it every time with NULL profile.
Track whether import has even seen segment of LV with log_lv,
and call fixup mirror only in this case.
Also avoid repeated lookup of get_segtype_from_string for
SEG_TYPE_NAME_MIRROR.
Use bigger memory pool chunk size and reduces amount of
memory pool extensions when handling larger metadata, but do not
make it noticable bigger when handling small ones...
Use same large value also when allocating VG memory pool.
Instead of allocating string from a pool, for shorted strings
use buffer on stack since the string after the use in _find_or_make_node()
as no longer needed.
Eventually we may enhance code also for TOK_STRING_ESCAPED and TOK_STRING,
but they appear to be unused for _section().
For the most common part check for '#' when it's known it's not a space.
And also when we checked for '\n' we dont need to check again isspace().
Also help a bit more 'gcc' optimizer to grab buffer char just once and
simplify jump to next characted in the buffer when checking for token.
Avoid double dm_pool allocation call by copying string
for node name and config value directly after the end
of node/value structure.
It would be likely better to not copy these strings at all
and derefence it from the original string however that
needs futher changes in the code base.
This code is faster when calculating crc32 checksum for larger
block areas. There is also SIMD variant present in the code,
however ATM the influence on performance of lvm2 is not that big..
When BLKZEROOUT ioctl fails, it should not stop us from trying the direct
zeroing as a fallback action, since this is an optimization only.
We should be able to continue with new LV creation if we succeed
with that direct fallback then.
Related report: https://issues.redhat.com/browse/RHEL-58737
commit a125a3bb50 "lv_remove: reduce commits for removed LVs"
changed "lvremove <vgname>" from removing one LV at a time,
to removing all LVs in one vg write/commit. It also changed
the behavior if some of the LVs could not be removed, from
removing those LVs that could be removed, to removing nothing
if any LV could not be removed. This caused a regression in
shared VGs using sanlock, in which the on-disk lease was
removed for any LV that could be removed, even if the command
decided to remove nothing. This would leave LVs without a
valid ondisk lease, and "lock failed: error -221" would be
returned for any command attempting to lock the LV.
Fix this by not freeing the on-disk leases until after the
command has decided to go ahead and remove everything, and
has written the VG metadata.
Before the fix:
node1: lvchange -ay vg/lv1
node2: lvchange -ay vg/lv2
node1: lvs
lv1 test -wi-a----- 4.00m
lv2 test -wi------- 4.00m
node2: lvs
lv1 test -wi------- 4.00m
lv2 test -wi-a----- 4.00m
node1: lvremove -y vg/lv1 vg/lv2
LV locked by other host: vg/lv2
(lvremove removed neither of the LVs, but it freed
the lock for lv1, which could have been removed
except for the proper locking failure on lv2.)
node1: lvs
lv1 test -wi------- 4.00m
lv2 test -wi------- 4.00m
node1: lvremove -y vg/lv1
LV vg/lv1 lock failed: error -221
(The lock for lv1 is gone, so nothing can be done with it.)
Detect when we have mixed dos partition with gpt's PMBR partition.
This is not a sane configuration, but detect it anyway, just in case
someone configures such partition layout manually and forcefully and
incorrectly defines one of the partition types to be the GPT's PMBR.
For example:
❯ fdisk -l /dev/sdc
Device Boot Start End Sectors Size Id Type
/dev/sdc1 2048 67583 65536 32M 83 Linux
/dev/sdc2 67584 262143 194560 95M ee GPT
Before:
(The partition filter passes even though there's real existing dos
partition - the empty GPT PMBR overrides it.)
❯ pvcreate /dev/sdc
WARNING: PMBR signature detected on /dev/sdc at offset 510. Wipe it? [y/n]:
Wiping PMBR signature on /dev/sdc.
Physical volume "/dev/sdc" successfully created.
With this patch applied:
(The GPT PMBR does not override the existence of the dos partition.)
❯ pvcreate /dev/sdc
Cannot use /dev/sdc: device is partitioned
This provides better hints when trying to resize the fs on top of an LV.
Also needs a3f6d2f593 for proper operation.
❯ lvs -o name,size vg/swap
lv_name lv_size
swap 60.00m
Before:
❯ lvextend -L72m vg/swap
Size of logical volume vg/swap changed from 60.00 MiB (15 extents) to 72.00 MiB (18 extents).
Logical volume vg/swap successfully resized.
❯ lvreduce -L60m vg/swap
File system swap found on vg/swap.
File system device usage is not available from libblkid.
❯ lvreduce -L50m vg/swap
Rounding size to boundary between physical extents: 52.00 MiB.
File system swap found on vg/swap.
File system device usage is not available from libblkid.
After:
❯ lvextend -L72m vg/swap
Size of logical volume vg/swap changed from 60.00 MiB (15 extents) to 72.00 MiB (18 extents).
Logical volume vg/swap successfully resized.
❯ lvreduce -L60m vg/swap
File system swap found on vg/swap.
File system size (60.00 MiB) is equal to the requested size (60.00 MiB).
File system reduce is not needed, skipping.
Size of logical volume vg/swap changed from 72.00 MiB (18 extents) to 60.00 MiB (15 extents).
Logical volume vg/swap successfully resized.
❯ lvreduce -L50m vg/swap
Rounding size to boundary between physical extents: 52.00 MiB.
File system swap found on vg/swap.
File system size (60.00 MiB) is larger than the requested size (52.00 MiB).
File system reduce is required and not supported (swap).
blkid does not report FSLASTBLOCK for a swap device. However, blkid
does report FSSIZE for swap devices, so use this field (and including
the header size which is of FSBLOCKSIZE for the swap) instead to
set the "filesystem last block" which is used subsequently for
further calculations and conditions.
We already detect msdos partition table. If it is empty, that is, there
is just the partition header and no actual partitions defined, then the
filter-partitioned passes, otherwise not.
Do the same for GPT partition table.
New config setting sanlock_align_size can be used to configure
the sanlock lease size that lvmlockd will use on 4K disks.
By default, lvmlockd and sanlock use 8MiB align_size (lease size)
on 4K disks, which supports up to 2000 hosts (and max host_id.)
This can be reduced to 1, 2 or 4 (in MiB), to reduce lease i/o.
The reduced sizes correspond to smaller max hosts/host_id:
1 MiB = 250 hosts
2 MiB = 500 hosts
4 MiB = 1000 hosts
8 MiB = 2000 hosts (default)
(Disks with 512 byte sectors always use 1MiB leases and support
2000 hosts/host_id, and are not affected by this.)
In cases user is sure he is not using his 'rootfs' or 'swap' on LVs
managed with his command - it possible to completely bypass pinning
process to RAM which may eventually slightly speedup command execution,
(however at the risk the process can be eventually delayed by swapping).
Basicaly use this only at your risk...
TODO: add some dmeventd support for this.
Previously, lvmlockd detected the end of the lvmlock LV
by doing i/o to it until an i/o error was returned.
This triggered sanlock warning messages, so use the LV
size to avoid accessing beyond the end of the device.
Previously, every lvcreate would refresh the lvmlock LV
in case another machine had extended it. This involves
a lot of unnecessary work in most cases, so now compare
the LV size and device size to detect when a refresh is
needed.
lvremove of a thin lv while the pool is inactive would
leave the pool locked but inactive.
lvcreate of a thin snapshot while the pool is inactive
would leave the pool locked but inactive.
lvcreate of a thin lv could activate the pool to check
a threshold before the pool lock was acquired in lvmlockd.
The lv_hash wasn't being passed to the seg-specific text import
functions, so they were doing many find_lv() calls which consumes
a lot of time when there are many LVs in the metadata.
While performing udev sync semaphore's inc/dec operation, we use the
result from GETVAL semctl just to print a debug message with current
value of that sempahore, nothing else.
If the GETVAL fails for whetever reason while the actual inc/dec
completes successfully, just log a warning message about the GETVAL
(and print the debug messages without the actual semaphore value)
and return success for the inc/dec operation as a whole.
Clean up udev sync semaphore on fail path during its creation, otherwise
the caller will have no handle returned to clean it up itself and the
semaphore will keep staying in the system. The only way to clean it up
would be to call `dmsetup udevcomplete_all` which would destroy all
udev sync semaphores, not just the failed one, which we don't want.
The same message is printed while performing create/inc/dec operation and
the GETVAL semctl fails. Add a prefix so we know exactly in which of
these functions the issue actually happened.
LV with pvmove_ prefix is not allowed to be created by user
so bigger chance our selected name will never exist.
TODO: probably add code to get generic unused LV name...
Older gcc doesn't really like complex types (buffer, struct) to be
initialized without extra {} around such type.
So pick any other 'single type' var from a struct and set it to 0,
rest will do the compiler without emitting a warning.
Revert 373372c8ab and instead update
our validation code to handle LVs with empty segment - currently
we should need this only for pvmove operation, thus such LV should
have name 'pvmove%u'.
This fixes a problem where user tried i.e. pvmove on a VG with single
PV - as reported: https://github.com/lvmteam/lvm2/issues/148
Reported-by: bob@redhat.com
The .cache and compile_commands.json is used by popular source crawling and
indexing clang tools which in turn may be integrated with source code editors.
We may reuse the .cache directory for for other caches and temporary
files.
The /doc/.ikiwiki and /public are related to the ikiwiki.
Avoid possible udev race - since dmsetup create is
not using the same cookie logic as lvm2 commands,
try to avoid racing on some systems with udev scanning.
The option can be used in multiple ways (like --cachesettings):
--integritysettings key=val
--integritysettings 'key1=val1 key2=val2'
--integritysettings key1=val1 --integritysettings key2=val2
Use with lvcreate or lvconvert when integrity is first enabled
to configure:
journal_sectors
journal_watermark
commit_time
bitmap_flush_interval
allow_discards
Use with lvchange to configure (only while inactive):
journal_watermark
commit_time
bitmap_flush_interval
allow_discards
lvchange --integritysettings "" clears any previously configured
settings, so dm-integrity will use its own defaults.
lvs -a -o integritysettings displays configured settings.
log/command_log_report config setting defaults to 1 now if json or json_std
output format is used (either by setting report/output_format config
setting or using --reportformat cmd line arg).
This means that if we use json/json_std output format, the command log
messages are then part of the json output too, not interleaved as
unstructured text mixed with the json output.
If log/command_log_report is set explicitly in the config, then we still
respect that, no matter what output format is used currently. In this
case, users can still separate and redirect the output by using
LVM_OUT_FD, LVM_ERR_FD and LVM_REPORT_FD so that the different types
do not interleave with the json/json_std output.
In case of different PV sizes in a VG, the lvm2 allocator falls short
to define extended segments resiliently asked for 100%FREE RaidLV extension
and a RAID distinct allocation check fails. Fix is to release a memory pool
on the resulting error path.
Until the lvm2 allocator gets enhanced (WIP) to do such complex (and other)
allocations proper, a workaround is to extend a RaidLV to any free space on
its already allocated PVs by defining those PVs on the lvextend command line
then iteratively run further such lvextend commands to extend it to its
final intended size. Mind, this may be a non-trivial extension interation.
The cmd struct is now required in many more functions, and
it's added as a function arg for most direct dev-cache function
calls. The cmd struct is added to struct device (dev->cmd) so
that it can be accessed in many other cases where dev-cache
functions are being called from places where getting the cmd
struct is too difficult.
The dm devs cache is separate from the ordinary dev cache,
so give the function names distinct prefixes, using
"dm_devs_cache" to prefix dm devs cache functions.
When a PV is stacked on an LV, the PV needs to be
dropped from bcache before the LV is processed.
The LV can be found in dev-cache using its name
rather than the devno.
The list of dm devs was in the cmd struct and had a
different lifetime than the radix trees referencing
those dm devs. Now the list and radix trees are
created and destroyed together.
In the context of dm, 'device' refers to a dm device, but
in the context of lvm, 'device' refers to struct device.
Change some lvm function names to make that difference clearer.
dev_manager_get_device_list() -> dev_manager_get_dm_active_devices()
get_device_list() -> get_dm_active_devices()
device_get_uuid() -> dev_dm_uuid(), devno_dm_uuid()
The comment explained that the ex global lock was just
used to trigger global cache invalidation, which is no
longer needed. This extra locking can cause problems
with LVM-activate when local and shared VGs are mixed
(and the incorrect exit code for errors was causing
problems.)
vgchange -an vg is permitted when the vg lockspace
is not available, because LVs could still be active
for some reason, and they should be inactive when not
properly locked. In case lvmlockd was not running, or
the lockspace was not started, the command was
unnecessarily trying and failing to unlock every LV,
printing errors for every LV. We can skip this when
the lockspace is known to not be available.
vgchange --lockstop will fail with EBUSY if orphan locks in the
lock manager prevent stopping the lockspace. The orphan locks
can then be adopted and released, and the lockspace then stopped
cleanly.
Lock adoption is not part of standard command behavior, but can
be used for manual recovery or cleanup from unexpected failure
cases. Like other lockopt values, they are hidden options for
--lockopt. Different lock managers will behave differently.
Adopting locks with lvmlockd -A1 is more accurate and automatic.
--lockopt adoptls
. for vgchange --lockstart
. adopt existing ls, or fail if no existing lockspace is found
--lockopt adoptgl | adoptvg | adoptlv
. for commands using lvmlockd locks
. adopt orphan gl/vg/lv lock, or fail the lock request if
no orphan lock is found
. will fail if orphan lock exists with a different lock mode
. command may still continue with a failed shared lock request
--lockopt adopt
. for lockstart or any command using lvmlockd locks
. adopt existing lockspace, or start lockspace if none exists
. adopt orphan gl/vg/lv lock, or acquire new lock if no orphan found
. will fail if orphan lock exists with a different lock mode
. command may still continue with a failed shared lock request
. with dlm this option only works for ls
Stop printing "Skipping global lock: lockspace not found or started"
for vgchange --lockstart, since it's generally an inherent limitation
that the global lock isn't available until after locking is started.
Update the start delay warning to "a few seconds".
The lvb is used to hold lock versions, but lock verions are
no longer used (since the removal of lvmetad), so the lvb
is not actually useful. Disable their use for sanlock to
avoid the extra i/o required to maintain the lvb.
vgremove with --lockopt force should skip lvmlockd-related
steps and allow a forced vg cleanup, in addition to using
--nolocking to skip normal locking calls.
Previously, a command would call lockd_vg() for a local VG,
which would go to lvmlockd, which would send back ENOLS,
and the command would not care when it saw the VG was local.
The pointless back-and-forth to lvmlockd for local VGs can
be avoided by checking the VG lock_type in lvmcache (which
label_scan now saves there; this wasn't the case back when
the original lockd_vg logic was added.) If the lock_type
saved during label_scan indicates a local VG, then the
lockd_vg step is skipped.
The idea in the patch 6e6d4c62b for handling -suffix as
indication of private device needs to be disabled.
Some problematic cases are currently not resolvable and some
more thinking is needed.
Once fixed, we can revert this patch.
For large device sets our dm_hash can produce larger amounf of mapping
collision and we would need to further increase our has size.
So instead use the radix_tree code which is immune agains growing size
of devices and uses memory more effiecently to store all the paths.
Instead of less efficient 'btree' switch dev_cache to use
radix_tree, that is generating more efficient tree mapping.
Some direct use of btree iteration replace with our dev_iter code.
Convert the persisten filter to use more memory compact radix_tree as
dm_hash is bound to preallocated number of slots and stores whole
key together with value.
When DM uuid cache is available, we can possibly avoid unnecessary
status ioctl() when we check the device for 'usable' uuid.
If this test passes the existing code will got through the full check.
Move the code around caching active dm device devno, name and uuid
from device_mapper/libdm-iface to dev_cache file - as libdm layer
cares about 'decoding' ioctl data from kernel and caching for use by
lvm stays within lvm.
Introduce:
dev_cache_update_dm_devs
dev_cache_get_dm_dev_by_devno
dev_cache_get_dm_dev_by_uuid
Use radix_tree for searching.
Do not attach layer suffix to the UUID when activating component LV.
In this case we want to see allow this LV to be public, thus
such LV should not be using -layer suffix in its UUID.
This also requires that our 'cached' access will check for
both UUID (with & without suffix) which was unnoticed issue before.
This change is now necesssary since our udev rule automatically expects
any LV with -layer suffix is private and will prevent generaring
any systemd unit even when there are no 'DM' flags bits passed via
cookie mechanism while creating such LV.
In order to free SubLVs after a stripe removing reshape, lvconvert has
to be run without layout changes. Prevent a layout changing request
in case any such freed SubLVs exist and inform the user about the fact
requesting to release them first.
Calculates expected size before/after reshapes adding/removing stripes
to/from RaidLVs with levels 4/5/6/10 and compares it with the actual
one the block layer shows. Stripes reshaped to are listed in the
tst_stripes variable. mkfs/fsck/resize2fs the respective RaidLVs
to confirm ext4 can be resized accordingly without issues.
We automatically ignore these devs, when lvm2 create devs,
whoever when lvm2 database is dropped or someone just
created these devs with such formated UUID, there is no
other informantion then to check DM UUID.
SUSE kernels distribution enables CONFIG_PRINTK_CALLER by default.
One line of cat /dev/kmsg is like:
6,904,9506214456,-,caller=T24012;loop8: detected capacity change from 0
to 354304
If CONFIG_PRINTK_CALLER is off:
6,721,53563833,-;loop0: detected capacity change from 0 to 354304
',caller=T24012' is the redundant part needed to be handled.
Otherwise pos will be lager than buf size causing sz underflowed.
Then constructor of std::string will throw a exception to break
tests:
$ make check_local T=shell/000-basic.sh
VERBOSE=0 ./lib/runner \
--testdir . --outdir results \
--flavours ndev-vanilla --only shell/000-basic.sh --skip @
running 1 tests
running: [ndev-vanilla] shell/000-basic.sh
0:00.000Exception: basic_string::_M_create
make[1]: *** [Makefile:148: check_local] Error 1
make[1]: Leaving directory '/root/lvm2/test'
make: *** [Makefile:89: check_local] Error 2
Fix it with strchr for ';' as delimiter to get pos.
Reported-by: Su Yue <glass.su@suse.com>
To more easily adopt radix_tree over existing code base, add
abstraction over 'radix_tree_iterate' which basically builds
an array of all traversed values, and then it's just easy to
go over all array elements.
TODO: code should be converted to use radix_tree_iterate()
directly as it's more efficient.
Note: it can be possibly to rewrite recursive _iterate() usage
to linear travesal, not sure whether it's worth the effort
as conversion to 'radix_itree_iterator' is relatively simple.
Add simple 'wrapper' inline functions to insert or return ptr lookup value.
(So the user doesn't need to deal with 'union radix_value' locally and
also it makes easier to translage some lvm2 functions to use radix_tree).
Note: If the stored 'value' would NULL, it cannot be recognized
from a case of 'not found'. So usable only when 'values' stored with
tree are not NULL.
Some updates to _dump() debugging function so the printed result
can be more easily examined by human.
Also print 'prefix' as string when all chars are printable.
Add 'simple' variant of _dump also to 'simple' version of radix tree
(which is however normally not compiled).
Instead of using 'key state & key end' uint8_t* switch to use
void* key, & size_t keylen. This allows easier adaptation with
lvm code base with way too much casting with every use of function.
Also correctly mark const buffers to avoid compiled warnings and
casting.
Adapt the only bcache user ATM for API change.
Adapt unit test to match changed API.
Fix commit b65a2c3f3a "vgimportdevices: skip lvmlockd locking"
which intended to disable lvmlockd locking, but the lockd_gl_disable
flag was mistakenly set after lock_global() so it wasn't effective.
This caused vgimportdevices to fail unless locking was started.
Enums are single 'values' so not a proper type for bitfields.
(Probably better to use such values as defines).
Although here 'daemon_talk()' is part of library API, it's hidden
non-public API call - and moreover 'enum' and 'unsigned' are
using the same size, so linker shouldn't have any issue with
this symbol usage.
For this reason there are no 'versioning' tricks applied.
Size of these hashes was quite small, so raise the size of
hashed entries to reduce amount of hash collistion.
Select some unique/unused number for hash_create below 8192.
Replace call to get_dm_uuid_from_sysfs() with use of
device_get_uuid() which gets the same information,
but instead of several syscalls it need either 1 or even 0
when the information is cached with newer kernels.
We've got cached DM list before grabbing lock, so there
is some chance, that DM table has changed and we would
need to refresh this info.
TODO: benchmark, whether it would even make sense to refresh cache
and keep it content instead of using individual ioctl() for tree build.
Function that is working with DM target is located within
lib/activate directory.
This function is able to use cached dm_device_list when possible
to quickly resolve checks for device's UUID.
Function can fully replace get_dm_uuid_from_sysfs() and instead
of syscalls for open/read/close get the UUID with single ioctl.
When there is cached dm devs list, we can get many UUID from
a single syscall.
For better use of cached data located within cmd_context,
pass this structure from the top level function.
Also add missing '_' for static _dev_cache_index_devs.
No other change here.
Introduce function to find device's name and uuid for
a given major:minor.
This information is cached with dm_device_list which reads all the
info from single ioctl(DM_DEVICE_LIST).
Lvm keeps major:minor name & uuid for active devices in the system.
Skip scan and stat() for dirs and nodes within known /dev/ paths,
where no block devices are located.
Also strlen(_cache.dev_dir) just once.
TODO: add more dirs to _no_scan (configurable via lvm.conf ?)
Just like with _VAL strings, also _ARG strings do not need to
be present - as we can easily check for LONG opt version just
by adding attribute.
With attribute ARG_LONG_OPT string arg name[] becomes unused
and can be safely removed.
Also within _find_command_id_function() we do not need to handle
'command_enum == CMD_NONE' as separate case and just use single loop.
Previous commit 82617852a4
introduce bug in complession - as the rl_completion_matches()
needs to always advance to next element where the index
is held in static variable.
Add comment about this usage.
When PVs are created on LVs, remove the devices file entries
for the PVs when the LVs are removed. In general, the devices
file entries should be removed with lvmdevices --deldev when
the LVs are removed (lvremove is the equivalent of detaching
a device from the system when layering PVs on LVs.)
This change is effectively an automatic lvmdevices --deldev
command that is built into lvremove when the LV has a PV on it.
An OS installer can create system.devices for the system and
disks, but an OS image cannot create the system-specific
system.devices. The OS image can instead configure the
image so that lvm will create system.devices on first boot.
Image preparation steps to enable auto creation of system.devices:
- create empty file /etc/lvm/devices/auto-import-rootvg
- remove any existing /etc/lvm/devices/system.devices
- enable lvm-devices-import.path
- enable lvm-devices-import.service
On first boot of the prepared image:
- udev triggers vgchange -aay --autoactivation event <rootvg>
- vgchange activates LVs in the root VG
- vgchange finds the file /etc/lvm/devices/auto-import-rootvg,
and no /etc/lvm/devices/system.devices, so it creates
/run/lvm/lvm-devices-import
- lvm-devices-import.path is run when /run/lvm/lvm-devices-import
appears, and triggers lvm-devices-import.service
- lvm-devices-import.service runs vgimportdevices --rootvg --auto
- vgimportdevices finds /etc/lvm/devices/auto-import-rootvg,
and no system.devices, so it creates system.devices containing
PVs in the root VG, and removes /etc/lvm/devices/auto-import-rootvg
and /run/lvm/lvm-devices-import
Run directly, vgimportdevices --rootvg (without --auto), will create
a new system.devices for the root VG, or will add devices for the
root VG to an existing system.devices.
lvm2 for a while already optimizes 'vgremove' operation to
a single commit when possible if all LVs can be
easily deactivated.
So the number of LVs doesn't matter much - but the tested
case 'test_delete_non_complete_job' seems to be taking
some time anyway to capture the exception.
So just reducing the running time of the test significatanly
as we don't need to create 64LVs for 4 'execution mode' runs.
Order is used for man page generation (although not completely).
So place 'zero & error' to the end of list.
Keep linear,striped,snapshot in front.
For the rest use alphabetic order.
Move part of the 'inner' loop which is would be otherwise
always production same results for all 'opt_enum' values
out of the loop, so it can be evaluated just once.
Instead of resolving and storing 'command_fn'
withing 'struct command' use just funtion enum
and resolve function pointer just in place,
where it is really needed - first try to resolve
'new style' and fallback to 'old style' named.
Instead of storing command_id as string, direcly
translate string to enum index and use 'command_enum()'
to get string when needed for printing.
This way we can easily detect error in the structure
while parsing it - and we can later avoid separate
'translation' loop.
We can more efficiently use command_name struct to
lookup for lvm_command_enum and avoid many repeated
command name searches since we already know
the enum index that is now stored in 'struct command'.
Constity members in cmdline_context, would be nice, to replace
this static struct with couple function calls.
Also replace some 'while' loops with for loops, so code
is more readable.
Split struct command_name to the constant part (keep the name)
and new 'struct command_name_args' which holds runtime computed
info. To get to the _args part - we can easily use
lvm_command_enum as equivalent index.
Constified part 'struct command_name' is now fully stored
in .data.rel.ro segment, while command_name_args part goes
to .bss segment.
Code will be further reduced with next refactoring.
Refactor code to not allocate memory for rule->opts,
instead use uint16_t array of MAX_RULE_OPTS within cmd_rule.
Also print more info if array would not be enough (>= 8).
Add slight delay for waiting until 'started' dmeventd is
responsing to other 'dmeventd -i' command.
TODO: race observed here is somewhat unclear, might need some more
details
There is no reason to wait for 10sec when removing 'vg' at test
exit - we just need to kill 'sleep 10' process forked from flock.
(Not using 'flock -F' as this flag is not across all machines used
for testing)
This reverts commit d16a8f80e9.
So the correction was OK, however we missed to fix also
cut&paste bug here - as the second check should be
actually checking field->type.
Update definitions to add support for some more VDO options
when converting volumes to be used as thin-pool with vdo data volume.
Split some option in existing OO_LVCONVERT_VDO to OO_LVCONVERT_VDO_POOL
and reused then with OO_LVCONVERT_THINPOOL.
When convert already existing vdopool to be used as
thin-pool backend and user is passinng option for VDO configuration
process them - as we know converted LV is offline, we can do such
change easily instead of telling user to run separate lvchange later.
When converting volumes to be used for thin-pool with VDO, allow
users to control wipesingature behaviour.
By default volumes should be checked against signature, and if
they are present, we promt user whether he wants to process with
conversion and lose i.e. filesystem present on such volume.
Users that want to bypass prompt in script can use either --yes
or they can disable wipe signature -Wn.
Resolve reported leak by renaming existing pckk() to pvck_mf()
and wrapping pvck() over this. This function can easily
free allocated buffer within the subfunction.
When we switch supported_reserved_types_with_range to const
gcc repots this problem:
warning: ‘and’ of mutually exclusive equal-tests is always 0
!(iter->type & supported_reserved_types_with_range))) {
It's not clear from the history what was the actual intention of this
internal error test, let's assume the check wanted to make sure
that when DM_REPORT_FIELD_RESERVED_VALUE_RANGE is set,
some other fields from supported_reserved_types_with_range set
are also selected.
This was rather API mistake - the internal of libdm
do handle suffixes list as const string, just the API
was only using 'const char **'.
So the user may pass safely casted 'const char * const`.
man-generate --check actually noticed 2 same definitions
for snapshot create with 'lvreate -T [--snapshot]'
and 'lvcreate --snapshot [-T]'.
So drop the '-T' from second alternative variant as
thin type is already implied here.
Refactor code so the definitions may become 'static const'
and with configure_command_option_values() we update options
val_enum for actually running command option when used.
Also update _update_relative_opt() which is used for
generating man pages and command help.
Introduce enumeration for lvm2 commands - so we may
use enum cmd_COMMAND instead of string checking.
So running command now does not modified opt_names.
Function to _allocate_memory() was not compiled-in when lvm2 was
build with support for better tracking memory pool with valgrind.
Instead now correctly avoid this function only when running
withing valgrind environment.
Do not modify flags field from 'strcut command_name' and
instead control this via cmd_context get_vgname_from_options.
Flag GET_VGNAME_FROM_OPTIONS is currently used only by lvconvert.
Create leading dirs for lvmdbusd lock file, otherwise it fails to start:
| systemd[1]: Starting LVM2 D-Bus service...
| lvmdbusd[1602]: [1602]: Error during creation of lock file(/var/lock/lvm/lvmdbusd): errno(2), exiting!
Signed-off-by: Kai Kang <kai.kang@windriver.com>
In the past it was common for a single command to run
multiple lvmcache_label_scan, and this code was a way
to make each call select the same duplicate pvs. Now
commands run a single lvmcache_label_scan, so this is
not needed.
When reseting stream buffer - check for being run within valgrind
and only in this case skip this code.
Define VALGRIND_POOL was incorrectly used for this logic.
We can run and build this code just fine with VALGRIND_POOL
and just rely on runtime detection - so we do not close
descriptors also within running valgrind session.
Old system do not work well with -l% in findstring,
so use a different strategy to recognize whether -lreadline needs
another library (-ltinfo, -lncourses...)
(So we don't need to solve this via 'configure')
Also for now comment out -Wl,-z,pack-relative-relocs and leave it
for the package maintainer whether they want to use.
Possibly add some 'configure' autodetection for usable switch,
as it's relatively new feature..
Add some -Wl flags separatly and avoid their duplication.
Also add --as-needed only when system is using 'newer' readline
library - on these older system the usage of '--as-needed'
seems to be causing some hard to solve problem - so avoid it.
Attach -Wl,-z,relro,-z,now,-z,pack-relative-relocs,--as-needed
to LDFLAGS, but only if LDFLAGS already doesn't contain 'relro'
(so it's not given repeatedly).
Also start to use -z,now linkage also when building libraries
with default compilation - this avoid calling symbol resolver
while library function are using function needing resolving.
Note: Fedora or RHEL rpm building is using:
CFLAGS=$(rpm --eval %{build_cflags})
LDFLAGS=$(rpm --eval %{build_ldflags})
Also split -DUSE_SD_NOTIFY into DEFS from CFLAGS.
Use LDFLAGS separately with every use of CLDFLAGS and leave
this flag only for handling versioning.
This will reflect any LDFLAGS setting use during make.
Look not only for whole 64byte sequence,
but seek also 32byte, 16byte and 8byte parts of the
key.
Currently to pass memcpy ZMM problems add possible
workaround in the form of GLIBC_TUNABLES setting.
Avoid using 'lvs' from 'get' shell - as that would wait until
whole group of processes is finished.
TODO: rethink what would be the point of starting 'dmeventd' with lvs.
It seems to break some rules.
Refactor _start_daemon and add synchronization delay waiting
untill new forked dmeventd actually open fifos and is ready
to process messages.
This closes some race window in testing.
When no lock manager for the global lock had been set yet,
and the first global lock request was received, and both
dlm and sanlock were running, lvmlockd would assume it
should use the dlm for the global lock, and would start
the "lvm_global" dlm lockspace automatically. That's
not always correct, so don't assume the dlm should be used,
fail the lock request, and wait until a VG with a specific
lock type is started to determine the lock manager to use.
Detect also presence of 'vdoformat' tool.
Fallback to 'kvdo' modprobe only when dm-vdo fails
(removed ugly error message from log)
Also add extra check for scsi model being present
so the test can wait a bit if 'scsi_debug' takes longer time.
No need for 'aux' within aux.
Refactor existing code from tools/lvmcmdline.c to
libdaemon/server/daemon-stray.h daemon_close_stray_fds()
used to close stray descriptors above some specified Fd.
This is code parses content of /proc dir to minimize 'blind' closing
of all possible descriptors within rlimits range.
As we have the same code in few other places in it's more 'trivial'
version - these were actually sensitive to high amount of descriptors,
which might be configured on some system.
With this patch we effectively resolve this reported gitlab issue:
https://gitlab.com/lvmteam/lvm2/-/issues/5
TODO: Current placement might not be ideal - however considering
existing code base constrains it's not so simple.
ATM it uses lib/misc/lvm-file.h for custom_fds declaration
and rest of functinality is included in daemon header file.
Let's move proc into include/configure.h so this define can
be easily used across the source base.
Configure defines it - but ATM we do not provide any configure
option to change it - there should be no need to ever change it.
Handle the case, where we 'kill -9' running dmeventd,
and restarting 'dmeventd -R' manages to still contact this
'zombie' damone and manages to get list of monitored devices
out of this and eventually 'run' new and in this case
unexpected instance of dmeventd.
Some test do expect event_activation to be set.
So add explicit configuring of this setting in tests,
but also add new default which kind of does it globally
as it's expected default (yet our testing rpms might
be create with disabled event_activation)
By adding this to each test individually - it's now easy
to locate such tests...
Move static to main() local vars.
Initilize libdm logging also before starting _restart_dmeventd()
so function there can be also logged.
Move call of _info_dmevent() out of option processing - so some
options like -V are processed with higher priority.
Use _restart_dmeventd() with return values 0,1,2.
Also let's use already created fifos struct.
Make sure the _systemd_activation variable is properly initialized
from _systemd_handover before calling _restart_dmeventd() as
this variable is used there to decide where 1 or 2 should
be returned - aka either letting systemd to initilize
or restart dmeventd itself.
This is initial test how to disable event_activation on
machines, where lvm2-testsuite packages are installed
with its lvm.conf file.
TODO: find more elegant mechanism
Occasionally this test fails as soemtimes UUID actually
may constain LV[d] string causing it to grep mismatch
UUID and LV name and eventually fail test for wrong reason.
As a simple workaround print the LV name first and
check the name is followed by a space character.
This code is somewhat complex and involves recursion and pointer
shuffling which confuses coverity here.
Let's add masking comment for this warning as there is no double
free in this code.
Let's stop Coverity thinking here we are using freed FILE*
for anything else then comparing numbers.
For this use the original source of old_stream pointer.
Add new configurable option for building lvm2 with enable/disable
default autoactivation setting.
Might be useful for building i.e. rpms for systems where
this event_activation is not desired.
Parse timestamps included in kernel kmsg line and add current system
time to the messsage as well - this makes it easier to look over,
when chasing some messages in journal after some time...
Before initiation fifo communication, check whether there is
running dmeventd - in case there is no running instance, this
would be just blocked for 5 seconds trying to open fifo.
Also awk got annoyed by this \# char sequence which is reported
as incorrect, however older rpm builder were failing without this.
So let's just try other variant.
Thin-pool and cache-pool targets got already quite stable so let's try
to remove checking of pools when using lvremove or vgremove commands.
This skips checking pools when they are going to be removed, but it
also when removing thin volume that was the only user of a thin-pool.
In this case thin-pool will be still there and could be activated
again with another thin volume and thin_check will be executed
in this moment. In this case it can delay discovery of metadata damage.
Shortens processing of 'lvcreate -L -V -T' command and
avoid deactivation and its activation with thin_check of the empty
created thin-pool that will be used for the new thin volume
made with a single lvcreate command.
Shuffle code to parsing VDO message also for lvs segment status
so it can report correctly data usage for VDO LVs.
For this change move code and also change its API to use just mempool.
Fixes usage with upstream 6.9 vdo target driver.
When using message API for parsing VDO stats info, 0 was wrongly
used for fallback for trying the old sysfs API.
Switch to use ULLONG_MAX for values that could not have been obtained
through the message call.
Fixes lvdisplay info for freshly created VDO volume with 0 used data
blocks.
Make a seperate function to decode which ID should be user
for cvol meta or data volume - also avoids duplication of code.
As a result it's now also easier to see how the lvid is build.
Fix cutting away 1 character via incorrect usage of dm_strncpy
introduced in last batch of commits and use sizeof(buffer) to
get proper sizes.
In case of use strcpy_name_len() the replacement was invalid,
so we need to restore this case since sanlock uses buffer without
nul ending, so we would strip one more character from the buffer.
Also start to use dm_strncpy() without (void) for unchecked usage.
Add internal inline function wrapper for dm_strncpy().
Use it for calls where we test the result.
Avoids emitting warnings in Coverity for unchecked usage.
This sanity check actually confused in some way Coverity
giving it some assumption about array access.
Since these two checks basically validated compiler's capability
to add and then substract the number from char pointer we likely
don't really need them - as if this base math would not work,
compiler would be having far more troubles...
So drop them - and get rid of report:
Event overrun-call: Overrunning callee's array of size 513 by...
Reduce #lv and intervals.
We are trying to ensure that the daemon stops while it's busy processing
its internal queues. Decrease the number of LVs and intervals to allow
this test to complete in less time.
Add a global timeout value to be used for the threads to end waiting for
whatever it is they are blocked on. The values varied from 2-5 seconds,
which is way longer than needed. Value of 0.5 shows no CPU load when
service is running and is idle.
Shuffle code to avoid using static variable to pass parsed option.
Code is now easier to follow and also number of coverity reports
will go away.
There should be no functional change.
Add debug tracing for syscall failures.
Also switch some log_error to log_warn when command does not exit
with 'error' result and only warns user.
Easier error path handling.
Initialize some vars at declaration time.
Here we actually need to slowdown only $dev2 - since repair operation
is only reading data from this device and compares it with origin $dev1,
and if they match there is no write...
Move memset() to the initialization function define_commands().
There is also not much point in clearing memory on command's exit
so drop zeroing of ~2M of RAM.
Use \0 as EOL in compiled-in syntax description to avoid
unnecessary line copy that just replaced \n with \0.
Also use already splitted lines when possible.
Handle mismatch of reported 'dm raid' status, where the active
raid LV can be actually showing higher numebr of raid leg devices,
that the number of shown status characters.
This can happen if the raid leg is dropped during initial
resynchronization.
Setting db_persist is required for dm devices so that their properties
are carried over on switch-root from the initrd to the rootfs. This
logic has always lived in dracut
(https://github.com/dracutdevs/dracut/blob/master/modules.d/90dm/11-dm.rules).
However, this means that other initramfs generators each have to
implement and maintain the same rule which leads to unnecessary
duplication.
Instead, let's make the rule part of the upstream lvm rules, which
will ensure that generated initramfses will just work if they make
sure the lvm udev rules are installed, without having to figure out
that they have to add an extra rule themselves on top.
Identical rule in Arch Linux's lvm2 package: https://gitlab.archlinux.org/archlinux/packaging/packages/lvm2/-/blob/main/11-dm-initramfs.rules?ref_type=heads
For DM devices, the add/change/remove can appear as action for genuine
udev events.
However, there are more action types (bind, unbind, move, online, offline)
which never appear as actions for genuine DM udev events, but they can
still be synthesized (e.g. by writing "<action>" to "/sys/.../uevent" file
or by calling "udevadm trigger --action=<action>").
Let's also process these extra action types so that the udev-related content
is not lost completely, keeping all the symlinks and udev db entries just like
this was a synthetic udev event with "change" action.
Related to https://gitlab.com/lvmteam/lvm2/-/issues/4.
Bump the rules version in order to indicate that upper level rules
should consume DM_UDEV_DISABLE_OTHER_RULES_FLAG rather than DM_NOSCAN
and DM_SUSPENDED.
Also update the comments at the top of the file that describe the
exported properties, and add a note about internal device-mapper
properties.
Signed-off-by: Martin Wilck <mwilck@suse.com>
Reviewed-by: Peter Rajnoha <prajnoha@redhat.com>
DM_NOSCAN is not an official API any more and doesn't have to be
restored from the udev db. Rename it to .DM_NOSCAN.
Signed-off-by: Martin Wilck <mwilck@suse.com>
Reviewed-by: Peter Rajnoha <prajnoha@redhat.com>
DM_SUSPENDED is a device-mapper internal flag, which is not intended to be
used by other rules, and which is determined by 10-dm.rules from sysfs for
every uevent. Rename it to ".DM_SUSPENDED", so that it won't be saved in the
udev database.
Known consumers of DM_SUSPENDED are 66-kpartx.rules (from multipath-tools) and
99-systemd.rules (from systemd). These will have to be adapted.
11-dm-mpath.rules will be changed to use .DM_SUSPENDED.
Signed-off-by: Martin Wilck <mwilck@suse.com>
Reviewed-by: Peter Rajnoha <prajnoha@redhat.com>
DM_UDEV_DISABLE_OTHER_RULES_FLAG is used as the "output" flag of the
device-mapper rules, to be consumed by non-dm rules. It is a logical OR of
several conditions that might make dm devices inaccessible. 10-dm.rules
calculates it for every uevent, whether it's genuine or spurious.
DM_SUBSYSTEM_UDEV_FLAG0 is just another flag that needs to be or'd in. We
don't need to restore the previous state of DM_UDEV_DISABLE_OTHER_RULES_FLAG.
Actually, doing so is wrong if the flag has previously been set because the
device was suspended, and the device isn't suspended anymore.
Signed-off-by: Martin Wilck <mwilck@suse.com>
Reviewed-by: Peter Rajnoha <prajnoha@redhat.com>
We use DM_UDEV_DISABLE_OTHER_RULES_FLAG to tell upper non-DM layers
to keep their hands off the device in question, for any reason.
One possible reason is that the device is supended; another is that
the cookie carries the flag of the same name.
DM_SUSPENDED is not restored from the db, but evaluated anew for every
uevent. Therefore DM_UDEV_DISABLE_OTHER_RULES_FLAG shouldn't be
restored, either. Use a new variable DM_COOKIE_DISABLE_OTHER_RULES_FLAG
to save and restore the original value from the cookie.
Signed-off-by: Martin Wilck <mwilck@suse.com>
Reviewed-by: Peter Rajnoha <prajnoha@redhat.com>
DISK_RO is set in the environment of a block-device uevent if and only if
the read-only (ro) attribute of the device just changed (the kernel
function set_disk_ro() was called). It is not synoymous with the "ro" sysfs
attribute; the device could very well be write-protected if DISK_RO is not
set. Device mapper-level probing is possible for DISK_RO events, but it makes
little sense, because the device propreties haven't changed as far as dm is
concerned. But we should import possible previously set device properties
to avoid confusing follow-up rules. We should do this for both DISK_RO=1
and DISK_RO=0 events.
Signed-off-by: Martin Wilck <mwilck@suse.com>
Reviewed-by: Peter Rajnoha <prajnoha@redhat.com>
ID_FS_TYPE is the most important udev property for most follow-up
rules. It must be imported from the udev db if blkid can't be run.
Signed-off-by: Martin Wilck <mwilck@suse.com>
Reviewed-by: Peter Rajnoha <prajnoha@redhat.com>
When using cached LV with cachevols (so not with cachepool),
the loaded table could have been using more then one mapping line
for sub devices - resulting into data corruption in some cases
when i.e. taking snapshot of such cached LV with and instead of
single line - 2 lines were generated into DM table as the code
skipped protection again repeated addition.
vg-fast_cvol-cdata: 0 16384 linear 253:2 16384
vg-fast_cvol-cdata: 16384 16384 linear 253:2 16384
New code is also refactoring to use _add_new_cvol_subdev_to_dtree
(similar _add_cvol_subdev.. ) and also the addition of subdev has
been moved after check for already processed node.
Also the cachevol sub devices are now added with the insertion
of cachevol with cached LV.
Document usage of chaining external origins as it is now possibly to
create thinA LV in poolA which use thinB LV in poolB as it external
origin and user could create chain of such LV.
When using swap_lv_identifiers() we were basiclly exchanging 'names'
and only according to the caller some more data were 'transfered'.
However in most cases we should swap properties like 'hostname' as
the creation information should be preserved.
So let's do the function more universal.
Improve support for building DM tree when there is a chain
of external origins used for LV.
For this we cannot use track_external_lv_deps as this works
only for LV with just one external origin in its device tree.
Instead add directly 'dev' to the instead of add whole LV.
This avoid possibly recurive endless loop, however we may eventally
have some problems with undiscovered/missing devices in DM tree.
When we have some existing LV and this LV is being converted to
external origin - during the DM table manipulation there is a short
moment when the LV is being 'resumed' as 'read-only' volume
while still being live as 'rw' volume i.e. we could have had
a single thin LV active twice.
To avoid such weird scenarios of dual access to a same volume, we
just postpone a resume until a moment, where the existing volume
is already suspended thus no I/O can be in flight to such device.
Note: however there is slight catch - that we now have basically
a different 'risk' case where a resume of such i.e. new external
origin LV might fail and we are already in suspend tree state -
resolving error path in this situation is untrivial as well...
Fix/support creation and usage of the external origin
across thin-pools - so thin LV can use thin LV from
some other thin-pool as external origin (read-only).
When creating external origin via 'lvcreate --type thin'
add the validation for LV being usable as external origin
since certain LVs cannot be really used this way.
Also call this function early during lvcreate cmdline arg
validation se we do not need to do unecesary operation.
Over the time the code for preloading detached LVs got unnecessarily
complicate. But actually we need to preload only LVs that
were previously non-toplevel (invisible) LVs and became visible
toplevel LVs in the precommitted metadata.
If there would be needed some other rule, it would likely be a bug in
conversion code forgetting to set visibility flag on detached LV.
This reduces number of unnecessary repeated DM tree preloading.
When building in other dir, ensure srcdir is used to find helper
script (via vpath) through $< usage.
Add also missing [INSTALL] prints for some installed files.
External origins for thin volumes can be also used at the same time
as old(thick) snapshot origins. However in this case it's possible
the LV is only active as being 'external' origin, but old snapshot LVs
are not active. For this case before handling these
LVs for un/monitoring check the active state of origin LV.
This should prevent warnings of monitoring failures.
Make recursive directory path creation reusable via
dir_create_recursive.
While we already have dm_create_dir() - it's not taking mode arg,
so let's make lvm's internal file helper function.
Instead of parsing the whole /proc/kallsyms use faster variant
of using modprobe tool logic.
lvm2 here wants to know whether the particular DM cache policy is
present in the kernel - however since the cache policy does not have
any kernel module parameters and it can be built-in to a kernel
there is no /sys/modules directory in such case and we would need to call
modprobe everytime we want detect such case.
The old solution tried to look for particular kernel symbol
(and like not the right way, as smq_exit might be actually ommitted).
New version checks MODULES_PATH/`uname -r`/modules.builtin for
whether is present cache policy module instead of CPU expensive parsing
of kallsyms.
If lvm.conf has use_devicesfile=0 and /etc/lvm/device/system.devices
exists, then rename it to system.devices-unused.YYYYMMDD.HHMMSS.
This prevents an old, incorrect system.devices from being used in
the future if lvm.conf is changed to use_devicesfile=1.
Create backup copies of system.devices in /etc/lvm/devices/backup
named system.devices-YYYYMMDD.HHMMSS.NNNN. NNNN is the version
counter from the file.
Each time that an lvm command writes a new system.devices file,
it also writes the same file in the backup directory.
A new comment line is added to system.devices with HASH=<num>
where <num> is a crc calculated from the uncommented lines in
system.devices. This lets lvm detect if the file has been
modified outside of lvm itself.
If system.devices is edited directly, the next time a command
reads the file, the crc will not match the HASH value. The
command will then rewrite system.devices with the correct HASH
value, and create a backup reflecting the edits.
A default limit of 50 backup files is kept, configurable by
lvm.conf devicesfile_backup_limit (set to 0 to disable backups.)
If LVM LVs happen to contain PVs, they are passed to the lvm udev
rule for processing, where they should be ignored. PVs on LVs
most likely belong to VM images, and don't belong to the host
which sees the LV. It's unsafe for the host to use these PVs.
Without this change, the LV would be processed by pvscan which
would generally ignore it, either because of the devices file,
or because of the default lvm policy to not consider LVs as
potential PVs. This change makes the udev rule consistent
with that policy and avoids the unnecessary system messages
produced when pvscan ignores the LV.
Since understanding the reason for choosing the appmachineid over the
direct use of machineid is not easily found, I extended to help text to
clarify this a bit.
Function to recalc chunk_size according to dev hints needs to be
used after chunk_size is being set to thin pool segment - correct
this ordering mistake introduced in previous refactoring commit.
Previous patch that introduced support for thinpool with vdo
not correctly handled header size - as this part is not fully usable
yet. We are going to try to use the 0, but current state of code is not
yet compliant to this logic so keep vdo_header_size during conversion
and alos correctly pass through virtual_extents to vdo formating.
Add code to handle creation of thin-pool with VDO data backend
which can be seen as compressed deduplicated thin-pool.
To avoid need of changing to many internal APIs, pass the conversion
parameters for create thin-pool data volume via cmd_context.
Introduce struct vdo_convert_params {} to pass-in all the parameters
needed for the conversion of an LV to a vdopool + vdo LV.
Function convert_vdo_lv() is also able to create a new LV and swap
segments, so the passed in LV can be later on use for futher
conversion so this refactoring makes it ready for more enhanced
usage.
Introduce vdo_convert_params and use vdo_params from this structure
also with lvcreate_params.
Later we will use this for convertion of thin-pool data volume to VDO.
If a RaidLV mapping is required to be refreshed as a result of temporarily failed
and recurred RAID leg device (pairs) caused by writes to the LV during failure,
the requirement is reported by volume health character r' in position 9 of the
LV's attribute field (see 'man lvs' about additional volume health characters).
As this character can be overlooked, this patch adds messages to the top
of the lvs command output informing the user explicitely about the fact.
Fix typos in previous commit 3589e515d.
Both commands default [raid_](min|max)recoveryrate to 0 but ensure
min_recovery_rate is not larger than max_recoveryrate. This results
in command failure without requesting the user to also define
max_recovery_rate >= min_recovery_rate.
Fix both commands by defining max_recovery_rate = min_recoveryrate
in case "lvcreate/lvchange --minrecoveryrate Size ..." requests a
larger value than current maxrecoveryrate without also giving option
Both commands default [raid_](min|max)recoveryrate to 0 but ensure
min_recovery_rate is not larger than max_recoveryrate. This results
in command failure without requestinng the user to also define
max_recovery_rate >= min_recovery_rate.
Fix both commands by defining max_recovery_rate = min_recoveryrate
in case "lvcreate/lvchange --minrecoveryrate Size ..." requests a
larger value than current maxrecoveryrate without also giving option
"--maxrecoveryrate Size ..." with a size greater or equal than min.
The first lv_attr flag is 'i' or 'I' for a raid image.
(i: raid image, I: out of sync raid image)
For integrity raid images (_iorig), the flag was not being set.
pvs -A|--allpvs
Show PVs that would otherwise be excluded by the devices file.
pvscan -A|--allpvs
Show PVs that would otherwise be excluded by the devices file.
For those devices that are included by the devices file,
their device ID is displayed in place of the usual "lvm2"
format and size.
(pvs -a|--all is unchanged, and shows devices not formatted as PVs.)
A pvid string read from system.devices could be less
then ID_LEN since system.devices fields can be edited.
Ensure the pvid buffer is ID_LEN+1 even if the string
read from the file is shorter.
Include info in the temp file to confirm that it should be used.
The temp file is meant to suppress repeated, identical searches
for the same PVIDs on the same set of devices. Write to the file
a count and hash of the missing PVIDs and a count and hash of the
devices to search. A subsequent command will ignore and remove
the temp file if any of these values differ. We don't want to
suppress a search if a change has occured, and a missing PV could
be found by scanning devices.
Problematic scenario:
. the device for a PV has no wwid, so it's identified in system.devices
with IDTYPE=devname IDNAME=/dev/foo
. user adds/enables a wwid for the device
. on reboot, the device name changes, e.g. now /dev/bar
. the code that searches for the new device name includes an
optimization to skip looking on devs that have a wwid, on
the basis that a device with a wwid won't have IDTYPE=devname
. this optimization causes lvm to not look for the PV on /dev/bar
since that device now has a wwid, so the PV is not found
. the optimization is enabled by search_for_devnames="auto"
. change the default to search_for_devnames="all" which does not
use the problematic optimization
- add new comparison between old and new entries, and use this
as the basis for new dedicated output for check and update
- add new --refresh option to search for missing PVIDs on all
devices, and possibly update the device ID
- internally, only use the term "refresh" for cases where a
new device ID may be found and assigned for a missing PVID
During vdo testing with smaller block devices the test needs to determine
which PVs in a VG are unused. This was leveraging LvCommon.Devices, but
that isn't populated when a LV is resides on a LV pool instead of a PV.
This is a known limitation of the code at this time. For now we will walk
all the PVs in the VG looking for ones that don't have any associated LVs
and use them instead.
Signed-off-by: Tony Asleson <tasleson@redhat.com>
When getting raid status from some older kernels, we may get an 'a'
for 'A' leg when doing initial synchronization.
This may prohibit removal of newly synchronized leg until synchronization
is finished.
So in this case change the status to look like being reported
from a newer kernel version.
Incorrectly matching a dev to a devname id (due to changing devnames)
before matching the dev to a proper device id, can result in the
dev not being matched to the real id.
When reporting in JSON format, we need to be able to find the 'last
displayed row', not just 'last row' as we did before. This is used
to decide whether to put the JSON_SEPARATOR (the ',' character)
between the lines when reporting in JSON format.
This is mainly important in case we use a combination of JSON format
and a report marked with DM_REPORT_OUTPUT_MULTIPLE_TIMES flag.
Such report may be reused several times with different selection
criteria each time. In that case, the report always contains all lines
in memory, even though some of them do not need to pass the selection
criteria that are currently used.
Without DM_REPORT_OUTPUT_MULTIPLE_TIMES flag, the report only contains
the lines that have passed the selection criteria, so the this wasn't
an issue in this case.
Fix suggested by Lars Ellenberg and reported here: https://github.com/lvmteam/lvm2/issues/130
It looks like there is some kernel bug/limitation
that may cause invalid table load processing:
dmsetup load LVMTEST-LV1
device-mapper: reload ioctl on LVMTEST-LV1 failed: Invalid argument
md/raid:mdX: reshape_position too early for auto-recovery - aborting.
md: pers->run() failed ...
device-mapper: table: 253:38: raid: Failed to run raid array (-EINVAL)
device-mapper: ioctl: error adding target to table
However ATM there is not much we can do then make delays bigger.
TODO: fixing md core...
With commit d7e922480e
lvconvert -m may fail if we try to remove 1st. leg that
is out-of-sync while other leg is in-sync.
Hot fix allows to proceed with such down conversion.
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Fixes commit 63b469c160
"device_id: fix hints with device ids"
It's not correct for internal filtering to be a factor
in validating the set of devs that are the basis for hints.
Instead, look for inconsistencies between a hint entry and
the corresponding devices file entry.
Extend the test a bit futher so we can keep logic of resize
working similarly well for older and newer systems.
Test uses new 'aux have_fsinfo'function to regnize compiled
version of lvm.
Apply the same logic for 'lvreduce' which exists for newer
systems (compiled with HAVE_BLKID_SUBLKS_FSINFO)
also for older systems for one very common practical case where
the active LV does not have any blkid known signature/filesystem.
New variant recognized this situation and allowed to proceed
without requesting a prompt, while the older variant always
requested confirmation prompt.
With this patch command now works equily for both variants
for 'active LV' without signature and allows to reduce LV
without prompting.
Simplify configuration with enabling options:
- For --enable-dmeventd automatically --enable-cmdlib.
- For --enable-dbus-service auto --enable-notify-dbus.
Fix check linux/fiemap.h for dmfilemapd.
Before checking seg_type of the first area, check there is
some existing area.
Since we now support error and zero segtypes, these do not have
any PV area present.
When new app links with current dm_task_run() that propagates ioctl()
errno - ensure it will not be usable with older version without this
supported feature.
Introduce couple configure options:
--without-blkid
--without-systemd
--without-udev
These can be useful for build with --disable-shared
where those libraries are often distibuted without
their static libraries.
However such compiled static binaries then have limitted usage.
Since LVM 2.02 it appears that compiling LVM on a platform which
lacks a shared library linker/loader will fail to produce any
binaries (dmsetup, lvm2, etc).
Adds support for the standard --disable-shared flag, and
use it to disable attempts at building shared libraries.
Modified-by: zkabelac
Fix a bug in _stats_set_aux() that causes bogus data to appear
in the 'userdata' field of stats reports when previously grouped
regions are ungrouped:
/var/tmp/File With Spaces: Created new group with 1 region(s) as group ID 0.
Removed group ID 0 on fedora-root
Name GrpID RgID ObjType RgStart RgSize #Areas ArSize ProgID UserData
fedora-root - 0 region 6.39g 100.00m 1 100.00m dmstats #-
^^
This is the aux_data separator character followed by empty user data.
The _stats_set_aux() function should only emit the separator if
there is a valid group descriptor for the region.
Creating a group with a name that contains whitespace causes an error in
the @stats_set_aux message:
device-mapper: message ioctl on (253:0) failed: Invalid argument
Could not create regions from file /var/tmp/File With Spaces.
Fix this by quoting the group alias and backslash escaping any
embedded space characters.
Refactor the function that parses regions from @stats_list data to
separate the parsing of variable string data from the fixed parts
of the @stats_list response.
Fix a double free in the error path from _stats_create_group() by
clearing the group struct embedded in the dm_stats handle before
returning:
device-mapper: message ioctl on (253:0) failed: Invalid argument
Could not create regions from file /var/tmp/File With Spaces.
free(): double free detected in tcache 2
Aborted (core dumped)
When setting up dm-verity devices with signed root hashes it is very
useful to have a recognizable error code when a key is not present in
the kernel keyring. Turns out the kernel actually returns ENOKEY in that
case, but this gets lost in libdevmapper.
This fixes this: in _create_and_load_v4() it copies the error code from
the ioctl from the sub-tasks back to the main task field on failure.
This is not enough to make libcryptsetup actually propagate the ENOKEY
correctly, that also needs a patch to libcryptsetup, but this is part of
the puzzle.
Fix some interactions between device IDs and hints. Hints
may limit the scanned devices which should not always trigger
a search for the PVs that were intentionally not scanned.
Hints should also be invalidated if they contain a device
that's become excluded by an internal filter such as the
device_id filter.
Search for a PV on other devices if it's a devname entry
and the name doesn't exist on the system. This restores
code that should not have been removed in commit 1901a47df
"device_id: fix conditions for device_ids_refresh"
throtling mirror device is becoming useless with faster CPUS,
as way to many data can be transferred before throttling steps-in.
So prefer using dm-delay for test and keep throttling as fallback.
Fix commit 847f1dd99c
"device_id: rewrite validation of devname entries"
which began calling device_ids_refresh() in cases where it
was unnecessary, leading to extra PV searches and warnings.
Specifically, a command like "lvs <vg>" would use the hints
file to scan only devices for the named VG. This means that
scanning other PVs would be skipped, and device IDs of those
PVs could not be validated because there are no PVID values
to verify. This missing info would cause messages about
the missing info, and would cause device_ids_refresh to
search for the PVs that had been intentionally skipped.
Detection of how the command is supposed to behave actually depends on
the configure.h compilation and whether binary is compiled with
HAVE_BLKID_SUBLKS_FSINFO.
This makes it somewhat complicated in a way how to recognize which
behavior is expected.
Currently we can eventually recognize by checking error output
of some 'random' lvresize command and see if the --fs checksize is
actually recognized and rejected. If this changes - test needs
to be updated.
After umout we may race with system udevd rule - so
just retry once again after 1s sleep - that should be
enough - otherwise we would need some loop here...
Multi-line echo command are problemat across variety of bash version
and may have produce shorter results.
Convert to stable heredoc string with 'tab' skipping <<- for better
formating.
Since we now push more data into journal, parser reading this file
for --continue mode need to be adapted.
Also properly align batch mode with '.' for max test length name.
This is mainly useful in internal testing - but keep sysfs dir also
passed to filter.
Also drop use of static variable within sysfs filter and base whole
config at creation time.
Restore fsync() call For more accurate tracking by buildbot.
Try different rather tricky way of static_cast to use
already opened FD instead of seperate open(),fsync(),close().
It's pretty strange there is no way to enforce fsync() for
C++ iostreams. Flush() is actully not equal.
Add Timespec class to increase time resolution to miliseconds
(can switch to microseconds if ever needed).
Use more const and const_interators and pass by reference.
Output rusage also to list result file.
Reduce inlining of C++ constructors.
If the system changes, locate PVs that appear on different devices,
and update the device IDs in the devices file. A system change is
detected by saving the DMI product_uuid or hostname in the devices
file, and comparing it to the current system value. If a root PV
is restored or copied to a new system with different devices, then
the product_uuid or hostname should change, and trigger lvm to
locate PVIDs from system.devices on new devices.
When exit on file is present in a system and term/break signal is
catched, them dmeventd is no longger refusing to exit.
For the correct shutdown, there should be ideally unmonitoring call,
however in some case it's very hard to implement this correct procedure.
With this 'exit on' file dmeventd at least avoid 'blocking' shutdown,
before systemd kills use with -9 anyway possibly even in some unwanted
stated of internal dmeventd processing (i.e. in the middle of some lvm
command processing).
To quickly get info about some internal dmeventd status,
implment 'dmeventd -i' support.
Reported messages are some 'raw' internal informations mainly
useful to developers.
Instead of just exiting in the middle of monitoring,
unregisted all monitored devices first and then exit.
To speedup this path, all send internal SIGINT when thread
unregiters itself, to wakup-up main sleeping loop.
Make a local copy of the 'idx' string to avoid
overlapping during the rebuild of name.
This fixes cases where users specified raid
component LVs for moving.
Reported-by: kotarou3@github.com
No write outside of $LVM_TEST_DIR (removed /test access).
Use 'aux prepare_scsi_debug_dev' for automated scsi_debug handling
Properly use "" around shell vars.
Smarter read of PVS values.
Relax requirement to only work with real /dev dir.
Handle the case of device teardown where the first pass
could have only a single, but opened device, for removal.
In such case we want to at least once go through
the udev_wait and retry removal again.
TODO: maybe a sleep .1 might be usable as well with udev_wait
Instead of relying on 'pvs' output - check directly system
configuation and use lvmdevice accrording to use_devicesfile setting.
Also drop use of --fs ignore for filesystem extension for better
backward compatibility with older lvm version.
Shuffle code a bit so the '--no-snapshot' path does not execute
a few unnecessary commands.
Work also with devices that may have ':' inside their generated
/dev/disk/by-id
Ensure there is no race with systems' auto activation while using
the snapshot for conversion.
Update system's vdoconf.yml after the use of snapshot for conversion.
Skip unnecesary prompt for 'convert' while using snapshot and query only
for final snaphot merge.
Prohibit conversion for a device with the PV header.
Enhance 'trap' protection for more signals.
Improve clean() recovery path.
Replace bash 'test' command with [].
Correct some output message to print $TOOL.
Support also options without '-' in the middle i.e. --nosnapshot.
For shellcheck predefine all variables extracted from vdoconf.yml.
Since lvmdbusd testing tends to do its own logging,
try for now to disable very generic logging mechnanism
of the test suite and see the result.
Some lvmdbusd test seems to rely on some log/file logic
which is modified with the use of these shell vars.
Require VDO version 6.2.3.
Skip the part of the test that needs vdo wrapper and 2 different
versions of vdoprepareforlvm to prepare shifted VDO header
at the 2MiB offset.
Hunt also for devices with LVMTEST prefix in UUID.
Call teardown_devs_prefixed - so if they hold RAM or SCSI
they are closed before trying to remove kernel modules.
When creating thin pool or check pool there is allocated LV
for metadata and for such allocation user should be able to
specify list of PVs on cmdline.
Also fix unused passed list of PV for thick to thin conversion,
where the code was using whole PV set from a VG (but since it's
been not enabled on cmdline, user could not hit this issue).
Also remove unneeded initialization of use_pvh.
When the import is used on a system, that uses devices file,
the final activation was impossible for the case the converted
volume was not present in devices file.
Currently add volume automatically in such case.
Also add some more debugging output from the script.
TODO: Consider enhnacing lvconvert with extending devices file.
VDO is using specific path for some device paths.
i.e. for /dev/sda it could be /dev/disk/by-id/scsi-xxxxx.
This used to be not a problem before lvm2 started to use snapshot,
but now it needs to replace matching device path.
So switch to the path naming used in vdoconf.yml file.
Add '--headings none|abbrev|full|0|1|2' command line option to select
the heading type.
none|0 - no headings
abbrev|1 - column name abbreviations
full|2 - full column names (column names are equal to exact names
that -o|--options also accepts to set report output)
Reuse existing report/headings config setting to make it possible to
change the type of headings to display:
0 - no headings
1 - column name abbreviations (default and original functionality)
2 - full column names (column names are equal to exact names that
-o|--options also accepts to set report output)
Also, add '--headings none|abbrev|full|0|1|2' command line option
so we are able to select the heading type for each LVM reporting
command directly.
Add new DM_REPORT_OUTPUT_FIELD_IDS_IN_HEADINGS report output flag.
If enabled, column IDs are reported instead of column names in report
headings.
The 'column IDs' are IDs as found in 'const struct dm_report_field_type *fields'
array that is passed during report initialization (that is, a call to
dm_report_init/dm_report_init_with_selection). In this case, the 'id'
dm_report_field_type member is used instead of the 'heading' member.
While create new LV for pool volume, use name from 'pool_metadata%d' naming
sequence. This LV is later on renamed to pool_t/cmeta, but if there
is any error in the middle, we may evenutally leave some 'volume',
With this name it can be slightly more obvious how it got there,
but also when we handle _pmspare name - we get slightly more predictible
name used there for it.
However for a standard usage this commit shall no visible impact as the
name is used temporarily just for cleaning LV.
Set the lock_args string in addition to doing initialization.
lvconvert calls lockd_init_lv_args() directly, skipping
the normal lockd_init_lv() which usually sets lock_args.
lvmlockd locking for swapmetadata adjusted for commit
ac36153e99 lvconvert: preserve UUID for swapped metadata
Now that the LV uuid is swapped between LVs, the lvmlockd lock can
simply be moved between them, and the same lock can continue to be
used for the LV outside of the pool.
Once lvm2 repairs pool's metadata LV and preserves the original metadata LV
with unmodified metadata, for such LV in VG use newly created UUID for new
_pmspare and actually preserve UUID for this hidden _pmspare (if it
exists).
The lockd lock needs to be freed for the LV that is becoming
the new metadata LV, and a new lockd lock needs to be created
for the old metadata LV that is becoming an independent LV.
Fixes b3e45219c2
We incorrectly marked pv_major and pv_minor fields as being of string
type, even though the values were already correctly handled as integers
internally. This confused -S|--select that tried to compare string
values instead of integers.
Reported here: https://github.com/lvmteam/lvm2/issues/122
Previous fix was invalid (after some in-place shuffling)
'dd' copied goes to 'stderr' so we need to catch all output.
Grep needs to check output of tee tool.
Ensure 'C' locales are in use with 'dd'.
Use cachepool name for create name for metadata backup LV.
(so we do not generate 2 'sequences' of metadata filenames.)
Move path preparion before handling _pmspare.
Also drop extra call to sync_local_dev_names() as it's
already got in sync with call of exec_cmd().
When repair thinpool or cachepool, lvm2 leaves original metadata
volume backup. To avoid potential damage of those data, mark such
volume as 'read-only' and also allow user to use --setactivationskip
option for this volume.
TODO: likely better default would be to automatically skip, but
that might need some more thinking about recovery reporting doc.
Replace the use of internal /dev/mapper names with the use of
public LV names /dev/vg/lv for use with repair tools.
For this make the activation of _pmspare LV to be handled as
a component activation with public name.
Metadata is already atomatically activated this way (as readonly).
So if there is any 'error' happening, we leave public LVs in
system.
Pass more args with some 'aux' commands:
wipefs_a, enable_dev, disable_dev
(so it's a bit more efficient using single udev_wait call).
Use prepare_vg instead of prepare_pvs.
Keep using test directory for created files.
Trap errors and remove brd in this case.
Use some shell builtins to reduce fork count.
Use "$VAR".
Run 'pvs' with devlist (so not acceing other system devices).
New dmpd tools return version string in different format,
so update code to understand both variant.
Also hide some shell var setting to local functions.
lvresize --usepolicy requires resized LVs to be active.
(So it's not only required for shared VG).
The test for active pool needs to use lv_info to query 'layer'
otherwise the pool is considered inactive if it was not activated
explicitely - thun 'implicit' activation with VDO or ThinLV was
not managed by --usepolicy option.
Some linkers do need libdevmapper-event-lvm2.so.2.03,
so add also this symlink to the tests /lib dir.
This fixes the need to use previous LD_LIBRARY_PATH complex
setting and now works with much shorter list.
Last commit c38b668fc3 was pushed
with type 'scanf' instead of 'sscanf' needed for buffer reading.
Interestingly this caused scanning from 'stdin' descriptor and
thus failures in lvm shell usage.
The code in init_log_file relies on the process name (COMM) to not
contain whitespaces. This change fixes it by looking up the right-most
parenthesis to safely jump past COMM.
For more context see:
https://www.openwall.com/lists/oss-security/2022/12/21/6
Code is only used with testing, so it should have no impact on regular
users.
Reported-by: Hugues Evrard <hevrard@google.com>
mm
With 3596558861 it's been introduced
a more fine grained description.
However 'disabled' might be actually more confusing then empty field,
so keep only the info about 'not enabled'aka dmevend is not allowed
to monitor LV which otherwise could be monitored.
Update pool conversion function to handle also conversion of
thick LV to thin LV by moving thick LV into thin pool data LV
and creating fully provissioned thin LV on top of this volume.
Reworking existing conversion to use insert_layer_for_lv co
the uuid is now kept with thin-pool - this should however not
really matter as we are doing full deactivation & activation cycle.
With conversion to thin LV user can use same set of arguments
to set chunk-size.
TODO: add some smart code to decide best values for chunks sizes.
For proper functionality of insert_layer_for_lv we need to
move more bits to layerd LV.
Add some missing new types and correct usage of caller,
so the new LV type is set after the movement.
Validate cache origin in front of the prompt.
Also add some rules to command description file.
TODO:
more validation needed also for lvcreate,
more complex rules with "OR" seems to be needed.
Enable support to cache thin/error/zero virtual targets.
Use can now select whether he want to cache whole thin-pool,
or an individual thin volume out of whole thin-pool.
Support using zero and error LVs as data volume for
cachepool is possibly useful for benchmarking, not much
can be expected from such setup.
Avoid activation when going to skip zeroing of 'error' segtype
(so it's not erroring out).
Also skip zeroing for 'zero' segtype LV (already being zero).
When lvm2 calculates the maximal usable COW size and crops the user
requested size to this value, don't return the error result from
the 'lvextend' operation.
We already apply the same logic when resizing thin-pool beyond
the supported maximal size.
FIXME: The return code error logic here is somewhat fuzzy.
Output from vdo manager may actually indent output with spaces,
so trim leading and ending space.
Also add support for verbosity flag for vdo conversion tool.
This vdo parameter existed in the early stage of integration of vdo into lvm2,
but later it's been removed from vdoformat tool - so actually if
there would be any non-zero value it would cause error on lvcreate.
Option was not stored on disk in lvm2 metadata.
Remove this vdo parameter from lvm2 sources.
(Although this vdo parameter will be still accepted on cmdline through
--vdosettings option, but it will be ignored.)
Fix the testing logic.
With raid5 device the layout of files on a filesystem does not define
which leg will actually contain the block we try to damage.
So test will now figure out which device has damaged block.
Use 'check' functionality and also drop unneeded random write as we
now can identify easily in other way.
Compilation may configure it's own /etc path so ensure the test
has a defined location for access to this dir during testing.
Also prepare machine_id filei (with the use of uuidgen tool)
for the test.
Ensure the volume after conversion has the properly aligned size to the
volume group extent size. This would be visible when using virtual size,
that cannot be divided by extent size.
Before the user had to manually adjust the size after conversion to get
access to all data stored on VDO volume.
The command "lvcreate --type thin --snapshot ..." to create a thin
snapshot would fail.
commit d651b340e6 removed the optional
"--type thin" from the command definition "lvcreate --snapshot LV_thin",
and added --type thin as AUTOTYPE. This was correct and should not have
changed anything if all the command defs were correct, but it broke
the "lvcreate --type thin --snapshot" case. It reveals a problem in a
different command definintion: "lvcreate --type thin LV_thin" that was
missing --snapshot in its OO list.
If we want to have const structure - use a const pointer to avoid,
however making individual members const make it only assingle
in construct time and we already violate this logic since
we memcpy into the structure - so drop these unnecessary consts...
Add support for usage of 'dm-snapshot' for doing whole device conversion
in a snapshot which could be merged once the whole conversion has been
made.
This helps with cases where there would be any unexpected failure in the
middle of conversion process and user can continue using original
device until problem in conversion is fixed.
Import tool now uses 'truncate' tool to create a small backend file (20M) for loop device
to hold COW exception store.
Option --vdo-config has been added to allow specifing location of vdo
configuration file.
Option --no-snapshot allows to use import tool without creation of
snapshot, however recovery after finished VDO conversion and unfinished
lvm2 conversion is then very difficult.
Option --uuid-prefix allow to specify DM UUID prefix for snapshot
device.
Use read with -r.
Improve metadata parser to handle volume_geometry bio_offset,
which needs to be substracted from 'region' start_block when present.
This bio_offset block is non-zero i.e. with converted VDO volumes.
Also fix some converted structure value (but they are not in use).
Fix in the code that matches devices to system.devices entries when
the devices have the same serial number. A non-PV device in
system.devices has no pvid value, and the code was segfaulting
when checking the null pvid value.
If we are executing 'report_for_selection' to do an internal report
just for the selection itself (not for display), we can call it more
than once. In that case, we are reusing the same selection handle
(e.g. inside 'process_each_lv_in_vg').
The selection handle has 'report_type' field which is a union of all
report types needed for the report based on selection fields we actually
use.
The 'report_type' is further clarified based on checks and rules inside
'_get_final_report_type' which 'report_for_selection' calls. Then, this
final report type unambiguously identifies proper branch to take in
'report_all_in_{pv,vg,lv}' that is called next.
If the 'report_for_selection' is called more than once with the same
selection handle, we need to make sure that we always restore the report
type to its initial value, so all the rules inside 'report_for_selection'
are applied correctly next time.
This patch fixes the missing restoration of the 'report_type' value in
'selection_handle' that is reused for recurring 'report_for_selection'
calls.
An example scenario where this failed was with selecting an LV for
removal with "lvremove --select" while using a field in the selection
that required extra DM info or DM status call for the LV (that is,
"Logical Volume Device Info Fields" and "Logical Volume Device Status Fields"
as visible in 'lvs -S help').
In previous lvm versions, trailing spaces at the end of a t10 wwid would
be replaced with underscores, so the IDNAME string in system.devices
would look something like "t10.123_". Current versions of lvm ignore
trailing spaces in a t10 wwid, so the IDNAME string used would be
"t10.123". The different values would cause lvm to not recognize a
device in system.devices with the trailing _. Fix this by ignoring
trailing underscores in the IDNAME string from system.devices.
The recent fix 05c2b10c5d ensures that raid LV images are not
using the same devices. This was happening in the lvextend commands
used by this test, so fix the test to use more devices to ensue
redundancy.
vgrename does not support -S|--select, so do not provide a hint about
using it. Instead, provide a hint about using VG uuid directly.
❯ vgs
WARNING: VG name vg1 is used by VGs DXjcSK-gWfu-5gLh-9Kbg-sG49-dtRr-GqXzGL and MVMfyM-sjOa-M2xV-AT4Y-JddR-h4SP-UO5Ttk.
Fix duplicate VG names with vgrename uuid, a device filter, or system IDs.
VG #PV #LV #SN Attr VSize VFree
vg1 1 0 0 wz--n- 124.00m 124.00m
vg1 1 0 0 wz--n- 124.00m 124.00m
(vgrename does not support -S|--select)
❯ vgrename vg1 vg2
WARNING: VG name vg1 is used by VGs DXjcSK-gWfu-5gLh-9Kbg-sG49-dtRr-GqXzGL and MVMfyM-sjOa-M2xV-AT4Y-JddR-h4SP-UO5Ttk.
Fix duplicate VG names with vgrename uuid, a device filter, or system IDs.
Multiple VGs found with the same name: skipping vg1
Use VG uuid in place of the VG name.
(vgchange does support -S|--select)
❯ vgchange --addtag a vg1
WARNING: VG name vg1 is used by VGs DXjcSK-gWfu-5gLh-9Kbg-sG49-dtRr-GqXzGL and MVMfyM-sjOa-M2xV-AT4Y-JddR-h4SP-UO5Ttk.
Fix duplicate VG names with vgrename uuid, a device filter, or system IDs.
Multiple VGs found with the same name: skipping vg1
Use --select vg_uuid=<uuid> in place of the VG name.
In case of e.g. 3 PVs, creating or extending a RaidLV causes SubLV
collocation thus putting segments of diffent rimage (and potentially
larger rmeta) SubLVs onto the same PV. For redundant RaidLVs this'll
compromise redundancy. Fix by detecting such bogus allocation on
lvcreate/lvextend and reject the request.
Check for running (possibly leftover) lvmdbusd running in the
system - as this daemon may interfere with this test as in this
case both be operating on same 'live' data in /run/lvm.
Make test faster by agregating sets of operation to work on a single
created filesystem yet checking all the variants of extension and reduction.
Split 'xfs' part into separate test and convert it for use of the
minimal size 300M nowdays required by mkfs.xfs.
lvreduce uses _lvseg_get_stripes() which was unable to get raid stripe
info with an integrity layer present. This caused lvreduce on a
raid+integrity LV to fail prematurely when checking stripe parameters.
An unhelpful error message about stripe size would be printed.
When lvmcache info is dropped because it's an md component,
then the lvmcache vginfo can also be dropped, but the list
iterator was still using the list head in vginfo, so break
from the loop earlier to avoid it.
Previous commit cause the pvmove could actually be started in unexpected
order - so make sure, we are not starting new pvmove in same VG until
the previous one is started.
Keep backups within test_dir instead of dev_dir (so it doesn't
leak large files there if the tests are run over real /dev dir).
Move restoring of dm_mirror throttling before test_dir is removed.
Not quite sure if this helps anything, some of testing
machines can't reliably remove scsi_debug, reporting
they are in use - but it's not easily reproducible...
aux wait_pvmove_lv_ready() now handles multiple pvmove LVs
at one go - which allows a bit fast checking - although
at some point we may need to switch to use delayed devs
since mirror throttling seems to be no longer working well,
as CPU are getting so fast, that most of data are already
pvmoved before throttling has any chance to do something...
When 'brd' device can be removed (is unused AKA not opened),
remove such device and use again for testing.
Let's assume user has no unused brd device left in the system.
When the 'tests' sometimes fail to cleanup devices, with this
change futher cleanup from some next test may evenually release
brd device and make it available for testing.
There is no easy way to detect, whether device supports zeroing,
and kernel also zeroes device when it's not directly supported,
but with extra message:
operation not supported error, dev X, sector Y op 0x9:(WRITE_ZEROES)...
So to avoid generating such message with every 'lvcreate', use for
zeroing of upto 8K just standard write of zeroed page.
(maybe we can go with even larger sizes).
Convert test to use only ext4 instead of 300M demanding XFS.
Shorten 'B' files to 4K and use 4K strip size with >raid1 arrays
so we do not risk spreading of the file across stripe.
Also use easier 'aux corrupt_dev()' method to introduce a bit
corruption into a block device with integrity.
TODO: shorten _wait_recalc (should't be needed).
Add function to corrupt some bytes in give file path presenting
a device. 1st. patern in just once replaced with 2nd. pattern.
Usable to simulate some bit corruption for integrity devices.
Convert test to use a single skeleton and only different pieces
keep in separate tests.
Lower raid disk usage to smaller size and switch to ext4
as way less demanding fileystem.
Instead of using size of 'empty header' in vdopool use fixed size 4K
for a 'wrappeing' vdo-pool device.
This fixes the issue when user tried to activate vdo-pool after
a conversion from vdo managed device with 'vgchange -ay' - where
this command activated all LVs with 'vdo-pool' wrapping device as well,
but this converted pool uses 0-length header.
This 4k size should usually prevent other tools like 'blkid' recognize
such device as anything - so it shouldn't cause any problems with
duplicate indentification of devices.
Correcting output results from configure script when
printing locking dir.
Shuffle dmfilemapd close to dmeventd option and tracing.
Couple indent changes.
syscall 186 is specific to x86 64bit. As this is different from arch
to arch and between same arch different arch size we will only grab
thread ID using built-in python support if it is supported.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2166931
When the daemon is starting we do an initial fetch of lvm state. If we
happened to get some type of failure with lvm during this time we would
exit. During error injection testing this happened enough that
the unit tests were unable to finish. Add retries to ensure we can get
started during error injection testing.
Previously we were injecting a missing key in the lv, vg, and pv.
Given the order of processing in lvmdbusd, this prevented us from
exercising all the error paths. Change to returning just 1 instead.
When we sort the LVs, we can stumble on a missing key, protect against
this as well.
Seen in error injection testing:
Traceback (most recent call last):
File "/home/tasleson/projects/lvm2/daemons/lvmdbusd/fetch.py", line 198, in update_thread
num_changes = load(*_load_args(queued_requests))
File "/home/tasleson/projects/lvm2/daemons/lvmdbusd/fetch.py", line 83, in load
rc = MThreadRunner(_main_thread_load, refresh, emit_signal).done()
File "/home/tasleson/projects/lvm2/daemons/lvmdbusd/utils.py", line 726, in done
raise self.exception
File "/home/tasleson/projects/lvm2/daemons/lvmdbusd/utils.py", line 732, in _run
self.rc = self.f(*self.args)
File "/home/tasleson/projects/lvm2/daemons/lvmdbusd/fetch.py", line 40, in _main_thread_load
(lv_changes, remove) = load_lvs(
File "/home/tasleson/projects/lvm2/daemons/lvmdbusd/lv.py", line 148, in load_lvs
return common(
File "/home/tasleson/projects/lvm2/daemons/lvmdbusd/loader.py", line 37, in common
objects = retrieve(search_keys, cache_refresh=False)
File "/home/tasleson/projects/lvm2/daemons/lvmdbusd/lv.py", line 72, in lvs_state_retrieve
lvs = sorted(cfg.db.fetch_lvs(selection), key=get_key)
File "/home/tasleson/projects/lvm2/daemons/lvmdbusd/lv.py", line 35, in get_key
pool = i['pool_lv']
KeyError: 'pool_lv'
Remove old code that became incorrect at some point.
It's probably a fragment of an old condition that was left
behind because it wasn't understood. We don't want to drop
the MISSING_PV flag just because the PV has no mda in use.
The device that was missing may have stale data, so the user
needs to decide if the device should be removed or restored.
Different sequences of steps that could be used to handle raid LVs
after VG takeover (what would happen in cluster failover) combined
with the loss of a disk.
When we are using -S|--select for non-reporting tools while using command log
reporting (log/report_command_log=1 setting), we need to create an internal
processing handle to handle the selection itself. In this case, the internal
processing handle to execute the selection (to process the -S|--select) has
a parent handle (that is processing the actual non-reporting command).
When this parent handle exists, we can't destroy the command log report
in destroy_processing_handle as there's still the parent processing to
finish. The parent processing may still generate logs which need to be
reported in the command log report. If the command log report was
destroyed prematurely together with destroying the internal processing
handle for -S|--select, then any subsequent log request from processing
the actual command (and hence an attermpt to access the command log report)
ended up with a segfault.
See also: https://bugzilla.redhat.com/show_bug.cgi?id=2175220
There is a window of time where the following can occur.
1. An API request is in process to the lvm shell, we have written some
command to the lvm shell and we are blocked on that thread waiting
2. A signal arrives to the daemon which causes us to exit. The signal
handling code path goes directly to the lvm shell and writes
"exit\n". This causes the lvm shell to simply exit.
3. The thread that was waiting for a response gets an EIO as the child
process has exited. This bubbles up a failure.
This is addressed by placing a lock in the lvm shell to prevent
concurrent access to the shell. We also gather additional debug data
when we get an error in the lvm shell read path. This should help if
the lvm shell exits/crashes on its own.
Replace spaces with \040 in directory paths from getmntent (mtab).
The recent commit 5374a44c57 compares mount point directory paths
from /etc/mtab and /proc/mounts, in order to detect when a mounted
LV has been renamed. The directory path comparison does not work
correctly when the path contains spaces because getmntent uses
ascii space chars and proc replaces spaces with \040.
Add some Gentoo based patches for better support of static linking.
This are not tested nor supported by upstream developers.
Usage requires presence of several libraries in their static form
which is however not commonly available.
Selinux modified by zkabelac to still work on older sofrware which
did not provided libselinux.pc at a time - see keep the old check
present and use pkg-config only when possible.
When user enables lockd libraris, code needs to fail, when then are
missing. Also when notify-dbus support if enabled, and libsystemd is
missing, abort configuration.
Check for pkg-config --libs libdlm_lt and test if the returned value
contains word 'pthread' - if so, it's likely a buggy result from
incorrect config file and use directly -ldlm_lt for this case.
There may be some unconventional methods of sharing disks with vgimport
where hints could cause confusion. Hints should be disabled when
sharing disks, or pvscan --cache used to regenerate hints as needed,
but invalidating hints from vgimport might help reduce confusion.
log_error("VDO minimum io size %u is unsupported [512, 4096].",
vtp->minimum_io_size);
(vtp->minimum_io_size<<SECTOR_SHIFT));
valid=false;
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
