1
0
mirror of git://sourceware.org/git/lvm2.git synced 2025-03-10 16:58:47 +03:00

18026 Commits

Author SHA1 Message Date
Zdenek Kabelac
50b89ccac3 vdo: support version 4
Properly parse VDO volumes formatted with geometry block version 4.0.

(cherry picked from commit b90c5d60156c59c33ac440dd5ea7681457684f64)
2023-07-13 15:06:43 +02:00
Zdenek Kabelac
e6fc4b2d36 lvm_import_vdo: spelling fixes
(cherry picked from commit 8a07f57ba478c16ca5d5eacf778ee2708109aa65)
2023-07-13 15:02:33 +02:00
Zdenek Kabelac
d50c0cdcda vdo: lvm_import_vdo correct parsing output
Output from vdo manager may actually indent output with spaces,
so trim leading and ending space.

Also add support for verbosity flag for vdo conversion tool.

(cherry picked from commit 8b75bbe47d2a2961b0fb029ad7a01dc37ee6a8e5)
2023-07-13 15:02:31 +02:00
Zdenek Kabelac
eb32a7ca3e vdo: indent for lvdisplay
(cherry picked from commit e48c9826e3069b06a5aa31e1e59200dce4915983)
2023-07-13 15:02:29 +02:00
Zdenek Kabelac
e001f919e8 vdo: check_point_frequence is unused option
This vdo parameter existed in the early stage of integration of vdo into lvm2,
but later it's been removed from vdoformat tool - so actually if
there would be any non-zero value it would cause error on lvcreate.
Option was not stored on disk in lvm2 metadata.

Remove this vdo parameter from lvm2 sources.

(Although this vdo parameter will be still accepted on cmdline through
 --vdosettings option, but it will be ignored.)

(cherry picked from commit 6ff65e675582265a673136f1edd5358b26c9e481)
2023-07-13 15:02:26 +02:00
Zdenek Kabelac
36ef0612d9 tests: update test for automatic virtual size correction
lvm_import_vdo will adjust the virtual size within conversion
so the volume can be use without any futher tweaking.

(cherry picked from commit 8d09124c6204d49f6bda833229c96e8a76665127)
2023-07-13 15:02:02 +02:00
Zdenek Kabelac
3603348e50 lvm_import_vdo: correct the converted virtual size
Ensure the volume after conversion has the properly aligned size to the
volume group extent size. This would be visible when using virtual size,
that cannot be divided by extent size.

Before the user had to manually adjust the size after conversion to get
access to all data stored on VDO volume.

(cherry picked from commit 657dde6208ddc8172b7d0e1466751046c8ddaeaa)
2023-07-13 15:02:00 +02:00
Zdenek Kabelac
d698509aa8 WHATS_NEW: update
(cherry picked from commit 7ac5dbfd09e771f087a9467aca8a0d5905084459)
2023-07-13 15:01:34 +02:00
Zdenek Kabelac
a621c3a6dc tests: updates
Add some more complicated conversion tests.

(cherry picked from commit 9105890d43bd1b6fe514114f671af2e946e13083)
2023-07-13 14:58:40 +02:00
Zdenek Kabelac
0fe0f35cda tests: update test to handle different status
Since now we change deduplication with V4 table line change,
the modification tends to be faster and we can capture for a few ms
also 'status' about opening or closing deduplication index.
Use 'grep -E' to handle both words.

(cherry picked from commit 0fed9b097120648301faa586970a47b8b4d629ff)
2023-07-13 14:58:13 +02:00
Zdenek Kabelac
f38f893139 tests: vdo manager wrapper tool updates
Support size specification in KiB units.
Add some theoretical support for some new options from vdoprepareforlvm.

(cherry picked from commit 7f661a24c4f7fa8f161f17da093e456f0d60e106)
2023-06-22 18:05:56 +02:00
Zdenek Kabelac
f8ef830ccc vdo: man updates for lvm_import_vdo
(cherry picked from commit 17ee5df8576acf628d88fd6840ce55aa1388d6d1)
2023-06-22 18:05:53 +02:00
Zdenek Kabelac
80a59e88c9 vdo: enhance lvm_vdo_import
Add support for usage of 'dm-snapshot' for doing whole device conversion
in a snapshot which could be merged once the whole conversion has been
made.
This helps with cases where there would be any unexpected failure in the
middle of conversion process and user can continue using original
device until problem in conversion is fixed.

Import tool now uses 'truncate' tool to create a small backend file (20M) for loop device
to hold COW exception store.

Option  --vdo-config has been added to allow specifing location of vdo
configuration file.

Option --no-snapshot allows to use import tool without creation of
snapshot, however recovery after finished VDO conversion and unfinished
lvm2 conversion is then very difficult.

Option --uuid-prefix allow to specify DM UUID prefix for snapshot
device.

Use read with -r.

(cherry picked from commit be6c34212b05d8c59c0ba68e26c56cddf61c9f04)
2023-06-22 18:05:45 +02:00
Marian Csontos
062b014c04 tests: cleanup some shellcheck warns
(manually backported some changes from commit 773bc013778df631f9ee14e7a79d5f02211b1e67)
2023-06-22 18:01:27 +02:00
Zdenek Kabelac
3b7c0786cc vdo: fix and enhance vdo metadata reader
Improve metadata parser to handle volume_geometry bio_offset,
which needs to be substracted from 'region' start_block when present.

This bio_offset block is non-zero i.e. with converted VDO volumes.

Also fix some converted structure value (but they are not in use).

(cherry picked from commit 00633f8b668d94a58756d96f5927fd690b8f41d3)
2023-06-22 17:43:17 +02:00
Zdenek Kabelac
5e6aa69ec0 vdo: document feature option
(cherry picked from commit 534269d0fb834fb9a3b115bf859319fc1acb2275)
2023-06-22 17:43:10 +02:00
Zdenek Kabelac
51e90d49ba vdo: use fixed size vdopool wrapper
Instead of using size of 'empty header' in vdopool use fixed size 4K
for a 'wrappeing' vdo-pool device.

This fixes the issue when user tried to activate vdo-pool after
a conversion from vdo managed device with 'vgchange -ay' - where
this command activated all LVs with 'vdo-pool' wrapping device as well,
but this converted pool uses  0-length header.

This 4k size should usually prevent other tools like 'blkid' recognize
such device as anything - so it shouldn't cause any problems with
duplicate indentification of devices.

(cherry picked from commit e8e6347ba3f59cbd2f7e92aa707543b90fa607a3)
2023-06-22 17:43:03 +02:00
Zdenek Kabelac
31e71842d4 tests: check failing vdo conversion
When we detect not usable vdo on a machine, check lvconvert fails.

(cherry picked from commit 06c5c29443fdd48730a282a5db16e46c793db27b)
2023-06-22 17:42:54 +02:00
Zdenek Kabelac
686d6ecdcc tests: vdo resizing
(cherry picked from commit aa09232dc4fb825f4e643bbe87a95e2318dd5a93)
2023-06-22 17:42:41 +02:00
Marian Csontos
7274949166 vdo: resize requires active vdopool volume
ATM kernel VDO target does not handle resize of inactive VDO LVs
so prevent users corrupting such LVs and require active such LVs.

(manually backported from commit c20f01a0cbb24e8276734a00f26a510cf6daaf21)
2023-06-22 16:55:47 +02:00
Zdenek Kabelac
a22a9aa39b vdo: fix and enhance vdo constain checking
Enhance checking vdo constains so it also handles changes of active VDO LVs
where only added difference is considered now.

For this also the reported informational message about used memory
was improved to only list consuming RAM blocks.

(cherry picked from commit 2451bc568feb9b3d0cb4d7c15e3c723f4d0c8cc9)
2023-06-21 09:49:50 +02:00
Zdenek Kabelac
52e019c613 lvcreate: fix error path return values
Return failing error code for return path, as 'return 0' in this
case was returning success.

(cherry picked from commit b9f35e07db41f3dd6bea2c91a8bc1bff93a4d406)
2023-06-21 09:46:54 +02:00
Zdenek Kabelac
696036c20f vdo: read live vdo size configuration
Introduce struct vdo_pool_size_config usable to calculate necessary
memory size for active VDO volume.

Function lv_vdo_pool_size_config() is able to read out this
configuration out of runtime DM table line.

(cherry picked from commit 1bed2cafe83096cf01cfff5cd2cd64ecb32b7d78)
2023-06-21 09:19:21 +02:00
Zdenek Kabelac
71c8df45c7 vdo: check memory only in non critical section
When we are actually resizing VDO device - we need to check size only in
non-critical section - otherwise we are checking

(cherry picked from commit 773b88e028ab2965a8c185f5f2147334f8f2bbfd)
2023-06-21 09:19:19 +02:00
Zdenek Kabelac
05b4d98a51 vdo: fix reader error path
Nothing to be closed on this error path.

(cherry picked from commit b6b1c19365d20d926d8aa39bf591731a7f9bb75a)
2023-06-21 09:19:14 +02:00
Zdenek Kabelac
57eb6c1b9c vdo: improve validation message
Rephrase.

(cherry picked from commit 403779333be419352d6062da592ea78e6096e12b)
2023-06-21 09:18:49 +02:00
Zdenek Kabelac
c66a3010e0 vdo: enhance detection of virtual size
Improve detection of VDO virtual size - so it's not reading VDO
metadata when VDO device is already active and instead we reuse
existing table line for knowing existing metadata size.

NOTE: if there is ever going to be added support for reduction
of VDO virtual size - this method will need to be reworked to
allow size difference only within 'extent_size' alignment.

(cherry picked from commit 8e9410594b3113386a667df400b2c229c745cb6a)
2023-06-21 09:18:44 +02:00
Zdenek Kabelac
f67de990a6 vdo: replace errors with debug
As we actully use reading of VDO metadata only as extra 'information' source,
and not error command - switch to 'log_debug()' severity with messages
out of parser code.

(cherry picked from commit 218c7d44b5ac64b6c38dfd40885a22008b5cec81)
2023-06-21 09:18:42 +02:00
Zdenek Kabelac
9399376608 device_mapper: vdo V4 avoid messaging
With V4 format build table line with compression and
deduplication and skip sending any messages to set up
these parameters.

(cherry picked from commit 36a923926c2c27c1a8a5ac262387d2a4d3e620f8)
2023-06-21 09:18:38 +02:00
Zdenek Kabelac
13ce88a2b2 dev_manager: accept misalined vdo pools.
Since lvm2 may create VDO pool virtual size aligned only on extent size
while VDO itself is just 4K aligned - we need to support such misalign.

(cherry picked from commit 2e79b005c2013fb03d8a48a3cfd8e70a982dd65b)
2023-06-21 09:18:34 +02:00
Zdenek Kabelac
fd16a667a9 device_mapper: add parser for vdo metadata
Add very simplistic parser of vdo metadata to be able to obtain
logical_blocks stored within vdo metadata - as lvm2 may
submit smaller value due to internal aligment rules.

To avoid creation of mismatching table line - use this number
instead the one provided by lvm2.

(cherry picked from commit 829ab017082eaad253ebd28ad7d7ae7f3936dbcb)
2023-06-21 09:18:31 +02:00
Zdenek Kabelac
fc1c105c75 tests: vdo emulation without vdo binary
Avoid inserting 'vdo' binary into path - and use
alias and VDO_BINARY shell vars for emulation.

(cherry picked from commit 15ad2b8e5585b89bc3f09a53567f17eb70f45714)
2023-06-21 09:18:19 +02:00
Zdenek Kabelac
c3d533a0ea vdo: extend volume and pool without flush
When the volume size is extended, there is no need to flush
IO operations (nothing can be targeting new space yet).
VDO target is supported as target that can safely work with
this condition.

Such support is also needed, when extending VDOPOOL size
while the pool is reaching its capacity - since this allows
to continue working without reaching 'out-of-space' condition
due to flushing of all in flight IO.

(cherry picked from commit e26c21cb8d5841141dcfdfc77f46da1108a81255)
2023-06-21 09:15:17 +02:00
Zdenek Kabelac
a4f39c5bbf vdo: reset errno before strtoull
Missed errno reset in commit ebad057579aeff0980a1b8af7eaacd56e62ed0c9.

(cherry picked from commit 309df239e3ee474f3a5337f8fa3c0a1f7b0a89e5)
2023-06-20 15:38:57 +02:00
Zdenek Kabelac
fee817aff4 vdo: use only verbose log level for reformating
When lvcreate is makeing VDO pool and user has not specified -V size,
ATM we actually run  'vdoformat' twice to get properly 'extent' aligned
size matching lvm2 properties - so the 2nd. run of vdoformat actually
can stay with 'log_verbose()' so the standard printed result
is not showing confusing info (which is now also correctly using
print_unless_silent)

(cherry picked from commit fc5bc5985d03aef5846cb98882d17815fc00ca15)
2023-06-20 15:36:50 +02:00
Zdenek Kabelac
f5c194afc5 vdo: suffle code for better error path handling
For failing dm_ no need to report 2nd. error,
but we missed to report error with 'updated==NULL'.

(cherry picked from commit 493acb9195cef185b38ae4e4ffb84b984e5cc08c)
2023-06-20 15:36:35 +02:00
Zdenek Kabelac
aa75359c7c vdo: enhance lvcreate validation
When creating VDO pool based of  % values, lvm2 is now more clever
and avoids to create 'unsupportable' sizes of physical backend
volumes as 16TiB is maximum size supported by VDO target
(and also limited by maximum supportable slabs (8192) based on slab
size.

If the requested virtual size is approaching max supported size 4PiB,
switch header size to 0.

(cherry picked from commit e2e31d9acf1b96ab741c22dc0a2fefd672996d3a)
2023-06-20 15:36:32 +02:00
Zdenek Kabelac
c160f54ec5 vdo: support v4 kernel target line
Check and use new available table line v4, if kernel supports it.

(cherry picked from commit 1c18ed3b4ab2f2d5a15995b8f0a18d7d1c4d60ca)
2023-06-20 15:36:30 +02:00
Zdenek Kabelac
d2a2720ccb vdo: add reformating to extent size aligned virtual size
Newer VDO kernel target require to have matching virtual size - this
however cause incompatiblity when lvcreate is let to format VDO data
device and read the usable size from vdoformat.
Altough this is a kernel regression and will likely get fixed,
lvm2 can actually reformat VDO device to use properly aligned VDO LV
size to make this problem disappear.

(cherry picked from commit a477490e812639fed3be495f215fcf1a7b65b7ee)
2023-06-20 15:36:25 +02:00
Zdenek Kabelac
783213e4da vdo: check vdo memory constrains
Add function to check for avaialble memory for particular VDO
configuration - to avoid unnecessary machine swapping for configs
that will not fit into memory (possibly in locked section).

Formula tries to estimate RAM size machine can use also with
swapping for kernel target - but still leaving some amount of
usable RAM.

Estimation is based on documented RAM usage of VDO target.

If the /proc/meminfo would be theoretically unavailable, try to use
'sysinfo()' function, however this is giving only free RAM without
the knowledge about how much RAM could be eventually swapped.

TODO: move _get_memory_info() into generic lvm2 API function used
by other targets with non-trivial memory requirements.

(cherry picked from commit ebad057579aeff0980a1b8af7eaacd56e62ed0c9)
2023-06-20 15:36:19 +02:00
Zdenek Kabelac
1a39e0c2c2 vdo: update info about memory
Add more info about kernel target memory allocation associated with
VDO pool usage.

(cherry picked from commit f445624c339a5c1436a47d2b51046869f183eb03)
2023-06-20 15:36:00 +02:00
Zdenek Kabelac
e82e23447f vdo: fix conversion of vdo_slab_size_mb 2nd
Patch 1b070f366ba57a6eb24df03241284732db5047e9 should have
been already fixing this issue but since it the incorrect
patch rebasing the change to vdo_slabSize got lost.
So again now with explicit one-line patch.

(cherry picked from commit d2667bc25bccaf0f70cc2ded0fd3f25a79cb4f6c)
2023-06-20 15:33:42 +02:00
Tony Asleson
8702e97022 lvmdbusd: Change unit test vdo minimum size
(cherry picked from commit 47c61907b4adbdead50f5bb5ac95c0f5d0fe263e)
2023-06-20 15:33:36 +02:00
Zdenek Kabelac
64a78a2904 tests: update for wrapper
Update calling vdo manager since our vdo wrapper has a simple shell
arg parser so it needs args without '='

Also correct using  DM_DEV_DIR for 'pvcreate'

(cherry picked from commit 4a498512077b4fe5cf6b806a91228bd23e513123)
2023-06-20 15:33:24 +02:00
Zdenek Kabelac
2456b39b34 tests: add lvm_vdo_wrapper
Introduce a replacement vdo manager wrapper for testing.
When using test suite on a system without vdo manager (which has got
deprecated) - we still need its functionality to prepare 'vdo volume'
for testing lvm_import_vdo.

Wrapper currently need 2 binaries from older 'vdo 6.2' package -
to be named:
oldvdoformat - format VDO metadata with older format
oldvdoprepareforlvm - shift vdo metadata by 1MiB

(cherry picked from commit 2ecfd503edadaf5d46115826c629754bf9fd573f)
2023-06-20 15:33:14 +02:00
Zdenek Kabelac
a147a14813 vdo: fix conversion of vdo_slab_size_mb
When converting VDO volume, the parameter vdo_slabSize was
incorrectly copied as vdo_blockMapCacheSize, however this parameter
is then no longer used for any table line creation so the wrong
value was only stored in metadata.

Also use just single get_kb_size_with_unit_ and remove it's duplicate
functionality with get_mb_size_with_unit_.

Use $VERB for vdo remove call.

(cherry picked from commit 1b070f366ba57a6eb24df03241284732db5047e9)
2023-06-20 15:30:59 +02:00
Andrew Walsh
c41468a8ac vdo: ensure VDO config is removed
Make sure to remove the VDO config after conversion
of LVM-backed VDO.

Addresses point 3 in rhbz#1987024#c5

(cherry picked from commit 522561e64b5fe73cb9d01c2ee2d4b7624b0ddff4)
2023-06-20 14:54:13 +02:00
Zdenek Kabelac
cce56ebaa6 vdo: report supported range in error path
(cherry picked from commit 9f3eff002cc229d3c22dfd7db6da69dadc0bd460)
2022-12-08 15:25:33 +01:00
Zdenek Kabelac
b16082b056 vdo: use defines also for configuration defines
Keep single source for most of values printed in lvm.conf
(still needs some conversion)

Correct max for logical threads to 60
(we may refuse some older configuration which might eventually
user higher numbers - but so far let's assume no user have ever set this
as it's been non-trivial and if would complicate code unnecessarily.)

Accept maximum of 4PiB for virtual size of VDO LV
(lvm2 will drop 'header borders to 0 for this case').

(cherry picked from commit b5c8e591ed9ee30b67e79d60705d3c0bb8509a2a)
2022-12-08 15:25:22 +01:00
Zdenek Kabelac
d0b5614d43 vdo: use single validator
Add era lenght validation into dm_vdo_validate_target_params()
and reuse this validator also for _check_lv_segment().

(cherry picked from commit 8ca2b1bc213188037ecedfbf76de53de871c7f5b)
2022-12-08 15:25:12 +01:00