1
0
mirror of git://sourceware.org/git/lvm2.git synced 2025-01-09 01:18:39 +03:00
Commit Graph

18026 Commits

Author SHA1 Message Date
Zdenek Kabelac
50b89ccac3 vdo: support version 4
Properly parse VDO volumes formatted with geometry block version 4.0.

(cherry picked from commit b90c5d6015)
2023-07-13 15:06:43 +02:00
Zdenek Kabelac
e6fc4b2d36 lvm_import_vdo: spelling fixes
(cherry picked from commit 8a07f57ba4)
2023-07-13 15:02:33 +02:00
Zdenek Kabelac
d50c0cdcda vdo: lvm_import_vdo correct parsing output
Output from vdo manager may actually indent output with spaces,
so trim leading and ending space.

Also add support for verbosity flag for vdo conversion tool.

(cherry picked from commit 8b75bbe47d)
2023-07-13 15:02:31 +02:00
Zdenek Kabelac
eb32a7ca3e vdo: indent for lvdisplay
(cherry picked from commit e48c9826e3)
2023-07-13 15:02:29 +02:00
Zdenek Kabelac
e001f919e8 vdo: check_point_frequence is unused option
This vdo parameter existed in the early stage of integration of vdo into lvm2,
but later it's been removed from vdoformat tool - so actually if
there would be any non-zero value it would cause error on lvcreate.
Option was not stored on disk in lvm2 metadata.

Remove this vdo parameter from lvm2 sources.

(Although this vdo parameter will be still accepted on cmdline through
 --vdosettings option, but it will be ignored.)

(cherry picked from commit 6ff65e6755)
2023-07-13 15:02:26 +02:00
Zdenek Kabelac
36ef0612d9 tests: update test for automatic virtual size correction
lvm_import_vdo will adjust the virtual size within conversion
so the volume can be use without any futher tweaking.

(cherry picked from commit 8d09124c62)
2023-07-13 15:02:02 +02:00
Zdenek Kabelac
3603348e50 lvm_import_vdo: correct the converted virtual size
Ensure the volume after conversion has the properly aligned size to the
volume group extent size. This would be visible when using virtual size,
that cannot be divided by extent size.

Before the user had to manually adjust the size after conversion to get
access to all data stored on VDO volume.

(cherry picked from commit 657dde6208)
2023-07-13 15:02:00 +02:00
Zdenek Kabelac
d698509aa8 WHATS_NEW: update
(cherry picked from commit 7ac5dbfd09)
2023-07-13 15:01:34 +02:00
Zdenek Kabelac
a621c3a6dc tests: updates
Add some more complicated conversion tests.

(cherry picked from commit 9105890d43)
2023-07-13 14:58:40 +02:00
Zdenek Kabelac
0fe0f35cda tests: update test to handle different status
Since now we change deduplication with V4 table line change,
the modification tends to be faster and we can capture for a few ms
also 'status' about opening or closing deduplication index.
Use 'grep -E' to handle both words.

(cherry picked from commit 0fed9b0971)
2023-07-13 14:58:13 +02:00
Zdenek Kabelac
f38f893139 tests: vdo manager wrapper tool updates
Support size specification in KiB units.
Add some theoretical support for some new options from vdoprepareforlvm.

(cherry picked from commit 7f661a24c4)
2023-06-22 18:05:56 +02:00
Zdenek Kabelac
f8ef830ccc vdo: man updates for lvm_import_vdo
(cherry picked from commit 17ee5df857)
2023-06-22 18:05:53 +02:00
Zdenek Kabelac
80a59e88c9 vdo: enhance lvm_vdo_import
Add support for usage of 'dm-snapshot' for doing whole device conversion
in a snapshot which could be merged once the whole conversion has been
made.
This helps with cases where there would be any unexpected failure in the
middle of conversion process and user can continue using original
device until problem in conversion is fixed.

Import tool now uses 'truncate' tool to create a small backend file (20M) for loop device
to hold COW exception store.

Option  --vdo-config has been added to allow specifing location of vdo
configuration file.

Option --no-snapshot allows to use import tool without creation of
snapshot, however recovery after finished VDO conversion and unfinished
lvm2 conversion is then very difficult.

Option --uuid-prefix allow to specify DM UUID prefix for snapshot
device.

Use read with -r.

(cherry picked from commit be6c34212b)
2023-06-22 18:05:45 +02:00
Marian Csontos
062b014c04 tests: cleanup some shellcheck warns
(manually backported some changes from commit 773bc01377)
2023-06-22 18:01:27 +02:00
Zdenek Kabelac
3b7c0786cc vdo: fix and enhance vdo metadata reader
Improve metadata parser to handle volume_geometry bio_offset,
which needs to be substracted from 'region' start_block when present.

This bio_offset block is non-zero i.e. with converted VDO volumes.

Also fix some converted structure value (but they are not in use).

(cherry picked from commit 00633f8b66)
2023-06-22 17:43:17 +02:00
Zdenek Kabelac
5e6aa69ec0 vdo: document feature option
(cherry picked from commit 534269d0fb)
2023-06-22 17:43:10 +02:00
Zdenek Kabelac
51e90d49ba vdo: use fixed size vdopool wrapper
Instead of using size of 'empty header' in vdopool use fixed size 4K
for a 'wrappeing' vdo-pool device.

This fixes the issue when user tried to activate vdo-pool after
a conversion from vdo managed device with 'vgchange -ay' - where
this command activated all LVs with 'vdo-pool' wrapping device as well,
but this converted pool uses  0-length header.

This 4k size should usually prevent other tools like 'blkid' recognize
such device as anything - so it shouldn't cause any problems with
duplicate indentification of devices.

(cherry picked from commit e8e6347ba3)
2023-06-22 17:43:03 +02:00
Zdenek Kabelac
31e71842d4 tests: check failing vdo conversion
When we detect not usable vdo on a machine, check lvconvert fails.

(cherry picked from commit 06c5c29443)
2023-06-22 17:42:54 +02:00
Zdenek Kabelac
686d6ecdcc tests: vdo resizing
(cherry picked from commit aa09232dc4)
2023-06-22 17:42:41 +02:00
Marian Csontos
7274949166 vdo: resize requires active vdopool volume
ATM kernel VDO target does not handle resize of inactive VDO LVs
so prevent users corrupting such LVs and require active such LVs.

(manually backported from commit c20f01a0cb)
2023-06-22 16:55:47 +02:00
Zdenek Kabelac
a22a9aa39b vdo: fix and enhance vdo constain checking
Enhance checking vdo constains so it also handles changes of active VDO LVs
where only added difference is considered now.

For this also the reported informational message about used memory
was improved to only list consuming RAM blocks.

(cherry picked from commit 2451bc568f)
2023-06-21 09:49:50 +02:00
Zdenek Kabelac
52e019c613 lvcreate: fix error path return values
Return failing error code for return path, as 'return 0' in this
case was returning success.

(cherry picked from commit b9f35e07db)
2023-06-21 09:46:54 +02:00
Zdenek Kabelac
696036c20f vdo: read live vdo size configuration
Introduce struct vdo_pool_size_config usable to calculate necessary
memory size for active VDO volume.

Function lv_vdo_pool_size_config() is able to read out this
configuration out of runtime DM table line.

(cherry picked from commit 1bed2cafe8)
2023-06-21 09:19:21 +02:00
Zdenek Kabelac
71c8df45c7 vdo: check memory only in non critical section
When we are actually resizing VDO device - we need to check size only in
non-critical section - otherwise we are checking

(cherry picked from commit 773b88e028)
2023-06-21 09:19:19 +02:00
Zdenek Kabelac
05b4d98a51 vdo: fix reader error path
Nothing to be closed on this error path.

(cherry picked from commit b6b1c19365)
2023-06-21 09:19:14 +02:00
Zdenek Kabelac
57eb6c1b9c vdo: improve validation message
Rephrase.

(cherry picked from commit 403779333b)
2023-06-21 09:18:49 +02:00
Zdenek Kabelac
c66a3010e0 vdo: enhance detection of virtual size
Improve detection of VDO virtual size - so it's not reading VDO
metadata when VDO device is already active and instead we reuse
existing table line for knowing existing metadata size.

NOTE: if there is ever going to be added support for reduction
of VDO virtual size - this method will need to be reworked to
allow size difference only within 'extent_size' alignment.

(cherry picked from commit 8e9410594b)
2023-06-21 09:18:44 +02:00
Zdenek Kabelac
f67de990a6 vdo: replace errors with debug
As we actully use reading of VDO metadata only as extra 'information' source,
and not error command - switch to 'log_debug()' severity with messages
out of parser code.

(cherry picked from commit 218c7d44b5)
2023-06-21 09:18:42 +02:00
Zdenek Kabelac
9399376608 device_mapper: vdo V4 avoid messaging
With V4 format build table line with compression and
deduplication and skip sending any messages to set up
these parameters.

(cherry picked from commit 36a923926c)
2023-06-21 09:18:38 +02:00
Zdenek Kabelac
13ce88a2b2 dev_manager: accept misalined vdo pools.
Since lvm2 may create VDO pool virtual size aligned only on extent size
while VDO itself is just 4K aligned - we need to support such misalign.

(cherry picked from commit 2e79b005c2)
2023-06-21 09:18:34 +02:00
Zdenek Kabelac
fd16a667a9 device_mapper: add parser for vdo metadata
Add very simplistic parser of vdo metadata to be able to obtain
logical_blocks stored within vdo metadata - as lvm2 may
submit smaller value due to internal aligment rules.

To avoid creation of mismatching table line - use this number
instead the one provided by lvm2.

(cherry picked from commit 829ab01708)
2023-06-21 09:18:31 +02:00
Zdenek Kabelac
fc1c105c75 tests: vdo emulation without vdo binary
Avoid inserting 'vdo' binary into path - and use
alias and VDO_BINARY shell vars for emulation.

(cherry picked from commit 15ad2b8e55)
2023-06-21 09:18:19 +02:00
Zdenek Kabelac
c3d533a0ea vdo: extend volume and pool without flush
When the volume size is extended, there is no need to flush
IO operations (nothing can be targeting new space yet).
VDO target is supported as target that can safely work with
this condition.

Such support is also needed, when extending VDOPOOL size
while the pool is reaching its capacity - since this allows
to continue working without reaching 'out-of-space' condition
due to flushing of all in flight IO.

(cherry picked from commit e26c21cb8d)
2023-06-21 09:15:17 +02:00
Zdenek Kabelac
a4f39c5bbf vdo: reset errno before strtoull
Missed errno reset in commit ebad057579.

(cherry picked from commit 309df239e3)
2023-06-20 15:38:57 +02:00
Zdenek Kabelac
fee817aff4 vdo: use only verbose log level for reformating
When lvcreate is makeing VDO pool and user has not specified -V size,
ATM we actually run  'vdoformat' twice to get properly 'extent' aligned
size matching lvm2 properties - so the 2nd. run of vdoformat actually
can stay with 'log_verbose()' so the standard printed result
is not showing confusing info (which is now also correctly using
print_unless_silent)

(cherry picked from commit fc5bc5985d)
2023-06-20 15:36:50 +02:00
Zdenek Kabelac
f5c194afc5 vdo: suffle code for better error path handling
For failing dm_ no need to report 2nd. error,
but we missed to report error with 'updated==NULL'.

(cherry picked from commit 493acb9195)
2023-06-20 15:36:35 +02:00
Zdenek Kabelac
aa75359c7c vdo: enhance lvcreate validation
When creating VDO pool based of  % values, lvm2 is now more clever
and avoids to create 'unsupportable' sizes of physical backend
volumes as 16TiB is maximum size supported by VDO target
(and also limited by maximum supportable slabs (8192) based on slab
size.

If the requested virtual size is approaching max supported size 4PiB,
switch header size to 0.

(cherry picked from commit e2e31d9acf)
2023-06-20 15:36:32 +02:00
Zdenek Kabelac
c160f54ec5 vdo: support v4 kernel target line
Check and use new available table line v4, if kernel supports it.

(cherry picked from commit 1c18ed3b4a)
2023-06-20 15:36:30 +02:00
Zdenek Kabelac
d2a2720ccb vdo: add reformating to extent size aligned virtual size
Newer VDO kernel target require to have matching virtual size - this
however cause incompatiblity when lvcreate is let to format VDO data
device and read the usable size from vdoformat.
Altough this is a kernel regression and will likely get fixed,
lvm2 can actually reformat VDO device to use properly aligned VDO LV
size to make this problem disappear.

(cherry picked from commit a477490e81)
2023-06-20 15:36:25 +02:00
Zdenek Kabelac
783213e4da vdo: check vdo memory constrains
Add function to check for avaialble memory for particular VDO
configuration - to avoid unnecessary machine swapping for configs
that will not fit into memory (possibly in locked section).

Formula tries to estimate RAM size machine can use also with
swapping for kernel target - but still leaving some amount of
usable RAM.

Estimation is based on documented RAM usage of VDO target.

If the /proc/meminfo would be theoretically unavailable, try to use
'sysinfo()' function, however this is giving only free RAM without
the knowledge about how much RAM could be eventually swapped.

TODO: move _get_memory_info() into generic lvm2 API function used
by other targets with non-trivial memory requirements.

(cherry picked from commit ebad057579)
2023-06-20 15:36:19 +02:00
Zdenek Kabelac
1a39e0c2c2 vdo: update info about memory
Add more info about kernel target memory allocation associated with
VDO pool usage.

(cherry picked from commit f445624c33)
2023-06-20 15:36:00 +02:00
Zdenek Kabelac
e82e23447f vdo: fix conversion of vdo_slab_size_mb 2nd
Patch 1b070f366b should have
been already fixing this issue but since it the incorrect
patch rebasing the change to vdo_slabSize got lost.
So again now with explicit one-line patch.

(cherry picked from commit d2667bc25b)
2023-06-20 15:33:42 +02:00
Tony Asleson
8702e97022 lvmdbusd: Change unit test vdo minimum size
(cherry picked from commit 47c61907b4)
2023-06-20 15:33:36 +02:00
Zdenek Kabelac
64a78a2904 tests: update for wrapper
Update calling vdo manager since our vdo wrapper has a simple shell
arg parser so it needs args without '='

Also correct using  DM_DEV_DIR for 'pvcreate'

(cherry picked from commit 4a49851207)
2023-06-20 15:33:24 +02:00
Zdenek Kabelac
2456b39b34 tests: add lvm_vdo_wrapper
Introduce a replacement vdo manager wrapper for testing.
When using test suite on a system without vdo manager (which has got
deprecated) - we still need its functionality to prepare 'vdo volume'
for testing lvm_import_vdo.

Wrapper currently need 2 binaries from older 'vdo 6.2' package -
to be named:
oldvdoformat - format VDO metadata with older format
oldvdoprepareforlvm - shift vdo metadata by 1MiB

(cherry picked from commit 2ecfd503ed)
2023-06-20 15:33:14 +02:00
Zdenek Kabelac
a147a14813 vdo: fix conversion of vdo_slab_size_mb
When converting VDO volume, the parameter vdo_slabSize was
incorrectly copied as vdo_blockMapCacheSize, however this parameter
is then no longer used for any table line creation so the wrong
value was only stored in metadata.

Also use just single get_kb_size_with_unit_ and remove it's duplicate
functionality with get_mb_size_with_unit_.

Use $VERB for vdo remove call.

(cherry picked from commit 1b070f366b)
2023-06-20 15:30:59 +02:00
Andrew Walsh
c41468a8ac vdo: ensure VDO config is removed
Make sure to remove the VDO config after conversion
of LVM-backed VDO.

Addresses point 3 in rhbz#1987024#c5

(cherry picked from commit 522561e64b)
2023-06-20 14:54:13 +02:00
Zdenek Kabelac
cce56ebaa6 vdo: report supported range in error path
(cherry picked from commit 9f3eff002c)
2022-12-08 15:25:33 +01:00
Zdenek Kabelac
b16082b056 vdo: use defines also for configuration defines
Keep single source for most of values printed in lvm.conf
(still needs some conversion)

Correct max for logical threads to 60
(we may refuse some older configuration which might eventually
user higher numbers - but so far let's assume no user have ever set this
as it's been non-trivial and if would complicate code unnecessarily.)

Accept maximum of 4PiB for virtual size of VDO LV
(lvm2 will drop 'header borders to 0 for this case').

(cherry picked from commit b5c8e591ed)
2022-12-08 15:25:22 +01:00
Zdenek Kabelac
d0b5614d43 vdo: use single validator
Add era lenght validation into dm_vdo_validate_target_params()
and reuse this validator also for _check_lv_segment().

(cherry picked from commit 8ca2b1bc21)
2022-12-08 15:25:12 +01:00