1
0
mirror of git://sourceware.org/git/lvm2.git synced 2024-12-22 17:35:59 +03:00
Commit Graph

17680 Commits

Author SHA1 Message Date
Leo Yan
fc0495ea04 tests: idm: Add testing for the fabric failure
When the fabric failure occurs, it will lose the connection with hosts
instantly, and after a while it can recovery back so that the hosts can
continue to access the drives.

For this case, the locking manager should be reliable for this case and
can dynamically handle this case and allows user to continue to use the
VG/LV with associated locking scheme.

This patch adds a testing to emulate the fabric faliure, verify LVM
commands for this case.  The testing usage is:

  # make check_lvmlockd_idm \
	LVM_TEST_BACKING_DEVICE=/dev/sdo3,/dev/sdp3,/dev/sdp4 \
	LVM_TEST_FAILURE=1 T=idm_fabric_failure.sh

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-06-03 09:39:32 -05:00
Leo Yan
874001ee6e tests: Add testing for lvmlockd failure
After the lvmlockd abnormally exits and relaunch the daemon, if LVM
commands continue to run, lvmlockd and the backend lock manager (e.g.
sanlock lock manager or IDM lock manager) should can continue to serve
the requests from LVM commands.

This patch adds a test to emulate lvmlockd failure, and verify the LVM
commands after lvmlockd recovers back.  Below is an example for testing
the case:

  # make check_lvmlockd_idm \
	LVM_TEST_BACKING_DEVICE=/dev/sdo3,/dev/sdp3,/dev/sdp4 \
	LVM_TEST_FAILURE=1 T=lvmlockd_failure.sh

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-06-03 09:39:32 -05:00
Leo Yan
8c7b2df41f tests: Support idm failure injection
When the drive failure occurs, the IDM lock manager and lvmlockd should
handle this case properly.  E.g. when the IDM lock manager detects the
lease renewal failure caused by I/O errors, it should invoke the kill
path which is predefined by lvmlockd, so that the kill path program
(like lvmlockctl) can send requests to lvmlockd to stop and drop lock
for the relevant VG/LVs.

To verify the failure handling flow, this patch introduces an idm
failure injection program, it can input the "percentage" for drive
failures so that can emulate different failure cases.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-06-03 09:39:32 -05:00
Leo Yan
f83e11ff43 tests: stress: Add multi-threads stress testing for PV/VG/LV
This patch is to add the stress testing, which launches three threads,
one thread is for creating/removing PV, one thread is for
creating/removing VG, and the last one thread is for LV operations.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-06-03 09:39:32 -05:00
Leo Yan
692fe7bb31 tests: stress: Add multi-threads stress testing for VG/LV
This patch is to add the stress testing, which launches two threads,
each thread creates LV, activate and deactivate LV in the loop; so this
can test for multi-threading in lvmlockd and its backend lock manager.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-06-03 09:39:32 -05:00
Leo Yan
fe660467fa tests: stress: Add single thread stress testing
This patch is to add the stress testing, which loops to create LV,
activate and deactivate LV in the single thread.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-06-03 09:39:32 -05:00
Leo Yan
5b361b197e tests: Add checking for lvmlockd log
Add checking for lvmlockd log, this can be used for the test cases which
are interested in the interaction with lvmlockd.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-06-03 09:39:32 -05:00
Leo Yan
2097c27c05 tests: Cleanup idm context when prepare devices
For testing idm locking scheme, it's good to cleanup the idm context
before run the test cases.  This can give a clean environment for the
testing.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-06-03 09:39:32 -05:00
Leo Yan
759b0392d5 tests: Support multiple backing devices
In current implementation, the option "LVM_TEST_BACKING_DEVICE" only
supports to specify one backing device; this patch is to extend the
option to support multiple backing devices by using comma as separator,
e.g. below command specifies two backing devices:

  make check_lvmlockd_idm LVM_TEST_BACKING_DEVICE=/dev/sdj3,/dev/sdk3

This can allow the testing works on multiple drives and verify the
locking scheme if can work as expected for multiple drives case.  For
example, for Seagate IDM locking scheme, if a VG uses two PVs, every PV
is resident on a drive, thus the locking operations will be sent to two
drives respectively; so the extension for "LVM_TEST_BACKING_DEVICE" can
help to verify different drive configurations for locking.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-06-03 09:39:32 -05:00
Leo Yan
c64dbc7ee8 tests: Enable the testing for IDM locking scheme
This patch is to introduce testing option LVM_TEST_LOCK_TYPE_IDM, with
specifying this option, the Seagate IDM lock manager will be launched as
backend for testing.  Also add the prepare and remove shell scripts for
IDM.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-06-03 09:39:32 -05:00
David Teigland
2bce6faed0 pvchange: fix file locking deadlock
Calling clear_hint_file() to invalidate hints would acquire
the hints flock before the global flock which could cause deadlock.
The lock order requires the global lock to be taken first.

pvchange was always invalidating hints, which was unnecessary;
only invalidate hints when changing a PV uuid.  Because of the
lock ordering, take the global lock before clear_hint_file which
locks the hints file.
2021-06-02 16:29:54 -05:00
David Teigland
e7f107c246 writecache: don't pvmove device used by writecache
The existing check didn't cover the unusual case where the
cachevol exists on the same device as the origin LV.
2021-06-02 11:12:20 -05:00
David Teigland
247f69f9aa writecache: fix lv_on_pmem
dev_is_pmem on pv->dev requires a pv segment or it could segfault.
2021-06-02 10:51:12 -05:00
Zdenek Kabelac
b725b5ea6e vdo: fix preload of kvdo
Commit 5bf1dba9eb broke load of kvdo
kernel module - correct it by loading kvdo instead of trying dm-vdo.
2021-05-26 16:12:20 +02:00
David Teigland
4a746f7ffc lvremove: fix removing thin pool with writecache on data 2021-05-24 16:09:35 -05:00
David Teigland
a65f8e0a62 enable command syntax for thin and writecache
converting an LV with a writecache to thin pool data in
addition to previous attaching writecache to thin pool data
2021-05-24 16:09:35 -05:00
Leo Yan
102294f978 configure: Add macro LOCKDIDM_SUPPORT
The macro LOCKDIDM_SUPPORT is missed in configure.h.in file, thus when
execute "configure" command, it has no chance to add this macro in the
automatic generated header include/configure.h.

This patch adds macro LOCKDIDM_SUPPORT into configure.h.in.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-05-21 09:27:30 -05:00
Leo Yan
8b904dc711 tools: Add support for "idm" lock type
This patch is to update the comment and code to support "idm" lock type
which is used for LVM toolkit.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-05-20 16:01:05 -05:00
Leo Yan
affe1af148 lib: locking: Parse PV list for IDM locking
For shared VG or LV locking, IDM locking scheme needs to use the PV
list assocated with VG or LV for sending SCSI commands, thus it requires
to use some places to generate PV list.

In reviewing the flow for LVM commands, the best place to generate PV
list is in the locking lib.  So this is why this patch parses PV list as
shown.  It iterates over all the PV nodes one by one, and compare with
the VG name or LV prefix string.  If any PV matches, then the PV is
added into the PV list.  Finally the PV list is sent to lvmlockd daemon.

Here as mentioned, it compares LV prefix string with the format
"lv_name_", the reason is it needs to find out all relevant PVs, e.g.
for the thin pool, it has LVs for metadata, pool, error, and raw LV, so
we can use the prefix string to find out all PVs belonging to the thin
pool.

For the global lock, it's not covered in this patch.  To avoid the egg
and chicken issue, we need to prepare the global lock ahead before any
locking can be used.  So the global lock's PV list is established in
lvmlockd daemon by iterating all drives with partition labeled with
"propeller".

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-05-20 16:01:05 -05:00
Leo Yan
ef1c57e68f lib: locking: Add new type "idm"
We can consider the drive firmware a server to handle the locking
request from nodes, this essentially is a client-server model.
DLM uses the kernel as a central place to manage locks, so it also
complies with client-server model for locking operations.  This is
why IDM and DLM are similar with each other for their wrappers.

This patch largely works by generalizing the DLM code paths and then
providing degeneralized functions as wrappers for both IDM and DLM.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-05-20 16:01:05 -05:00
Leo Yan
d02f5392a0 lvmlockd: idm: Hook Seagate IDM wrapper APIs
To allow the IDM locking scheme be used by users, this patch hooks the
IDM wrapper; it also introducs a new locking type "idm" and we can use
it for global lock with option '-g idm'.

To support IDM locking type, the main change in the data structure is to
add pvs path arrary.  The pvs list is transferred from the lvm commands,
when lvmlockd core layer receives message, it extracts the message with
the keyword "path[idx]".  Finally, the pv list will pass to IDM lock
manager as the target drives for sending IDM SCSI commands.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-05-20 16:01:04 -05:00
Leo Yan
7a8b7b4add lvmlockd: idm: Introduce new locking scheme
Alongside the existed locking schemes of DLM and sanlock, this patch is
to introduce new locking scheme: In-Drive-Mutex (IDM).

With the IDM support in the drive, the locks are resident in the drive,
thus, the locking lease is maintained in a central place: the drive
firmware.  We can consider this is a typical client-server model,
every host (or node) in the server cluster launches the request for
leasing mutex to a drive firmware, the drive firmware works as an
arbitrator to grant the mutex to a requester and it can reject other
applicants if the mutex has been acquired.  To satisfy the LVM
activation for different modes, IDM supports two locking modes:
exclusive and shareable.

Every IDM is identified with two IDs, one is the host ID and another is
the resource ID.  The resource ID is a unique identifier for what the
resource it's protected, in the integration with lvmlockd, the resource
ID is combined with VG's UUID and LV's UUID; for the global locking,
the bytes in resource ID are all zeros, and for the VG locking, the
LV's UUID is set as zero.   Every host can generate a random UUID and
use it as the host ID for the SCSI command, this ID is used to clarify
the ownership for mutex.

For easily invoking the IDM commands to drive, like other locking
scheme (e.g. sanlock), a daemon program named IDM lock manager is
created, so the detailed IDM SCSI commands are encapsulated in the
daemon, and lvmlockd uses the wrapper APIs to communicate with the
daemon program.

This patch introduces the IDM locking wrapper layer, it forwards the
locking requests from lvmlockd to the IDM lock manager, and returns the
result from drives' responding.

One thing should be mentioned is the IDM's LVB.  IDM supports LVB to max
7 bytes when stores into the drive, the most significant byte of 8 bytes
is reserved for control bits.  For this reason, the patch maps the
timestamp in macrosecond unit with its cached LVB, essentially, if any
timestamp was updated by other nodes, that means the local LVB is
invalidate.  When the timestamp is stored into drive's LVB, it's
possbile to cause time-going-backwards issue, which is introduced by the
time precision or missing synchronization acrossing over multiple nodes.
So the IDM wrapper fixes up the timestamp by increment 1 to the latest
value and write back into drive.

Currently LVB is used to track VG changes and its purpose is to notify
lvmetad cache invalidation when detects any metadata has been altered;
but lvmetad is not used anymore for caching metadata, LVB doesn't
really work.  It's possible that the LVB functionality could be useful
again in the future, so let's enable it for IDM in the first place.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2021-05-20 16:00:59 -05:00
Marian Csontos
3e2f09d78b post-release 2021-05-07 23:09:15 +02:00
Marian Csontos
01b05cf51d pre-release 2021-05-07 23:09:09 +02:00
Marian Csontos
64719861ec make: generate 2021-05-07 17:44:43 +02:00
David Teigland
00f603de2c tests: add lvextend-caches-on-thindata
to test lvextend of thin pool data while it has
cache|writecache attached
2021-05-06 16:23:26 -05:00
David Teigland
6b00c8c910 writecache: allow attaching to thin pool data 2021-05-06 16:23:03 -05:00
David Teigland
92fcfc59b2 tests: new lvextend-caches
to test lvextend of LVs with attached cache|writecache
2021-05-06 14:43:10 -05:00
David Teigland
318bb3a06b blkid: simplify fs block size check
Only the LV path name is needed for blkid query,
the step of getting a dev struct is not needed.
2021-05-05 16:15:10 -05:00
David Teigland
71933d3496 devices file: recommend removing filter
When the devices file is enabled and the filter is set,
recommand that the user remove the filter from lvm.conf
since it's ignored.
2021-05-05 13:59:42 -05:00
David Teigland
2419345b9d pvremove: use consistent error message
When the device is not a PV print
  "No PV found on device ..."

instead of
  "Failed to read lvm info for ... PVID ."

an earlier check had been added with a different
message for the same condition.
2021-05-04 13:08:02 -05:00
David Teigland
4dc5d4ac7e label_read_pvid: separate error and no-pvid
error reading dev and no pvid on dev were both
returning 0.  make it easier for callers to
know which, if they care.

return 1 if the device could be read, regardless
of whether a pvid was found or not.
set has_pvid=1 if a pvid is found and 0 if no
pvid is found.
2021-04-23 17:37:08 -05:00
David Teigland
fcbed26393 Revert "cov: check label_read_pvid return value"
This reverts commit bf461b99c6.

label_read_pvid returns 0 for non-PVs and these callers want
to handle non-PVs.
2021-04-23 17:12:24 -05:00
Zdenek Kabelac
9cdd4dcca7 make: generate 2021-04-23 23:02:58 +02:00
Zdenek Kabelac
64a8505b96 tests: use should for expected state
While we heavily try to spot arrays that are not yet in-sync,
some kernels tends to block our lvm2 command in kernel,
while we resume these smaller raid arrays even for 5 seconds.

But since the result is not really wrong - report these
check failures only as TEST WARNING.
2021-04-23 23:00:55 +02:00
Zdenek Kabelac
05eb90db68 cleanup: indent 2021-04-23 23:00:55 +02:00
Zdenek Kabelac
fccd6e0346 makefiles: add target for man-generator
Add supporting target for recreating man-generator when dependencies needs it.
2021-04-23 23:00:55 +02:00
Zdenek Kabelac
348c46c8fc man: add some resizing examples
Add some examples with -l% usage.
2021-04-23 23:00:55 +02:00
Zdenek Kabelac
a21028dea7 man: add missing _iorig suffix 2021-04-23 23:00:55 +02:00
Zdenek Kabelac
579c941321 man: document fsadm -l option
Missed -l option in man page, although users should prefer
lvresize -r when the also want to do a volume management,
as there they can specify i.e. extents for allocation.
Also mention dm-crypt support in command description.
2021-04-23 23:00:55 +02:00
Zdenek Kabelac
6f61de3009 args.h: bold command refference 2021-04-23 23:00:55 +02:00
Zdenek Kabelac
f678052385 shellcheck: updates
Ehance some shellcheck reported issues.
2021-04-23 23:00:55 +02:00
Zdenek Kabelac
6a099707c4 clang: remove unused assignment 2021-04-23 23:00:55 +02:00
Zdenek Kabelac
49caa9f3b1 clang: ensure vg is defined 2021-04-23 23:00:55 +02:00
Zdenek Kabelac
51fd232b8e clang: avoid possible use-after-free
If the 'act' has been already processed by add_client_result()
it could have been possibly release - so avoid accessin 'act->'
afterward and go for next item directly.
2021-04-23 23:00:55 +02:00
Zdenek Kabelac
21bdd0a359 clang: always initialized values 2021-04-23 23:00:55 +02:00
Zdenek Kabelac
8f85834a33 cov: convert to code that analyzer may better understand
Switch to code that is problematic to analyzer
2021-04-23 23:00:55 +02:00
Zdenek Kabelac
25b672417e cov: use correct enum type
Correct function declaration.
2021-04-23 23:00:55 +02:00
Zdenek Kabelac
395ce6c2bb cov: explicitely ignore return value 2021-04-23 23:00:55 +02:00
Zdenek Kabelac
2b3dcd754f cov: check return value
Log problems on fail path.
2021-04-23 23:00:55 +02:00