1
0
mirror of git://sourceware.org/git/lvm2.git synced 2024-12-21 13:34:40 +03:00
Commit Graph

7008 Commits

Author SHA1 Message Date
David Teigland
62533ae3fa vgchange -aay: optimize device list using pvs_online files
Port another optimization from pvscan -aay to vgchange -aay:
  "pvscan: only add device args to dev cache"

This optimization avoids doing a full dev_cache_scan, and
instead populates dev-cache with only the devices in the
VG being activated.

This involves shifting the use of pvs_online files from
the hints interface up to the higher level label_scan
interface.  This specialized label_scan is structured
around creating a list of devices from the pvs_online
files.  Previously, a list of all devices was created
first, and then reduced based on the pvs_online files.
The initial step of listing all devices was slow when
thousands of devices are present on the system.

This optimization extends the previous optimization that
used pvs_online files to limit the devices that were
actually scanned (i.e. reading to identify the device):
  "vgchange -aay: optimize device scan using pvs_online files"
2021-11-05 12:19:35 -05:00
David Teigland
f40fd88374 move code from pvscan.c to online.c
related to managing files in /run/lvm/pvs_online
and /run/lvm/vgs_online
2021-11-04 11:09:29 -05:00
David Teigland
594f6ca970 rename pvscan_cache_single to expect_missing_vg_device in cmd 2021-11-04 11:08:38 -05:00
David Teigland
726dd25969 add hints interface to the pvs_online file information
The information in /run/lvm/pvs_online/<pvid> files can
be used to build a list of devices for a given VG.

The pvscan -aay command has long used this information to
activate a VG while scanning only devices in that VG, which
is an important optimization for autoactivation.

This patch implements the same thing through the existing
device hints interface, so that the optimization can be
applied elsewhere.  A future patch will take advantage of
this optimization in vgchange -aay, which is now used in
place of pvscan -aay for event activation.
2021-11-04 10:58:16 -05:00
David Teigland
6ea8d975b2 lvmdevices: increase open file limit 2021-11-03 08:50:57 -05:00
David Teigland
b5b0369e4d filter-sysfs: skip when device id is set
When a device id is set for a device, using an idtype other
than devname, it means that sysfs has been used with the device
to match the device id.  So, checking for a sysfs entry for the
device in filter-sysfs is redundant.  For any other cases not
covered by this (e.g. devname ids), have filter-sysfs simply
stat /sys/dev/block/major:minor to test if the device exists
in sysfs.

The extensive processing done by filter-sysfs init is removed.
It was taking an immense amount of time with many devices, e.g.
. 1024 PVs in 520 VGs
. 520 concurrent vgchange -ay <vgname> commands
. vgchange scans only PVs in the named VG (based on pvs_online
  files from a pending patch)

A large number of the vgchange commands were taking over 1 min,
and nearly half of that time was used by filter-sysfs init.
With this patch, the vgchange commands take about half the time.
2021-11-02 16:54:53 -05:00
David Teigland
5d0964d127 hints: remove the cmd hints list
which is no longer used after commit
"toollib: remove all devices list from process_each_pv"
2021-11-01 16:01:45 -05:00
David Teigland
33e47182f7 pvscan: only add device args to dev cache
Optimize the common pvscan --cache command by only adding
the necessary devs to dev-cache.
2021-10-19 17:13:57 -05:00
Zdenek Kabelac
60dc44b707 dev-cache: enhance dir scan also for non-udev build 2021-10-18 21:50:56 +02:00
Zdenek Kabelac
88e0d68909 dev-cache: better detection of filesystem
It appear on some systems the first found dev might not be actually for
the filesytem - so use a better way through _cache.st_dev.
2021-10-18 21:16:53 +02:00
Zdenek Kabelac
a172a02a9a cleanup: use const char buffer 2021-10-18 19:17:27 +02:00
Zdenek Kabelac
9cf4eac250 dev-cache: skip different filesystems on dir scan
When scanning configured  /dev dir, avoid entring
directories with different filesystem.

This minimizes risk we will block on i.e. entring
directory with mount point.
2021-10-18 19:17:26 +02:00
Zdenek Kabelac
bae1083472 cov: check device_ids_write return code
At least 'stack' failure code path as the
function device_id_update_vg_uuid() is void.
2021-10-15 23:40:56 +02:00
Zdenek Kabelac
c2be6c38d5 cov: ensure id is always initialize
Always resed sd_id128_t id and report warning in case
sd_id128_get_machine_app_specific() does not exit with 0.
2021-10-15 23:40:56 +02:00
Zdenek Kabelac
882141eb8c cov: check pointer before dereferencing
Check pv2 is non-null before trying to deref its tags.
2021-10-15 23:40:56 +02:00
Zdenek Kabelac
2779830a06 cov: avoid using NULL info
Check lvmcache info exists before calling lvmcache_del_save_bad_mda().
2021-10-15 23:36:22 +02:00
Zdenek Kabelac
14cf579263 gcc: archive reoder struct vars
Easier for automatic struct initialization = { 0 } without warns
from older gcc.
2021-10-15 00:09:07 +02:00
Zdenek Kabelac
8784211958 debug: reduce some debugging noise
Resolve event_activation configure option just once.

Do not print debug_devs about 'bad' filtering, when
actually filter already printed reason for skipping

Do not trace more then once about backup being disabled.

No debug when unlinked file does not exists in pvscan.
2021-10-14 23:34:11 +02:00
Zdenek Kabelac
0bd5388b16 log: also build without systemd_journal_support 2021-10-14 23:34:11 +02:00
Zdenek Kabelac
b73e1cd4b3 configure: updates 2021-10-14 23:34:11 +02:00
Zdenek Kabelac
af0ce203e9 debug: better error message
Add more details about size difference.
2021-10-14 23:34:11 +02:00
Zdenek Kabelac
7e346ee2a5 archiving: update refactoring
Commit 5ea426e656 handled case with
file path without '/' incorrectly - there is valid use-case so
switch 'log_error()' to valid code branch.
2021-10-14 23:33:26 +02:00
David Teigland
c38473548e fix segfault handling duplicate PVs
cmd arg was missing when switching to use an alternative
duplicate dev.
2021-10-14 14:02:59 -05:00
David Teigland
e88163ceac fix lvmcache_add log_debug pvid
from previous cleanup
2021-10-13 17:30:21 -05:00
David Teigland
6fb497ef42 toollib: remove all devices list from process_each_pv
Reporting non-PVs / "all devices" is only done by
pvs -a or pvdisplay -a, so avoid the work managing
a list of all devices in process_each_pv.
In the case when it's needed, use the results of
label_scan which already determines which devs
are not PVs.
2021-10-13 17:29:32 -05:00
David Teigland
272d1ccac8 devices file: unmatching system id should just be warning
there are likely more cases than not where the systemid
does not match and the command should still run.
2021-10-13 13:42:04 -05:00
David Teigland
062ea3c418 fix syslog setting
Just setting lvm.conf level=N should not send messages to
syslog (now the journal by default.)

Sending messages to syslog should require setting lvm.conf
log { syslog=1 level=N }.
2021-10-11 17:11:01 -05:00
David Teigland
bbf8e7db74 logging: to the systemd journal
Configure via lvm.conf log/journal or command line --journal.

Possible values:
"command" records command information.
"output" records default command output.
"debug" records full command debugging.

Multiple values can be set in lvm.conf as an array.
One value can be set in --journal which is added to
values set in lvm.conf
2021-10-07 12:06:49 -05:00
David Teigland
0b6782fa01 pvscan: add options listlvs listvg checkcomplete
pvscan --cache <dev>
    . read only dev
    . create online file for dev

pvscan --listvg <dev>
    . read only dev
    . list VG using dev

pvscan --listlvs <dev>
    . read only dev
    . list VG using dev
    . list LVs using dev

pvscan --cache --listvg [--checkcomplete] <dev>
    . read only dev
    . create online file for dev
    . list VG using dev
    . [check online files and report if VG is complete]

pvscan --cache --listlvs [--checkcomplete] <dev>
    . read only dev
    . create online file for dev
    . list VG using dev
    . list LVs using dev
    . [check online files and report if VG is complete]
    . [check online files and report if LVs are complete]

[--vgonline]
can be used with --checkcomplete, to enable use of a vg online
file.  This results in only the first pvscan command to see
the complete VG to report 'VG complete', and others will report
'VG finished'.  This allows the caller to easily run a single
activation of the VG.

[--udevoutput]
can be used with --cache --listvg --checkcomplete, to enable
an output mode that prints LVM_VG_NAME_COMPLETE='vgname' that
a udev rule can import, and prevents other output from the
command (other output causes udev to ignore the command.)

The list of complete LVs is meant to be passed to lvchange -aay,
or the complete VG used with vgchange -aay.

When --checkcomplete is used, lvm assumes that that the output
will be used to trigger event-based autoactivation, so the pvscan
does nothing if event_activation=0 and --checkcomplete is used.

Example of listlvs
------------------

$ lvs -a vg -olvname,devices
  LV     Devices
  lv_a   /dev/loop0(0)
  lv_ab  /dev/loop0(1),/dev/loop1(1)
  lv_abc /dev/loop0(3),/dev/loop1(3),/dev/loop2(1)
  lv_b   /dev/loop1(0)
  lv_c   /dev/loop2(0)

$ pvscan --cache --listlvs --checkcomplete /dev/loop0
  pvscan[35680] PV /dev/loop0 online, VG vg incomplete (need 2).
  VG vg incomplete
  LV vg/lv_a complete
  LV vg/lv_ab incomplete
  LV vg/lv_abc incomplete

$ pvscan --cache --listlvs --checkcomplete /dev/loop1
  pvscan[35681] PV /dev/loop1 online, VG vg incomplete (need 1).
  VG vg incomplete
  LV vg/lv_b complete
  LV vg/lv_ab complete
  LV vg/lv_abc incomplete

$ pvscan --cache --listlvs --checkcomplete /dev/loop2
  pvscan[35682] PV /dev/loop2 online, VG vg is complete.
  VG vg complete
  LV vg/lv_c complete
  LV vg/lv_abc complete

Example of listvg
-----------------

$ pvscan --cache --listvg --checkcomplete /dev/loop0
  pvscan[35684] PV /dev/loop0 online, VG vg incomplete (need 2).
  VG vg incomplete

$ pvscan --cache --listvg --checkcomplete /dev/loop1
  pvscan[35685] PV /dev/loop1 online, VG vg incomplete (need 1).
  VG vg incomplete

$ pvscan --cache --listvg --checkcomplete /dev/loop2
  pvscan[35686] PV /dev/loop2 online, VG vg is complete.
  VG vg complete
2021-10-07 12:06:49 -05:00
David Teigland
3e1316bb09 system_id: new appmachineid option
The new system_id_source="appmachineid" will cause
lvm to use an lvm-specific derivation of the machine-id,
instead of the machine-id directly.  This is now
recommended in place of using machineid.
2021-10-07 12:06:49 -05:00
David Teigland
6c22392a3f config: change default use_devicesfile to 1 2021-10-07 12:06:49 -05:00
David Teigland
02626cbd33 config: comment all default settings 2021-10-07 12:06:49 -05:00
Zdenek Kabelac
3cf97a737b cleanup: use dm_list_empty
Instant check for 'dm_list_size() == 0' without any size calc.
2021-10-06 15:43:31 +02:00
Zdenek Kabelac
197f6aaad7 cleanup: skip zeroing buffer overwritten by memcpy
Since the buffer is going to be overwritten by memcpy, reset only last
char to 0.

Also more use of DM_LIST_INIT list.h macro.
2021-10-06 15:43:31 +02:00
Zdenek Kabelac
79e65a3596 cleanup: all lvmcache_add users pass char with 0
No local copy needed, both users submit zero-end string as pv/vgid.
2021-10-06 15:43:31 +02:00
Zdenek Kabelac
5ccb373d86 cleanup: always initialize historical ptr
Set ptr for all paths.
2021-10-06 15:43:31 +02:00
Zdenek Kabelac
a841babaf7 cleanup: remove unused code 2021-10-06 15:43:31 +02:00
Zdenek Kabelac
5ea426e656 archiving: refactor code to allocate less memory
Do not store full path with each archived name reduces memory usage if
the directory has thousands of entries and just add 'dir' path when
needed.

Also emit info print message to a user if the total size of archived
files for a VG is more then 128MiB or 8192 files.

TODO: Consider wheather adding a new 'lvm.conf  archive{option}' to support
trimming these wild archive sizes can make situation better.
We already support retain_min && retain_days  - but if user is
generating too many and too large archives with minutes - maybe archiving
should be disabled by a user - as it's not producing anything largely usable
and just slows-down command ??
If we add 'retain_max & retain_max_size' the condition will go against
each other and we need to chose priorities.

mm
2021-10-06 15:42:53 +02:00
Zdenek Kabelac
98c57f7ce4 metadata: enhance internal error
Consider missing config tree from vg read to be an internal error
since we do not want to 'regenerate' this one in expesive parsing way.

Also if there is any failure on recreating committed VG, make it also
a 'vg_write' error.
2021-10-06 15:39:58 +02:00
Zdenek Kabelac
b7e9ec0cbf debug: fix stack tracing
Emit stack trace only on log_error path, and avoid on 'expected'
not-found path.
2021-10-06 15:39:58 +02:00
Zdenek Kabelac
e30f3c8410 metadata: simplify code 2021-10-06 15:39:58 +02:00
David Teigland
8e62cbb18e device_id: update stacked PVs for vgchange uuid
If a VG uuid is changed, update the device_id of any
PVs stacked on LVs in the changed VG.
2021-10-04 16:54:44 -05:00
David Teigland
939b4bc587 handle bad metadata text in vg_read path
Corrupt metadata text (with good mda header) was being handled
in the label_scan phase, but not in the vg_read phase.  This
was sufficient because metadata areas would always be read and
checksummed during label_scan (metadata parsing was skipped
previously as an optimization.)

This changed with the optimization in
commit 61a6f9905e
"metadata: optimize reading metadata copies in scan"

Now, some metadata areas will not be read and checksummed
at all during the label_scan phase, only during the vg_read
phase.  This means that bad metadata text may first be detected
in the vg_read phase.  So, add equivalent bad metadata handling
to the vg_read path to match the label_scan path.
2021-09-28 15:17:43 -05:00
Zdenek Kabelac
e3b4c365a4 debug: shorter code 2021-09-27 18:56:14 +02:00
Zdenek Kabelac
f74d30d411 cleanup: use already known lv size 2021-09-27 18:56:14 +02:00
Zdenek Kabelac
a8a958cc04 cleanup: reuse existing code for free path 2021-09-27 18:56:14 +02:00
Zdenek Kabelac
dab4acb8f8 make: build without versionsort
Reported-by: adamboardman of gemian
2021-09-27 18:56:14 +02:00
Zdenek Kabelac
df70dca995 make: opened can be needed for either defines case
Reported-by: adamboardman of gemian
2021-09-27 18:56:14 +02:00
Zdenek Kabelac
9971459d19 make: replace legacy use rindex with strrchr
Seems already dropped by some systems.

Reported-by: adamboardman of gemian
2021-09-27 18:56:14 +02:00
Zdenek Kabelac
e585c10fad make: fix compilation for undefined RAID_INTERNAL
Reported-by: adamboardman of gemian
2021-09-27 18:56:14 +02:00