1
0
mirror of git://sourceware.org/git/lvm2.git synced 2024-12-22 17:35:59 +03:00
Commit Graph

368 Commits

Author SHA1 Message Date
David Teigland
9df6f601e0 Remove code for loading other metadata formats
other formats are not used.
2018-06-13 12:03:42 -05:00
Joe Thornber
d5da55ed85 device_mapper: remove dbg_malloc.
I wrote dbg_malloc before we had valgrind.  These days there's just
no need.
2018-06-08 13:40:53 +01:00
Heinz Mauelshagen
de66704253 segtype: add linear
Add linear segtype addressing FIXME in preparation
for linear <-> striped convenience conversion support
2018-06-05 16:23:18 +02:00
Joe Thornber
7f97c7ea9a build: Don't generate symlinks in include/ dir
As we start refactoring the code to break dependencies (see doc/refactoring.txt),
I want us to use full paths in the includes (eg, #include "base/data-struct/list.h").
This makes it more obvious when we're breaking abstraction boundaries, eg, including a file in
metadata/ from base/
2018-05-14 10:30:20 +01:00
David Teigland
57bb46c5e7 filter: use bcache for filter reads
Filters are still applied before any device reading or
the label scan, but any filter checks that want to read
the device are skipped and the device is flagged.

After bcache is populated, but before lvm looks for
devices (i.e. before label scan), the filters are
reapplied to the devices that were flagged above.
The filters will then find the data they need in
bcache.
2018-05-10 16:03:19 -05:00
David Teigland
c016b573ee clvmd: separate saved_vg from vginfo
The clvmd saved_vg data is independent from the normal lvm
lvmcache vginfo data, so separate saved_vg from vginfo.
Normal lvm doesn't need to use save_vg at all, and in clvmd,
lvmcache changes on vginfo can be made without worrying
about unwanted effects on saved_vg.
2018-05-03 14:54:48 -05:00
David Teigland
977d0a3613 filters: increase MAX_FILTERS for new filter
The new signature filter was added without increasing this.
2018-05-02 14:10:30 -05:00
David Teigland
24e7745d7a devices: ignore lvm1 and pool devices 2018-05-01 15:18:47 -05:00
David Teigland
c1cd18f21e Remove lvm1 and pool disk formats
There are likely more bits of code that can be removed,
e.g. lvm1/pool-specific bits of code that were identified
using FMT flags.

The vgconvert command can likely be reduced further.

The lvm1-specific config settings should probably have
some other fields set for proper deprecation.
2018-04-30 16:55:02 -05:00
David Teigland
c0973e70a5 dev_cache: clean up scan
Pull out all of the twisted logic and simply call dev_cache_scan
at the start of the command prior to label scan.
2018-04-20 11:22:48 -05:00
David Teigland
d9a77e8bb4 lvmcache: simplify metadata cache
The copy of VG metadata stored in lvmcache was not being used
in general.  It pretended to be a generic VG metadata cache,
but was not being used except for clvmd activation.  There
it was used to avoid reading from disk while devices were
suspended, i.e. in resume.

This removes the code that attempted to make this look
like a generic metadata cache, and replaces with with
something narrowly targetted to what it's actually used for.

This is a way of passing the VG from suspend to resume in
clvmd.  Since in the case of clvmd one caller can't simply
pass the same VG to both suspend and resume, suspend needs
to stash the VG somewhere that resume can grab it from.
(resume doesn't want to read it from disk since devices
are suspended.)  The lvmcache vginfo struct is used as a
convenient place to stash the VG to pass it from suspend
to resume, even though it isn't related to the lvmcache
or vginfo.  These suspended_vg* vginfo fields should
not be used or touched anywhere else, they are only to
be used for passing the VG data from suspend to resume
in clvmd.  The VG data being passed between suspend and
resume is never modified, and will only exist in the
brief period between suspend and resume in clvmd.

suspend has both old (current) and new (precommitted)
copies of the VG metadata.  It stashes both of these in
the vginfo prior to suspending devices.  When vg_commit
is successful, it sets a flag in vginfo as before,
signaling the transition from old to new metadata.

resume grabs the VG stashed by suspend.  If the vg_commit
happened, it grabs the new VG, and if the vg_commit didn't
happen it grabs the old VG.  The VG is then used to resume
LVs.

This isolates clvmd-specific code and usage from the
normal lvm vg_read code, making the code simpler and
the behavior easier to verify.

Sequence of operations:

- lv_suspend() has both vg_old and vg_new
  and stashes a copy of each onto the vginfo:
  lvmcache_save_suspended_vg(vg_old);
  lvmcache_save_suspended_vg(vg_new);

- vg_commit() happens, which causes all clvmd
  instances to call lvmcache_commit_metadata(vg).
  A flag is set in the vginfo indicating the
  transition from the old to new VG:
  vginfo->suspended_vg_committed = 1;

- lv_resume() needs either vg_old or vg_new
  to use in resuming LVs.  It doesn't want to
  read the VG from disk since devices are
  suspended, so it gets the VG stashed by
  lv_suspend:
  vg = lvmcache_get_suspended_vg(vgid);

If the vg_commit did not happen, suspended_vg_committed
will not be set, and in this case, lvmcache_get_suspended_vg()
will return the old VG instead of the new VG, and it will
resume LVs based on the old metadata.
2018-04-20 11:22:45 -05:00
David Teigland
a7cb76ae94 scan: use bcache for label scan and vg read
New label_scan function populates bcache for each device
on the system.

The two read paths are updated to get data from bcache.

The bcache is not yet used for writing.  bcache blocks
for a device are invalidated when the device is written.
2018-04-20 11:19:24 -05:00
Joe Thornber
00f1b208a1 [io paths] Unpick agk's aio stuff 2018-04-20 11:03:58 -05:00
Zdenek Kabelac
afdbb28f72 toolcontext: light context missed to set-up mem mempool
If cmd->mem was null, then systemd generator was failing on:

(gdb) bt
dm_pool_alloc_aligned (p=0x0, s=96, alignment=8) at mm/pool-fast.c:95
dm_pool_alloc (p=0x0, s=96) at mm/pool-fast.c:90
dm_pool_zalloc (p=0x0, s=96) at mm/pool.c:74
config_file_read_fd (mem=0x0, cft=0x55f4339dbad0, dev=0x55f4339dfac0, reason=DEV_IO_MDA_CONTENT, offset=0, size=82293, offset2=0, size2=0,
    checksum_fn=0x0, checksum=0, checksum_only=0, no_dup_node_check=0, ioflags=0, config_file_read_fd_callback=0x0, config_file_read_fd_context=0x0) at config/config.c:567
config_file_read (mem=0x0, cft=0x55f4339dbad0) at config/config.c:658
config_file_open_and_read (config_file=0x7f49aef14540 <config_file> "/var/tmp/lvm/etc/lvm/lvm.conf", source=CONFIG_FILE, cmd=0x55f4339d6260)
    at config/config.c:282
_load_config_file (cmd=0x55f4339d6260, tag=0x7f49aeca15da "", local=0) at commands/toolcontext.c:824
_init_lvm_conf (cmd=0x55f4339d6260) at commands/toolcontext.c:853
create_config_context () at commands/toolcontext.c:1814
lvm_config_find_bool (libh=0x0, config_path=0x55f431a884ad "global/use_lvmetad", fail=0) at lvm_base.c:144
main ()
2018-02-12 22:13:53 +01:00
Zdenek Kabelac
83258e3385 toolcontext: do not change stream for pthreaded programs
With pthreaded daemons like 'dmeventd' using  liblvm via plugin,
lvm2 actually should not 'play' with streams at all - as there
could be parallel outputs running.

As a current quick workaround just disable change for pthreaded
program (gettid() != getpid()).

TODO: it's possible the change of buffering actually doesn't serve us
any measurable benefit and could be dropped as whole later...

Meanwhile this patch is fixing this occasional valgrind race report:

Invalid read of size 4
   at 0x571892C: vfprintf (in /usr/lib64/libc-2.26.9000.so)
   by 0x57216B3: fprintf (in /usr/lib64/libc-2.26.9000.so)
   by 0x5042886: dm_event_log (libdevmapper-event.c:925)
   by 0x10B015: _dmeventd_log (dmeventd.c:125)
   by 0x10D289: _unregister_for_event (dmeventd.c:1146)
   by 0x10E52E: _handle_request (dmeventd.c:1583)
   by 0x10E6D7: _do_process_request (dmeventd.c:1631)
   by 0x10E7C6: _process_request (dmeventd.c:1660)
   by 0x1101A4: main (dmeventd.c:2285)
 Address 0x6264d30 is 192 bytes inside a block of size 552 free'd
   at 0x4C2ED68: free (vg_replace_malloc.c:530)
   by 0x573907D: fclose@@GLIBC_2.2.5 (in /usr/lib64/libc-2.26.9000.so)
   by 0x6AC5C00: reopen_standard_stream (log.c:189)
   by 0x6A8E62C: destroy_toolcontext (toolcontext.c:2271)
   by 0x6BA5C22: lvm_fin (lvmcmdline.c:3339)
   by 0x6BD5EF3: lvm2_exit (lvmcmdlib.c:123)
   by 0x6856013: dmeventd_lvm2_exit (dmeventd_lvm.c:103)
   by 0x66535B8: unregister_device (dmeventd_thin.c:432)
   by 0x10CBBC: _do_unregister_device (dmeventd.c:926)
   by 0x10CD74: _monitor_unregister (dmeventd.c:979)
   by 0x10D094: _monitor_thread (dmeventd.c:1066)
   by 0x54B35E0: start_thread (in /usr/lib64/libpthread-2.26.9000.so)
   by 0x57C30EE: clone (in /usr/lib64/libc-2.26.9000.so)
 Block was alloc'd at
   at 0x4C2DBBB: malloc (vg_replace_malloc.c:299)
   by 0x573932B: fdopen@@GLIBC_2.2.5 (in /usr/lib64/libc-2.26.9000.so)
   by 0x6AC5DC2: reopen_standard_stream (log.c:200)
   by 0x6A8D11D: create_toolcontext (toolcontext.c:1898)
   by 0x6BA5B6B: init_lvm (lvmcmdline.c:3319)
   by 0x6BD5BC8: cmdlib_lvm2_init (lvmcmdlib.c:34)
   by 0x6BD5F04: lvm2_init (lvm2cmd.c:20)
   by 0x6855EA7: dmeventd_lvm2_init (dmeventd_lvm.c:67)
   by 0x665305F: register_device (dmeventd_thin.c:352)
   by 0x10CB7A: _do_register_device (dmeventd.c:916)
   by 0x10CEE4: _monitor_thread (dmeventd.c:1006)
   by 0x54B35E0: start_thread (in /usr/lib64/libpthread-2.26.9000.so)
   by 0x57C30EE: clone (in /usr/lib64/libc-2.26.9000.so)
....
Process terminating with default action of signal 6 (SIGABRT): dumping core
   at 0x570016B: raise (in /usr/lib64/libc-2.26.9000.so)
   by 0x5701520: abort (in /usr/lib64/libc-2.26.9000.so)
   by 0x57437D8: __libc_message (in /usr/lib64/libc-2.26.9000.so)
   by 0x5743831: __libc_fatal (in /usr/lib64/libc-2.26.9000.so)
   by 0x5744056: _IO_vtable_check (in /usr/lib64/libc-2.26.9000.so)
   by 0x574751C: __overflow (in /usr/lib64/libc-2.26.9000.so)
   by 0x574191A: fputc (in /usr/lib64/libc-2.26.9000.so)
   by 0x50428E3: dm_event_log (libdevmapper-event.c:934)
   by 0x10B015: _dmeventd_log (dmeventd.c:125)
   by 0x10D289: _unregister_for_event (dmeventd.c:1146)
   by 0x10E52E: _handle_request (dmeventd.c:1583)
   by 0x10E6D7: _do_process_request (dmeventd.c:1631)
   by 0x10E7C6: _process_request (dmeventd.c:1660)
   by 0x1101A4: main (dmeventd.c:2285)
2018-02-09 10:56:40 +01:00
Alasdair G Kergon
8c7bbcfb0f device: Basic config and setup to support async I/O. 2018-02-08 20:15:14 +00:00
Alasdair G Kergon
4b9806ab6f toolcontext: Add paired label_init to refresh_toolcontext.
label_init() and label_exit() should be paired.
2018-01-02 22:00:31 +00:00
Alasdair G Kergon
22b6c482ec config: Split config buffer processing into new fn.
Wrap its parameters into struct process_config_file_params allocated
from a mempool now passed into the config_file_read* fns.
2018-01-02 21:10:46 +00:00
Zdenek Kabelac
110dac870c cleanup: use existing define with prefix 2017-12-04 15:38:50 +01:00
Alasdair G Kergon
02e9876665 log: Add io debug class 2017-11-15 01:02:15 +00:00
Alasdair G Kergon
f1cc5b12fd tidy: Add missing underscores to statics. 2017-10-18 15:58:13 +01:00
David Teigland
6ac1e04b3a replicator: remove the code
It has not been used in a long time and is not
expected to be used further.
2017-10-13 16:20:42 -05:00
Alasdair G Kergon
3654f478e1 toolcontext: Improve invalid units error message. 2017-07-27 00:51:50 +01:00
Zdenek Kabelac
ec93f37b86 toolcontext: action for LVM_RUN_BY_DMEVENTD env var
When LVM_RUN_BY_DMEVENTD is set to 1, ensure there will
be no interaction with dmeventd.
2017-01-23 14:55:47 +01:00
Zdenek Kabelac
4a7f2155c1 clean: move code to lib part
Move actual processing part of the lvm2_disable_dmeventd_monitoring()
into a /lib part so we can reuse the code later for other cases.
2017-01-23 14:55:28 +01:00
Zdenek Kabelac
1d58074d9f debug: more stacktrace corrections
Continue previous patch dropping some unneeded stack traces
after printed log_error/warn messages.
2016-11-25 14:58:28 +01:00
Zdenek Kabelac
d8fc4d093e conf: support zero for missing_stripe_filler
Make it easier to replace missing segments with 'zero' returning
target - otherwise user would have to create some extra target
to provide zeros as /dev/zero can't be used (not a block device).

Also break code loop when segment is found and make it an INTERNAL_ERROR
where it's missing.
2016-11-11 16:58:16 +01:00
Zdenek Kabelac
e54cce245f cleanup: skip checking for just assigned string
When 'stripe_filler' has been just set to 'error',
do not check it again for not being 'error'.
2016-11-11 16:52:24 +01:00
Peter Rajnoha
b5e093624d toolcontext: read all configuration sources when checking config values in lvm2-activation-generator through lighweight toolcontext handler
We added lightweight toolcontext handle to avoid useless initialization
of some parts of the context and also to avoid problems when using the
handle very soon at system boot, like in lvm2-activation-generator
through lvm2app interface. However, we missed reading all the other
config sources like lvmlocal.conf as well as any tag config - we need to
read these too to get the final config value which may be overriden in
any of these additional config sources.

Currently, we use this lightweight toolcontext handle to read
global/use_lvmetad and global/use_lvmpolld config values in
lvm2-activation-generator using lvm2app interface (lvm_config_find_bool
lvm2app function).
2016-09-23 14:57:44 +02:00
Peter Rajnoha
e8985c71bc config: add support for CFG_DISALLOW_INTERACTIVE flag to mark settings as not suitable for override in interactive mode
Some settings are not suitable for override in interactive/shell
mode because such settings may confuse the code and it may end
up with unexpected behaviour. This is because of the fact that
once we're in the interactive/shell mode, we have already applied
some settings for the shell itself and we can't override them
further because we're already using those settings to drive the
interactive/shell mode. Such settings would get ignored silently
or, in worse case, they would mess up the existing configuration.
2016-08-09 18:49:11 +02:00
Peter Rajnoha
f21afddeb7 lvm: shell: extend log report to cover whole lvm shell's main loop
When lvm commands are executed in lvm shell, we cover the whole lvm
command execution within this shell now. That means, all messages logged
and status caught during each command execution is now recorded in the
log report, including overall command's return code.
2016-08-09 18:49:10 +02:00
Peter Rajnoha
1fde4bf4d0 refactor: move report grouping and log reporting handles from processing_handle to cmd_context
With patches that will follow, this will make it possible to widen log
report coverage when commands are executed from lvm shell so the amount
of messages that may end up in stderr/stdout instead of log report are
minimized.
2016-08-09 18:24:45 +02:00
Peter Rajnoha
06ce9b4e42 log: separate output and make it possible to use given FDs
Currently, the output is separated in 3 parts and each part can go into
a separate and user-defined file descriptor:

  - common output (stdout by default, customizable by LVM_OUT_FD environment variable)
  - error output (stderr by default, customizable by LVM_ERR_FD environment variable)
  - report output (stdout by default, customizable by LVM_REPORT_FD environment variable)

For example, each type of output goes to different output file:

  [0] fedora/~ # export LVM_REPORT_FD=3

  [0] fedora/~ # lvs fedora vg/abc 1>out 2>err 3>report

  [0] fedora/~ # cat out

  [0] fedora/~ # cat err
    Volume group "vg" not found
    Cannot process volume group vg

  [0] fedora/~ # cat report
    LV   VG     Attr       LSize   Layout     Role       CTime
    root fedora -wi-ao----  19.00g linear     public     Wed May 27 2015 08:09:21
    swap fedora -wi-ao---- 500.00m linear     public     Wed May 27 2015 08:09:21

Another example in LVM shell where the report goes to "report" file:

  [0] fedora/~ # export LVM_REPORT_FD=3
  [0] fedora/~ # lvm 3>report

  (in lvm shell)
  lvm> vgs

  (content of "report" file)
  [1] fedora/~ # cat report
    VG     #PV #LV #SN Attr   VSize  VFree
    fedora   1   2   0 wz--n- 19.49g    0

  (in lvm shell)
  lvm> lvs

  (content of "report" file)
  [1] fedora/~ # cat report
    VG     #PV #LV #SN Attr   VSize  VFree
    fedora   1   2   0 wz--n- 19.49g    0
    LV   VG     Attr       LSize   Layout     Role       CTime
    root fedora -wi-ao----  19.00g linear     public     Wed May 27 2015 08:09:21
    swap fedora -wi-ao---- 500.00m linear     public     Wed May 27 2015 08:09:21
2016-08-09 18:24:45 +02:00
Peter Rajnoha
751163a743 libdm: log: remove log_print_bypass_report calls and register new print_log_libdm for libdm during lvm initialization instead
This fixes commit f50d4011cd which
introduced a problem when using older lvm2 code with newer libdm.
In this case, the old LVM didn't recognize new _LOG_BYPASS_REPORT flag
that libdm-report code used. This ended up with no output at all
from libdm where log_print_bypass_report was called because the
_LOG_BYPASS_REPORT was not masked properly in lvm2's print_log fn
which was called as callback function for logging.

With this patch, the lvm2 registers separate print_log_libdm logging
function for libdm instead. The print_log_libdm is exactly the same
as print_log (used throughout lvm2 code) but it checks whether we're
printing common line on output where "common" means not going to stderr,
not a warning and not an error and if we are, it adds the
_LOG_BYPASS_REPORT flag so the log_print goes directly to output, not
to any log report.

So this achieves the same goal as in f50d4011cd,
just doing it in a way that newer libdm is still compatible with older
lvm2 code (libdm-report is the only code using log_print).

Looking at the opposite mixture - older libdm with newer lvm2 code,
that won't be compilable because the new log report functionality
that is in lvm2 also requires new dm_report_group_* libdm functions
so we don't need to care here.
2016-06-23 14:45:52 +02:00
Peter Rajnoha
79eaaee50a filters: add comments about internal filter position in filter chain 2016-06-23 09:37:51 +02:00
David Teigland
ebd2758dab vgimportclone: add native command
This is cleaner and more efficient than the script.
The args and usage are unchanged.
2016-06-22 13:13:10 -05:00
Peter Rajnoha
89e2aef63a tools: add 'lvm lastlog' command for interactive query and display of last command's log
If we're running in lvm shell, we can keep last command's log report
for further query with possible different selection criteria for easy
log lookup.
2016-06-20 11:33:43 +02:00
David Teigland
199b7b55c2 lvmcache: fix duplicate handling with multiple scans
Some commands scan labels to populate lvmcache multiple
times, i.e. lvmcache_init, scan labels to fill lvmcache,
lvmcache_destroy, then later repeat

Each time labels are scanned, duplicates are detected,
and preferred devices are chosen.  Each time this is done
within a single command, we want to choose the same
preferred devices.  So, check for existing preferences
when choosing preferred devices.

This also fixes a problem with the list of unused duplicate
devs when run in an lvm shell.  The devs had been allocated
from cmd memory, resulting in invalid list entries between
commands.
2016-06-07 15:15:51 -05:00
David Teigland
7fd4119d24 liblvm: allow config settings to be read without full lvm cmd
A program may be using liblvm2app for simply checking a config
setting in lvm.conf.  In this case, a full lvm context is not
needed, only cmd->cft (which are the config settings read from
lvm.conf).

lvm_config_find_bool() can now be passed a NULL lvm context
in which case it will only create cmd->cft, check the config
setting asked for, and destroy the cmd.
2016-05-18 09:04:25 -05:00
David Teigland
87d9406725 lib: fix init error handling
When setting up a toolcontext, the lib init function
was detecting an error when there was none, and then
it was returning an incompletely initialized cmd struct
instead of NULL.  The effect was that the lib would try
to use the uninitialized cmd struct and segfault.
This would happen if a non-fatal error occurred during
cmd setup, e.g. user permission failed on lvmetad socket,
causing cmd to fall back to scanning and not use lvmetad.

The only real error condition is when create_toolcontext
returns NULL.  If cmd is returned, the lib can use it.
2016-05-12 13:07:52 -05:00
David Teigland
1134ab6324 lvmetad: warn about making changes while not using lvmetad
If lvmetad is running, and a command opts to not use it
(--config global/use_lvmetad=0), and the command changes
metadata, then the metadata change is not visible to
lvmetad.  Subsequent commands using lvmetad to change
metadata may cause corruption based on the invalid
lvmetad state.

Eventually we can set the disabled state in lvmetad
to prevent this problem, but for now print a warning
about the possibility.
2016-04-21 10:28:01 -05:00
David Teigland
e44c4806db lvmetad: check pid for warning case
When command is not using lvmetad because
use_lvmetad=0 in the config, but the lvmetad
pidfile exists, print a warning (previously
this checked for the socket existing instead
of the pidfile existing.)
2016-04-21 09:50:59 -05:00
David Teigland
5e9e43074a lvmetad: rework command connection setup and checking
The lvmetad connection is created within the
init_connections() path during command startup,
rather than via the old lvmetad_active() check.

The old lvmetad_active() checks are replaced
with lvmetad_used() which is a simple check that
tests if the command is using/connected to lvmetad.

The old lvmetad_set_active(cmd, 0) calls, which
stopped the command from using lvmetad (to revert to
disk scanning), are replaced with lvmetad_make_unused(cmd).
2016-04-19 14:00:02 -05:00
David Teigland
2d5dc6512e dbus: add notification from commands
When a command modifies a PV or VG, or changes the
activation state of an LV, it will send a dbus
notification when the command is finished.  This
can be enabled/disabled with a config setting.
2016-03-07 10:06:09 -06:00
David Teigland
57cd94b9e3 pvs: replace 'unknown device' with [unknown]
A config setting can restore the old string.
2016-03-01 11:12:03 -06:00
M.H. Tsai
3b76e9fd98 config: fix verbose type to int
'verbose' was marked as a boolean option while it
takes integer args - so it has limited usage to 0 or 1,
but we supported 0-4 at least.

Fix it by switching to corrent int type.
(Hopefully noone was trying to use this variable as true/yes/false/no
way - as the would be unsupported/undocumented).
2016-02-11 18:30:39 +01:00
Peter Rajnoha
136fd8f2f6 conf: add metadata/check_pv_device_sizes 2016-01-22 14:16:00 +01:00
Zdenek Kabelac
fcbef05aae doc: change fsf address
Hmm rpmlint suggest fsf is using a different address these days,
so lets keep it up-to-date
2016-01-21 12:11:37 +01:00
Peter Rajnoha
1417ed304b cleanup: rename 'invisible devices' to 'hidden devices' 2016-01-13 15:22:46 +01:00
Peter Rajnoha
e168b5de75 conf: add report/mark_invisible_devices 2016-01-13 12:01:10 +01:00
Zdenek Kabelac
45e749493c cleanup: ensuring string is not NULL
Coverity cannot see the string cannot be NULL so make it explicit.
2015-11-09 17:04:10 +01:00
Zdenek Kabelac
72d700b064 toolcontext: do not change already set dm_log
If the dm_log is already set to non-default logger,
avoid to change it.
2015-10-13 14:39:37 +02:00
Peter Rajnoha
b77497cbd8 filters: make sure regex filter is evaluated before any filter that needs disk access
The regex filter (controlled by devices/filter lvm.conf setting) was
evaluated as the very last filter. However, this is not optimal when
it comes to restricting disk access - users define devices/filter
as well as devices/global_filter to avoid this.

The devices/global_filter is already positioned at the beginning of the
filter chain. We need to do the same for devices/filter.

Filter chains before this patch:

  A: when lvmetad is not used:
       persistent_filter -> sysfs_filter -> global_regex_filter ->
       type_filter -> usable->filter -> mpath_component_filter ->
       partition_filter -> md_component_filter -> fw_raid_filter ->
       regex_filter

  B: when lvmetad is used:

    B1: to update lvmetad:
      sysfs_filter -> global_regex_filter -> type_filter ->
      usable_filter -> mpath_component_filter -> partition_filter ->
      md_component_filter -> fw_raid_filter

    B2: to retrieve info from lvmetad:
      persistent_filter -> usable_filter -> regex_filter

From the chain list above we can see that particularly in case when
lvmetad is not used, the regex filter is the very last one that is
processed. If lvmetad is used, it doesn't matter much as there's
the global_regex_filter which is used instead when updating lvmetad
and when retrieving info from lvmetad, putting regex_filter in front
of usable_filter wouldn't change much since usabled_filter is not
reading disks directly.

This patch puts the regex filter to the front even in case lvmetad
is not used, hence reinstating the state as it was before commit
a7be3b12df (which moved the regex_filter
position in the chain). Still, the arguments for the commit
a7be3b12df still apply and they're
still satisfied since component filters (MD, mpath...) are evaluated
first just before updating lvmetad.

So with this patch, we end up with:

  A: when lvmetad is not used:
       persistent_filter -> sysfs_filter -> global_regex_filter ->
       regex_filter -> type_filter -> usable->filter ->
       mpath_component_filter -> partition_filter ->
       md_component_filter -> fw_raid_filter

  B: when lvmetad is used:

    B1: to update lvmetad:
      sysfs_filter -> global_regex_filter -> type_filter ->
      usable_filter -> mpath_component_filter -> partition_filter ->
      md_component_filter -> fw_raid_filter

    B2: to retrieve info from lvmetad:
      persistent_filter -> regex_filter -> usable_filter

This way, specifying the regex_filter in non-lvmetad case causes
the devices to be filtered based on regex first before processing
any other filters which can access disks (like md_component_filter).

This patch also streamlines the code for better readability.
2015-09-08 15:28:10 +02:00
Peter Rajnoha
b3997469b5 toolcontext: do not set cmd->initialized_connections = 0 on destroy_toolcontext
The whole cmd context is freed completely in destroy_toolcontext, so do
not write to any of dead cmd variables.
2015-08-03 16:17:17 +02:00
Peter Rajnoha
f54198eed6 toolcontext: use refresh_filters in refresh_toolcontext
Use refresh_filters instead of destroy_filters and init_filters
in refresh_toolcontext fn which deals with cmd->initialized.filters
correctly on refresh.
2015-07-31 10:25:36 +02:00
Peter Rajnoha
f6473baffc toolcontext: add switches to create_toolcontext for connections and filters init
Make it possible to decide whether we want to initialize connections and
filters together with toolcontext creation.

Add "filters" and "connections" fields to struct
cmd_context_initialized_parts and set these in cmd_context.initialized
instance accordingly.

(For now, all create_toolcontext calls do initialize connections and
filters, we'll change that in subsequent patch appropriately.)
2015-07-30 13:54:09 +02:00
Peter Rajnoha
3e343ba5ef refactor: toolcontext: move lvmetad and lvmpolld init into separate function
Move original lvmetad and lvmpolld initialization code from
_process_config fn to their own functions _init_lvmetad and
_init_lvmpolld (both covered with single _init_connections fn).
2015-07-30 13:54:09 +02:00
Peter Rajnoha
6b0c464a34 refactor: toolcontext: add struct cmd_context_initialized_parts
Add struct cmd_context_initialized_parts to wrap up information
about which cmd context pieces are initialized and add variable
of this struct type into struct cmd_context.

Also, move existing "config_initialized" variable that was directly
part of cmd_context into the new cmd_context.initialized wrapper.

We'll be adding more items into the struct cmd_context_initialized_parts
with subsequent patches...
2015-07-30 13:54:05 +02:00
Peter Rajnoha
34a4109946 config: use find_config_tree_array for all arrays
Use find_config_tree_array for all config arrays. Also, add
INTERNAL_ERROR in case there should have been at least default
value defined for a setting but it was not returned for some
reason (either config_settings.h misconfiguration or other config
tree error printed by functions called by find_config_tree_array).
2015-07-15 10:52:23 +02:00
Peter Rajnoha
3b6840e099 config: replace find_config_tree_node with find_config_tree_array where appropriate 2015-07-08 13:03:08 +02:00
Peter Rajnoha
67a61cce1b config: add find_config_tree_array
Before, we used general find_config_tree_node function to retrieve
array values. This had a downside where if the node was not found,
we had to insert default values directly in-situ after the
find_config_tree_node call. This way, we had two copies of default
values - one in config_settings.h and the other one directly in the
code where we found out that find_config_tree_node returned NULL and
hence we needed to fall back to defaults.

With separate find_config_tree_array used for array config values,
we keep all the defaults centrally in config_settings.h because
the new find_config_tree_array automatically returns these defaults
if it can't find any value set in the configuration.

This patch just makes the behaviour exactly the same for arrays as
for any other non-array type where we call find_config_tree_<type>
already, hence making the internal interface for handling array
values consistent with the rest of the config types.
2015-07-08 12:59:22 +02:00
Peter Rajnoha
77f0e7a450 cleanup: time: error out on incorrect time_format and indentation in config_settings.h 2015-06-29 16:17:33 +02:00
Peter Rajnoha
1587236089 toolcontext: use proper set of chars to check time format against 2015-06-29 14:45:53 +02:00
Peter Rajnoha
125cd06698 conf: make time format configurable
Make it possible to define format for time that is displayed.
The way the format is defined is equal to the way that is used
for strftime function, although not all formatting options as
used in strftime are available for LVM2 - the set is restricted
(e.g. we do not allow newline to be printed). The lvm.conf
comments contain the whole list that LVM2 accepts for time format
together with brief description (copied from strftime man page).

For example:
(defaults used - the format is the same as used before this patch)
$ lvs -o+time vg/lvol0 vg/lvol1
  LV    VG   Attr       LSize Time
  lvol0 vg   -wi-a----- 4.00m 2015-06-25 16:18:34 +0200
  lvol1 vg   -wi-a----- 4.00m 2015-06-29 09:17:11 +0200

(using 'time_format = "@%s"' in lvm.conf - number of seconds
since the Epoch)
$ lvs -o+time vg/lvol0 vg/lvol1
  LV    VG   Attr       LSize Time
  lvol0 vg   -wi-a----- 4.00m @1435241914
  lvol1 vg   -wi-a----- 4.00m @1435562231
2015-06-29 14:30:35 +02:00
Ondrej Kozina
c3d351ec9b new debug class for lvmpolld client code 2015-05-12 17:16:37 +02:00
Ondrej Kozina
e587b0677b lvmpolld: Add standalone polldaemon.
See doc/lvmpolld_overview.txt
2015-05-09 00:59:18 +01:00
Alasdair G Kergon
87941ccd17 systemid: Use correct mempool.
libmem is necessary when this ends up in cmd->system_id.
2015-03-18 23:25:30 +00:00
Zdenek Kabelac
1260b86b2b config: use timestamp with nanosecond precision
Since kernel 2.6 we can use more precise timestamping,
so e.g. we could better recognize configs are slightly
older then generated .cache file.
2015-03-18 13:42:56 +01:00
David Teigland
c32efc7f7e system_id: apply consistent naming
In log messages refer to it as system ID (not System ID).

Do not put quotes around the system_id string when printing.

On the command line use systemid.

In code, metadata, and config files use system_id.

In lvmsystemid refer to the concept/entity as system_id.
2015-02-27 13:32:00 -06:00
Alasdair G Kergon
4ff9abd01f toolcontext: Fix lvmlocal.conf load fail path. 2015-02-25 16:36:47 +00:00
Alasdair G Kergon
ac6a4cd707 systemid: Allow empty systemid with warnings.
Add warning messages when empty system ID is set.
2015-02-25 14:12:24 +00:00
Alasdair G Kergon
df227be37c lvm1: Reenable sys ID.
Move the lvm1 sys ID into vg->lvm1_system_id and reenable the #if 0
LVM1 code.  Still display the new-style system ID in the same
reporting field, though, as only one can be set.
Add a format feature flag FMT_SYSTEM_ON_PVS for LVM1 and disallow
access to LVM1 VGs if a new-style system ID has been set.
Treat the new vg->system_id as const.
2015-02-23 23:03:52 +00:00
Alasdair G Kergon
3d406e5a8d config: Adjust system_id defaults.
Allow cmd->unknown_system_id to be cleared during toolcontext
refresh.
Set a default value of "none" for global/system_id_source.
Allow local/system_id to be empty so it's not impossible for
a later config file to remove it.
2015-02-23 21:20:51 +00:00
Alasdair G Kergon
e15b439bf3 systemid: Define file content more precisely.
In a file containing a system ID:
  Any whitespace at the start of a line is ignored;
  Blank lines are ignored;
  Any characters after a # are ignored along with the #.
  The system ID is obtained by processing the first line with
non-ignored characters.
  If further lines with non-ignored characters follow, a warning is
issued.
2015-02-23 20:49:15 +00:00
Alasdair G Kergon
c2ed5feee5 systemid: Add warnings if invalid.
Add WARNING messages if there are problems setting the requested
system ID.
Ban "localhost" as a prefix regardless of the system_id_source.
Use cmd->hostname instead of calling uname again.
Make system_id_source values case-insensitive (as with new settings like
log_debug_classes) and also accept machine-id to match the filename.
2015-02-23 20:36:27 +00:00
Alasdair G Kergon
6b6934b009 toolcontext: Include system_id in refresh.
Move _init_system_id() into _process_config() so it's also called when
refreshing the toolcontext.  Add WARNING to log_warn messages.
2015-02-23 20:21:00 +00:00
Alasdair G Kergon
72e6888dc3 toolcontext: Move lvmlocal.conf loading.
Load lvmlocal.conf after basic initialisation from lvm.conf.
2015-02-23 20:11:00 +00:00
Alasdair G Kergon
0551d1c56e systemid: Require alphanumeric 1st character.
Require system ID to begin with an alphanumeric character.
Rename fn to make clear it's only validation for systemid
and always terminate result rather than imposing this on the caller.
2015-02-23 19:47:03 +00:00
Alasdair G Kergon
2a1189ebc3 config: Reinstate recursive tags setting.
In 2.02.99, _init_tags() inadvertently began to ignore the
dm_config_tree struct passed to it.  "tags" sections are not
merged together, so the "tags" section in the main config file was
being processed repeatedly and other "tags" sections were ignored.
2015-02-23 17:40:58 +00:00
David Teigland
d726246f78 system_id: warn if the system_id_file cannot be opened 2015-02-13 11:37:11 -06:00
David Teigland
8cdec4c434 system_id: use for VG ownership
See included lvmsystemid(7) for full description.
2015-02-13 10:10:27 -06:00
Peter Rajnoha
1c7a509bed filters: add firmware RAID filter
Just like MD filtering that detects components of software RAID (md),
add detection for firmware RAID.

We're not adding any native code to detect this - there are lots of
firmware RAIDs out there which is just out of LVM scope. However,
with current changes with which we're able to get device info from
external sources (e.g. external_device_info_source="udev"), we can
do this easily if the external device status source has this kind
of information - which is the case of "udev" source where the results
of blkid scans are stored.

This detection should cover all firmware RAIDs that blkid can detect and
which are identified as:
  ID_FS_TYPE = {adaptec,ddf,hpt45x,hpt37x,isw,jmicron,lsi_mega,nvidia,promise_fasttrack,silicon_medley,via}_raid_member
2015-01-30 13:17:12 +01:00
Peter Rajnoha
bf8943b0f6 conf: add devices/external_device_info_source to lvm.conf 2015-01-30 13:01:12 +01:00
Peter Rajnoha
c50a90c9e6 filter-composite: add external device info hooks
Composite filter is a filter that can put several filters in one set.
This patch adds a switch when creating the composite filter which will
enable or disable external device info handles for all the filters
the composite filter encompasses.

We want to use this external device info for majority of the filters
which are in the "lvmetad filter chain" (or the respective part if
we're not using lvmetad).

Following patches will use the enabled external device handle in
concrete filters from the composite filter...
2015-01-30 13:01:12 +01:00
Zdenek Kabelac
97cccfbf1c segtype: drop cmdcontex pointer
Users of context provide their cmd pointer,
so don't keep it inside segtype.
2014-10-30 23:58:49 +01:00
Zdenek Kabelac
fd1aa7fd5a segments: disable init of free segment
Since it is not needed and would require separate
code for validation disable its initialization.
2014-10-24 16:39:31 +02:00
Peter Rajnoha
5011cac9cf filters: add cmd->full_filter - composite of cmd->filter and cmd->lvmetad_filter
There are actually three filter chains if lvmetad is used:
  - cmd->lvmetad_filter used when when scanning devices for lvmetad
  - cmd->filter used when processing lvmetad responses
  - cmd->full_fiilter (which is just cmd->lvmetad_filter + cmd->filter chained together) used
    for remaining situations

This patch adds the third one - "cmd->full_filter" - currently this is
used if device processing does not fall into any of the groups before,
for example, devices which does not have the PV label yet and we're just
creating a new one or we're processing the devices where the list of the
devices (PVs) is not returned by lvmetad initially.

Currently, the cmd->full_filter is used exactly in these functions:
  - lvmcache_label_scan
  - _pvcreate_check
  - pvcreate_vol
  - lvmdiskscan
  - pvscan
  - _process_each_label

If lvmetad is used, then simply cmd->full_filter == cmd->filter because
cmd->lvmetad_filter is NULL in this case.
2014-10-02 13:06:46 +02:00
Peter Rajnoha
a5f01dad22 filters: refresh filters when lvmetad use is toggled
We need to use proper filter chain when we disable lvmetad use
explicitly in the code by calling lvmetad_set_active(0) while
overriding existing configuration. We need to reinitialize filters
in this case so proper filter chain is used. The same applies
for the other way round - when we enable lvmetad use explicitly in
the code (though this is not yet used).
2014-09-30 16:08:05 +02:00
Peter Rajnoha
c2981cf921 filters: use usable device filter and separate lvmetad filter chain so it's not reevaluated for any lvmetad response
With this change, the filter chains used look like this now:

  A) When *lvmetad is not used*:
    - persistent filter -> regex filter -> sysfs filter ->
      global regex filter -> type filter ->
      usable device filter(FILTER_MODE_NO_LVMETAD) ->
      mpath component filter -> partitioned filter ->
      md component filter

  B) When *lvmetad is used* (two separate filter chains):
     - the lvmetad filter chain used when scanning devs for lvmetad update:
       sysfs filter -> global regex filter -> type filter ->
       usable device filter(FILTER_MODE_PRE_LVMETAD) ->
       mpath component filter -> partitioned filter ->
       md component filter

     - the filter chain used for lvmetad responses:
       persistent filter -> usable device filter(FILTER_MODE_POST_LVMETAD) ->
       regex filter
2014-09-30 13:22:11 +02:00
Peter Rajnoha
80ac8f37d6 filters: fix incorrect filter indexing in composite filter array
Caused by recent changes - a7be3b12df.
If global filter was not defined, then part of the code
creating composite filter (the cmd->lvmetad_filter) incorrectly
increased index value even if this global filter was not created
as part of the composite filter. This caused a gap with "NULL"
value in the composite filter array which ended up with the rest
of the filters after the gap to be ignored and also it caused a mem
leak when destroying the composite filter.
2014-09-11 09:30:03 +02:00
Petr Rockai
a7be3b12df lvmetad: Re-organise filters to properly avoid scans of component devices.
If a PV label is exposed both through a composite device (MD for example) and
through its component devices, we always want the PV that lvmetad sees to be the
composite, since this is what all LVM commands (including activation) will then
use. If pvscan --cache is triggered for multiple clones of the same PV, the last
to finish wins. This patch basically re-arranges the filters so that
component-device filters are part of the global_filter chain, not of the
client-side filter chain. This has a subtle effect on filter evaluation order,
but should not alter visible semantics in the non-lvmetad case.
2014-09-10 22:58:02 +01:00
Peter Rajnoha
d169ff1e03 conf: add report/list_item_separator lvm.conf option
For example:

$ lvm dumpconfig report/list_item_separator
list_item_separator=","

$ lvs -o name,tags vg/lvol1
  LV    LV Tags
  lvol1 a,x,y

$ lvm dumpconfig report/list_item_separator
list_item_separator=":"

$ lvs -o name,tags vg/lvol1
  LV    LV Tags
  lvol1 ay
2014-07-10 16:18:45 +02:00
Peter Rajnoha
da545ce3b4 tools: add --binary arg to pvs,vgs,lvs and {pv,vg,lv}display -C and report/binary_values_as_numeric lvm.conf option
The --binary option, if used, causes all the binary values reported
in reporting commands to be displayed as "0" or "1" instead of descriptive
literal values (value "unknown" is still used for values that could not be
determined).

Also, add report/binary_values_as_numeric lvm.conf option with the same
functionality as the --binary option (the --binary option prevails
if both --binary cmd option and report/binary_values_as_numeric lvm.conf
option is used at the same time). The report/binary_values_as_numeric is
also profilable.

This makes it easier to use and check lvm reporting command output in scripts.
2014-07-04 15:40:17 +02:00
Peter Rajnoha
3208396ce5 coverity: fix issues reported by coverity 2014-06-24 14:58:53 +02:00
Zdenek Kabelac
00af0d13c9 cleanup: more readable
Older gcc complained a bit about uninitialized vars
so reorder code for better readability.
2014-06-19 12:02:48 +02:00
Peter Rajnoha
5abdb52fdc report: select: refactor: move str_list to libdm
The list of strings is used quite frequently and we'd like to reuse
this simple structure for report selection support too. Make it part
of libdevmapper for general reuse throughout the code.

This also simplifies the LVM code a bit since we don't need to
include and manage lvm-types.h anymore (the string list was the
only structure defined there).
2014-06-17 16:27:20 +02:00
Peter Rajnoha
9e3e4d6994 config: differentiate command and metadata profiles and consolidate profile handling code
- When defining configuration source, the code now uses separate
  CONFIG_PROFILE_COMMAND and CONFIG_PROFILE_METADATA markers
  (before, it was just CONFIG_PROFILE that did not make the
  difference between the two). This helps when checking the
  configuration if it contains correct set of options which
  are all in either command-profilable or metadata-profilable
  group without mixing these groups together - so it's a firm
  distinction. The "command profile" can't contain
  "metadata profile" and vice versa! This is strictly checked
  and if the settings are mixed, such profile is rejected and
  it's not used. So in the end, the CONFIG_PROFILE_COMMAND
  set of options and CONFIG_PROFILE_METADATA are mutually exclusive
  sets.

- Marking configuration with one or the other marker will also
  determine the way these configuration sources are positioned
  in the configuration cascade which is now:

  CONFIG_STRING -> CONFIG_PROFILE_COMMAND -> CONFIG_PROFILE_METADATA -> CONFIG_FILE/CONFIG_MERGED_FILES

- Marking configuration with one or the other marker will also make
  it possible to issue a command context refresh (will be probably
  a part of a future patch) if needed for settings in global profile
  set. For settings in metadata profile set this is impossible since
  we can't refresh cmd context in the middle of reading VG/LV metadata
  and for each VG/LV separately because each VG/LV can have a different
  metadata profile assinged and it's not possible to change these
  settings at this level.

- When command profile is incorrect, it's rejected *and also* the
  command exits immediately - the profile *must* be correct for the
  command that was run with a profile to be executed. Before this
  patch, when the profile was found incorrect, there was just the
  warning message and the command continued without profile applied.
  But it's more correct to exit immediately in this case.

- When metadata profile is incorrect, we reject it during command
  runtime (as we know the profile name from metadata and not early
  from command line as it is in case of command profiles) and we
  *do continue* with the command as we're in the middle of operation.
  Also, the metadata profile is applied directly and on the fly on
  find_config_tree_* fn call and even if the metadata profile is
  found incorrect, we still need to return the non-profiled value
  as found in the other configuration provided or default value.
  To exit immediately even in this case, we'd need to refactor
  existing find_config_tree_* fns so they can return error. Currently,
  these fns return only config values (which end up with default
  values in the end if the config is not found).

- To check the profile validity before use to be sure it's correct,
  one can use :

    lvm dumpconfig --commandprofile/--metadataprofile ProfileName --validate

  (the --commandprofile/--metadataprofile for dumpconfig will come
   as part of the subsequent patch)

- This patch also adds a reference to --commandprofile and
  --metadataprofile in the cmd help string (which was missing before
  for the --profile for some commands). We do not mention --profile
  now as people should use --commandprofile or --metadataprofile
  directly. However, the --profile is still supported for backward
  compatibility and it's translated as:

    --profile == --metadataprofile for lvcreate, vgcreate, lvchange and vgchange
                 (as these commands are able to attach profile to metadata)

    --profile == --commandprofile for all the other commands
                (--metadataprofile is not allowed there as it makes no sense)

- This patch also contains some cleanups to make the code handling
  the profiles more readable...
2014-05-20 16:21:48 +02:00
Peter Rajnoha
9a324df3b3 config: fix incorrect profile initialization on cmd context refresh
When cmd refresh is called, we need to move any already loaded profiles
to profiles_to_load list which will cause their reload on subsequent
use. In addition to that, we need to take into account any change
in config/profile configuration setting on cmd context refresh
since this setting could be overriden with --config.

Also, when running commands in the shell, we need to remove the
global profile used from the configuration cascade so the profile
is not incorrectly reused next time when the --profile option is
not specified anymore for the next command in the shell.

This bug only affected profile specified by --profile cmd line
arg, not profiles referenced from LVM metadata.
2014-05-19 15:39:55 +02:00
Peter Rajnoha
c42f72867a config: attach cft_check_handle to each config tree instead of global cmd_context
Before, the cft_check_handle used to direct configuration checking
was part of cmd_context. It's better to attach this as part of the
exact config tree against which the check is done. This patch moves
the cft_check_handle out of cmd_context and it attaches it to the
config tree directly as dm_config_tree->custom->config_source->check_handle.

This change makes it easier to track the config tree check results
and provides less space for bugs as the results are directly attached
to the tree and we don't need to be cautious whether the global value
is correct or not (and whether it needs reinitialization) as it was
in the case when the cft_check_handle was part of cmd_context.
2014-05-19 15:38:04 +02:00
Peter Rajnoha
4360fdf89c libdevmapper: add dm_units_to_factor for size unit parsing
Actually moving the existing code from LVM to libdm for reuse.
2014-04-28 10:25:43 +02:00