lvchange: reject writemostly/writebehind on raid1 during resync

The MD kernel raid1 personality does no use any writemostly leg as the primary.

In case a previous linear LV holding data gets upconverted to
raid1 it becomes the primary leg of the new raid1 LV and a full
resynchronization is started to update the new legs.

No writemostly and/or writebehind setting may be allowed during
this initial, full synchronization period of this new raid1 LV
(using the lvchange(8) command), because that would change the
primary (i.e the previous linear LV) thus causing data loss.

lvchange has a bug not preventing this scenario.

Fix rejects setting writemostly and/or writebehind on resychronizing raid1 LVs.

Once we have status in the lvm2 metadata about the linear -> raid upconversion,
we may relax this constraint for other types of resynchronization
(e.g. for user requested "lvchange --resync ").

New lvchange-raid1-writemostly.sh test is added to the test suite.

Resolves: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=855895

Makefile.in

@@ -59,6 +59,8 @@ liblvm: lib
 daemons: lib libdaemon tools
 tools: lib libdaemon device-mapper
 po: tools daemons
+man: tools
+all_man: tools
 scripts: liblvm libdm
 
 lib.device-mapper: include.device-mapper

VERSION

@@ -1 +1 @@
-2.02.165(2)-git (2016-08-15)
+2.02.169(2)-git (2016-11-30)

VERSION_DM

@@ -1 +1 @@
-1.02.134-git (2016-08-15)
+1.02.138-git (2016-11-30)

WHATS_NEW

@@ -1,5 +1,140 @@
-Version 2.02.165 - 
-===================================
+Version 2.02.169 - 
+=====================================
+  Reject writemostly/writebehind in lvchange during resynchronization.
+  Deactivate active origin first before removal for improved workflow.
+  Fix regression of accepting options --type and -m with lvresize (2.02.158).
+  Add lvconvert --swapmetadata, new specific way to swap pool metadata LVs.
+  Add lvconvert --startpoll, new specific way to start polling conversions.
+  Add lvconvert --mergethin, new specific way to merge thin snapshots.
+  Add lvconvert --mergemirrors, new specific way to merge split mirrors.
+  Add lvconvert --mergesnapshot, new specific way to combine cow LVs.
+  Split up lvconvert code based on command definitions.
+  Split up lvchange code based on command definitions.
+  Generate help output and man pages from command definitions.
+  Verify all command line items against command definition.
+  Match every command run to one command definition.
+  Specify every allowed command definition/syntax in command-lines.in.
+  Add extra memory page when limiting pthread stack size in clvmd.
+  Support striped/raid0* <-> raid10_near conversions
+  Support shrinking of RaidLvs
+  Support region size changes on existing RaidLVs
+  Avoid parallel usage of cpg_mcast_joined() in clvmd with corosync.
+  Support raid6_{ls,rs,la,ra}_6 segment types and conversions from/to it.
+  Support raid6_n_6 segment type and conversions from/to it.
+  Support raid5_n segment type and conversions from/to it.
+  Support new internal command _dmeventd_thin_command.
+  Introduce new dmeventd/thin_command configurable setting.
+  Use new default units 'r' for displaying sizes.
+  Also unmount mount point on top of MD device if using blkdeactivate -u.
+  Restore check preventing resize of cache type volumes (2.02.158).
+  Add missing udev sync when flushing dirty cache content.
+  vgchange -p accepts only uint32 numbers.
+  Report thin LV date for merged LV when the merge is in progress.
+  Detect if snapshot merge really started before polling for progress.
+  Checking LV for merging origin requires also it has merged snapshot.
+  Extend validation of metadata processing.
+  Enable usage of cached volumes as snapshot origin LV.
+  Fix displayed lv name when splitting snapshot (2.02.146).
+  Warn about command not making metadata backup just once per command.
+  Enable usage of cached volume as thin volume's external origin.
+  Support cache volume activation with -real layer.
+  Improve search of lock-holder for external origin and thin-pool.
+  Support status checking of cache volume used in layer.
+  Avoid shifting by one number of blocks when clearing dirty cache volume.
+  Extend metadata validation of external origin LV use count.
+  Fix dm table when the last user of active external origin is removed.
+  Improve reported lvs status for active external origin volume.
+  Fix table load for splitted RAID LV and require explicit activation.
+  Always active splitted RAID LV exclusively locally.
+  Do not use LV RAID status bit for segment status.
+  Check segtype directly instead of checking RAID in segment status.
+  Reusing exiting code for raid image removal.
+  Fix pvmove leaving -pvmove0 error device in clustered VG.
+  Avoid adding extra '_' at end of raid extracted images or metadata.
+  Optimize another _rmeta clearing code.
+  Fix deactivation of raid orphan devices for clustered VG.
+  Fix lvconvert raid1 to mirror table reload order.
+  Add internal function for separate mirror log preparation.
+  Fix segfault in lvmetad from missing NULL in daemon_reply_simple.
+  Simplify internal _info_run() and use _setup_task_run() for mknod.
+  Better API for internal function _setup_task_run.
+  Avoid using lv_has_target_type() call within lv_info_with_seg_status.
+  Simplify internal lv_info_with_seg_status API.
+  Decide which status is needed in one place for lv_info_with_seg_status.
+  Fix matching of LV segment when checking for it info status.
+  Report log_warn when status cannot be parsed.
+  Test segment type before accessing segment members when checking status.
+  Implement compatible target function for stripe segment.
+  Use status info to report merge failed and snapshot invalid lvs fields.
+
+Version 2.02.168 - 30th November 2016
+=====================================
+  Display correct sync_percent on large RaidLVs
+  lvmdbusd --blackboxsize <n> added, used to override default size of 16
+  Allow a transiently failed RaidLV to be refreshed
+  Use lv_update_and_reload() inside mirror code where it applies.
+  Preserve mirrored status for temporary layered mirrors.
+  Use transient raid check before repairing raid volume.
+  Implement transient status check for raid volumes.
+  Only log msg as debug if lvm2-lvmdbusd unit missing for D-Bus notification.
+  Avoid duplicated underscore in name of extracted LV image.
+  Missing stripe filler now could be also 'zero'.
+  lvconvert --repair accepts --interval and --background option.
+  More efficiently prepare _rmeta devices when creating a new raid LV.
+
+Version 2.02.167 - 5th November 2016
+====================================
+  Use log_error in regex and sysfs filter to describe reason of failure.
+  Fix blkdeactivate to deactivate dev stack if dev on top already unmounted.
+  Prevent non-synced raid1 repair unless --force
+  Prevent raid4 creation/conversion on non-supporting kernels
+  Add direct striped -> raid4 conversion
+  Fix raid4 parity image pair position on conversions from striped/raid0*
+  Fix a few unconverted return code values for some lvconvert error path.
+  Disable lvconvert of thin pool to raid while active.
+  Disable systemd service start rate limiting for lvm2-pvscan@.service.
+
+Version 2.02.166 - 26th September 2016
+======================================
+  Fix lvm2-activation-generator to read all LVM2 config sources. (2.02.155)
+  Fix lvchange-rebuild-raid.sh to cope with older target versions.
+  Use dm_config_parse_without_dup_node_check() to speedup metadata reading.
+  Fix lvconvert --repair regression
+  Fix reported origin lv field for cache volumes. (2.02.133)
+  Always specify snapshot cow LV for monitoring not internal LV. (2.02.165)
+  Fix lvchange --discard|--zero for active thin-pool.
+  Enforce 4MiB or 25% metadata free space for thin pool operations.
+  Fix lock-holder device for thin pool with inactive thin volumes.
+  Use --alloc normal for mirror logs even if the mimages were stricter.
+  Use O_DIRECT to gather metadata in lvmdump.
+  Ignore creation_time when checking for matching metadata for lvmetad.
+  Fix possible NULL pointer derefence when checking for monitoring.
+  Add lvmreport(7) man page.
+  Don't install lvmraid(7) man page when raid excluded. (2.02.165)
+  Report 0% as dirty (copy%) for cache without any used block.
+  Fix lvm2api reporting of cache data and metadata percent.
+  Restore reporting of metadata usage for cache volumes (2.02.155).
+  Support raid scrubbing on cache origin LV.
+
+Version 2.02.165 - 7th September 2016
+=====================================
+  Add lvmraid(7) man page.
+  Use udev db to check for mpath components before running pvscan for lvmetad.
+  Use lsblk -s and lsblk -O in lvmdump only if these options are supported.
+  Fix number of stripes shown in lvcreate raid10 message when too many.
+  Change lvmdbusd to use new lvm shell facilities.
+  Do not monitor cache-pool metadata when LV is just being cleared.
+  Add allocation/cache_pool_max_chunks to prevent misuse of cache target.
+  Give error not segfault in lvconvert --splitmirrors when PV lies outside LV.
+  Fix typo in report/columns_as_rows config option name recognition (2.02.99).
+  Avoid PV tags when checking allocation against parallel PVs.
+  Disallow mirror conversions of raid10 volumes.
+  Fix dmeventd unmonitoring when segment type (and dso) changes.
+  Don't allow lvconvert --repair on raid0 devices or attempt to monitor them.
+  No longer adjust incorrect number of raid stripes supplied to lvcreate.
+  Move lcm and gcd to lib/misc.
+  Fix vgsplit of external origins. (2.02.162)
+  Prohibit creation of RAID LVs unless VG extent size is at least the page size.
   Suppress some unnecessary --stripesize parameter warnings.
   Fix 'pvmove -n name ...' to prohibit collocation of RAID SubLVs
 

WHATS_NEW_DM

@@ -1,5 +1,50 @@
-Version 1.02.134 - 
-===================================
+Version 1.02.138 - 
+=====================================
+  Add extra memory page when limiting pthread stack size in dmeventd.
+  Avoids immediate resume when preloaded device is smaller.
+  Do not suppress kernel key description in dmsetup table output.
+  Support configurable command executed from dmeventd thin plugin.
+  Support new R|r human readable units output format.
+  Thin dmeventd plugin reacts faster on lvextend failure path with umount.
+  Add dm_stats_bind_from_fd() to bind a stats handle from a file descriptor.
+  Do not try call callback when reverting activation on error path.
+  Fix file mapping for extents with physically adjacent extents.
+  Validation vsnprintf result in runtime translate of dm_log (1.02.136).
+  Separate filemap extent allocation from region table.
+  Fix segmentation fault when filemap region creation fails.
+  Fix performance of region cleanup for failed filemap creation.
+  Fix very slow region deletion with many regions.
+
+Version 1.02.137 - 30th November 2016
+=====================================
+  Document raid status values.
+  Always exit dmsetup with success when asked to display help/version.
+
+Version 1.02.136 - 5th November 2016
+====================================
+  Log failure of raid device with log_error level.
+  Use dm_log_with_errno and translate runtime to dm_log only when needed.
+  Make log messages from dm and lvm library different from dmeventd.
+  Notice and Info messages are again logged from dmeventd and its plugins.
+  Dmeventd now also respects DM_ABORT_ON_INTERNAL_ERRORS as libdm based tool.
+  Report as non default dm logging also when logging with errno was changed.
+  Use log_level() macro to consistently decode message log level in dmeventd.
+  Still produce output when dmsetup dependency tree building finds dev missing.
+  Check and report pthread_sigmask() failure in dmeventd.
+  Check mem alloc fail in _canonicalize_field_ids().
+  Use unsigned math when checking more then 31 legs of raid.
+  Fix 'dmstats delete' with dmsetup older than v1.02.129
+  Fix stats walk segfault with dmsetup older than v1.02.129
+
+Version 1.02.135 - 26th September 2016
+======================================
+  Fix man entry for dmsetup status.
+  Introduce new dm_config_parse_without_dup_node_check().
+  Don't omit last entry in dmstats list --group.
+
+Version 1.02.134 - 7th September 2016
+=====================================
+  Improve explanation of udev fallback in libdevmapper.h.
 
 Version 1.02.133 - 10th August 2016
 ===================================

acinclude.m4

@@ -61,3 +61,174 @@ AC_DEFUN([AC_TRY_LDFLAGS],
         ifelse([$4], [], [:], [$4])
     fi
 ])
+
+# ===========================================================================
+#      http://www.gnu.org/software/autoconf-archive/ax_gcc_builtin.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+#   AX_GCC_BUILTIN(BUILTIN)
+#
+# DESCRIPTION
+#
+#   This macro checks if the compiler supports one of GCC's built-in
+#   functions; many other compilers also provide those same built-ins.
+#
+#   The BUILTIN parameter is the name of the built-in function.
+#
+#   If BUILTIN is supported define HAVE_<BUILTIN>. Keep in mind that since
+#   builtins usually start with two underscores they will be copied over
+#   into the HAVE_<BUILTIN> definition (e.g. HAVE___BUILTIN_EXPECT for
+#   __builtin_expect()).
+#
+#   The macro caches its result in the ax_cv_have_<BUILTIN> variable (e.g.
+#   ax_cv_have___builtin_expect).
+#
+#   The macro currently supports the following built-in functions:
+#
+#    __builtin_assume_aligned
+#    __builtin_bswap16
+#    __builtin_bswap32
+#    __builtin_bswap64
+#    __builtin_choose_expr
+#    __builtin___clear_cache
+#    __builtin_clrsb
+#    __builtin_clrsbl
+#    __builtin_clrsbll
+#    __builtin_clz
+#    __builtin_clzl
+#    __builtin_clzll
+#    __builtin_complex
+#    __builtin_constant_p
+#    __builtin_ctz
+#    __builtin_ctzl
+#    __builtin_ctzll
+#    __builtin_expect
+#    __builtin_ffs
+#    __builtin_ffsl
+#    __builtin_ffsll
+#    __builtin_fpclassify
+#    __builtin_huge_val
+#    __builtin_huge_valf
+#    __builtin_huge_vall
+#    __builtin_inf
+#    __builtin_infd128
+#    __builtin_infd32
+#    __builtin_infd64
+#    __builtin_inff
+#    __builtin_infl
+#    __builtin_isinf_sign
+#    __builtin_nan
+#    __builtin_nand128
+#    __builtin_nand32
+#    __builtin_nand64
+#    __builtin_nanf
+#    __builtin_nanl
+#    __builtin_nans
+#    __builtin_nansf
+#    __builtin_nansl
+#    __builtin_object_size
+#    __builtin_parity
+#    __builtin_parityl
+#    __builtin_parityll
+#    __builtin_popcount
+#    __builtin_popcountl
+#    __builtin_popcountll
+#    __builtin_powi
+#    __builtin_powif
+#    __builtin_powil
+#    __builtin_prefetch
+#    __builtin_trap
+#    __builtin_types_compatible_p
+#    __builtin_unreachable
+#
+#   Unsuppored built-ins will be tested with an empty parameter set and the
+#   result of the check might be wrong or meaningless so use with care.
+#
+# LICENSE
+#
+#   Copyright (c) 2013 Gabriele Svelto <gabriele.svelto@gmail.com>
+#
+#   Copying and distribution of this file, with or without modification, are
+#   permitted in any medium without royalty provided the copyright notice
+#   and this notice are preserved.  This file is offered as-is, without any
+#   warranty.
+
+#serial 3
+
+AC_DEFUN([AX_GCC_BUILTIN], [
+    AS_VAR_PUSHDEF([ac_var], [ax_cv_have_$1])
+
+    AC_CACHE_CHECK([for $1], [ac_var], [
+        AC_LINK_IFELSE([AC_LANG_PROGRAM([], [
+            m4_case([$1],
+                [__builtin_assume_aligned], [$1("", 0)],
+                [__builtin_bswap16], [$1(0)],
+                [__builtin_bswap32], [$1(0)],
+                [__builtin_bswap64], [$1(0)],
+                [__builtin_choose_expr], [$1(0, 0, 0)],
+                [__builtin___clear_cache], [$1("", "")],
+                [__builtin_clrsb], [$1(0)],
+                [__builtin_clrsbl], [$1(0)],
+                [__builtin_clrsbll], [$1(0)],
+                [__builtin_clz], [$1(0)],
+                [__builtin_clzl], [$1(0)],
+                [__builtin_clzll], [$1(0)],
+                [__builtin_complex], [$1(0.0, 0.0)],
+                [__builtin_constant_p], [$1(0)],
+                [__builtin_ctz], [$1(0)],
+                [__builtin_ctzl], [$1(0)],
+                [__builtin_ctzll], [$1(0)],
+                [__builtin_expect], [$1(0, 0)],
+                [__builtin_ffs], [$1(0)],
+                [__builtin_ffsl], [$1(0)],
+                [__builtin_ffsll], [$1(0)],
+                [__builtin_fpclassify], [$1(0, 1, 2, 3, 4, 0.0)],
+                [__builtin_huge_val], [$1()],
+                [__builtin_huge_valf], [$1()],
+                [__builtin_huge_vall], [$1()],
+                [__builtin_inf], [$1()],
+                [__builtin_infd128], [$1()],
+                [__builtin_infd32], [$1()],
+                [__builtin_infd64], [$1()],
+                [__builtin_inff], [$1()],
+                [__builtin_infl], [$1()],
+                [__builtin_isinf_sign], [$1(0.0)],
+                [__builtin_nan], [$1("")],
+                [__builtin_nand128], [$1("")],
+                [__builtin_nand32], [$1("")],
+                [__builtin_nand64], [$1("")],
+                [__builtin_nanf], [$1("")],
+                [__builtin_nanl], [$1("")],
+                [__builtin_nans], [$1("")],
+                [__builtin_nansf], [$1("")],
+                [__builtin_nansl], [$1("")],
+                [__builtin_object_size], [$1("", 0)],
+                [__builtin_parity], [$1(0)],
+                [__builtin_parityl], [$1(0)],
+                [__builtin_parityll], [$1(0)],
+                [__builtin_popcount], [$1(0)],
+                [__builtin_popcountl], [$1(0)],
+                [__builtin_popcountll], [$1(0)],
+                [__builtin_powi], [$1(0, 0)],
+                [__builtin_powif], [$1(0, 0)],
+                [__builtin_powil], [$1(0, 0)],
+                [__builtin_prefetch], [$1("")],
+                [__builtin_trap], [$1()],
+                [__builtin_types_compatible_p], [$1(int, int)],
+                [__builtin_unreachable], [$1()],
+                [m4_warn([syntax], [Unsupported built-in $1, the test may fail])
+                 $1()]
+            )
+            ])],
+            [AS_VAR_SET([ac_var], [yes])],
+            [AS_VAR_SET([ac_var], [no])])
+    ])
+
+    AS_IF([test yes = AS_VAR_GET([ac_var])],
+        [AC_DEFINE_UNQUOTED(AS_TR_CPP(HAVE_$1), 1,
+            [Define to 1 if the system has the `$1' built-in function])], [])
+
+    AS_VAR_POPDEF([ac_var])
+])

aclocal.m4

@@ -536,4 +536,5 @@ AC_DEFUN([AM_RUN_LOG],
    echo "$as_me:$LINENO: \$? = $ac_status" >&AS_MESSAGE_LOG_FD
    (exit $ac_status); }])
 
+
 m4_include([acinclude.m4])

conf/command_profile_template.profile.in

@@ -39,7 +39,7 @@ report {
 	list_item_separator=","
 	prefixes=0
 	quoted=1
-	colums_as_rows=0
+	columns_as_rows=0
 	binary_values_as_numeric=0
 	time_format="%Y-%m-%d %T %z"
 	devtypes_sort="devtype_name"

conf/example.conf.in

@@ -428,6 +428,12 @@ allocation {
 	# 32KiB to 1GiB in multiples of 32.
 	# This configuration option does not have a default value defined.
 
+	# Configuration option allocation/cache_pool_max_chunks.
+	# The maximum number of chunks in a cache pool.
+	# For cache target v1.9 the recommended maximumm is 1000000 chunks.
+	# Using cache pool with more chunks may degrade cache performance.
+	# This configuration option does not have a default value defined.
+
 	# Configuration option allocation/thin_pool_metadata_require_separate_pvs.
 	# Thin pool metdata and data will always use different PVs.
 	thin_pool_metadata_require_separate_pvs = 0
@@ -659,7 +665,7 @@ global {
 
 	# Configuration option global/units.
 	# Default value for --units argument.
-	units = "h"
+	units = "r"
 
 	# Configuration option global/si_unit_consistency.
 	# Distinguish between powers of 1024 and 1000 bytes.
@@ -1150,7 +1156,8 @@ activation {
 	# Configuration option activation/missing_stripe_filler.
 	# Method to fill missing stripes when activating an incomplete LV.
 	# Using 'error' will make inaccessible parts of the device return I/O
-	# errors on access. You can instead use a device path, in which case,
+	# errors on access. Using 'zero' will return success (and zero) on I/O
+	# You can instead use a device path, in which case,
 	# that device will be used in place of missing stripes. Using anything
 	# other than 'error' with mirrored or snapshotted volumes is likely to
 	# result in data corruption.
@@ -1684,11 +1691,11 @@ activation {
 	# This configuration option has an automatic default value.
 	# quoted = 1
 
-	# Configuration option report/colums_as_rows.
+	# Configuration option report/columns_as_rows.
 	# Output each column as a row.
 	# If set, this also implies report/prefixes=1.
 	# This configuration option has an automatic default value.
-	# colums_as_rows = 0
+	# columns_as_rows = 0
 
 	# Configuration option report/binary_values_as_numeric.
 	# Use binary values 0 or 1 instead of descriptive literal values.
@@ -2042,6 +2049,15 @@ dmeventd {
 	# warning is repeated when 85%, 90% and 95% of the pool is filled.
 	thin_library = "libdevmapper-event-lvm2thin.so"
 
+	# Configuration option dmeventd/thin_command.
+	# The plugin runs command with each 5% increment when thin-pool data volume
+	# or metadata volume gets above 50%.
+	# Command which starts with 'lvm ' prefix is internal lvm command.
+	# You can write your own handler to customise behaviour in more details.
+	# User handler is specified with the full path starting with '/'.
+	# This configuration option has an automatic default value.
+	# thin_command = "lvm lvextend --use-policies"
+
 	# Configuration option dmeventd/executable.
 	# The full path to the dmeventd binary.
 	# This configuration option has an automatic default value.

configure

@@ -676,6 +676,7 @@ PTHREAD_LIBS
 M_LIBS
 POOL
 PKGCONFIG
+ODIRECT
 OCFDIR
 OCF
 MIRRORS
@@ -820,6 +821,8 @@ HAVE_PIE
 POW_LIB
 LIBOBJS
 ALLOCA
+SORT
+WC
 CHMOD
 CSCOPE_CMD
 CFLOW_CMD
@@ -5233,6 +5236,202 @@ else
   CHMOD="$ac_cv_path_CHMOD"
 fi
 
+if test -n "$ac_tool_prefix"; then
+  # Extract the first word of "${ac_tool_prefix}wc", so it can be a program name with args.
+set dummy ${ac_tool_prefix}wc; ac_word=$2
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
+$as_echo_n "checking for $ac_word... " >&6; }
+if ${ac_cv_path_WC+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  case $WC in
+  [\\/]* | ?:[\\/]*)
+  ac_cv_path_WC="$WC" # Let the user override the test with a path.
+  ;;
+  *)
+  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
+for as_dir in $PATH
+do
+  IFS=$as_save_IFS
+  test -z "$as_dir" && as_dir=.
+    for ac_exec_ext in '' $ac_executable_extensions; do
+  if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
+    ac_cv_path_WC="$as_dir/$ac_word$ac_exec_ext"
+    $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5
+    break 2
+  fi
+done
+  done
+IFS=$as_save_IFS
+
+  ;;
+esac
+fi
+WC=$ac_cv_path_WC
+if test -n "$WC"; then
+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: $WC" >&5
+$as_echo "$WC" >&6; }
+else
+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+fi
+
+
+fi
+if test -z "$ac_cv_path_WC"; then
+  ac_pt_WC=$WC
+  # Extract the first word of "wc", so it can be a program name with args.
+set dummy wc; ac_word=$2
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
+$as_echo_n "checking for $ac_word... " >&6; }
+if ${ac_cv_path_ac_pt_WC+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  case $ac_pt_WC in
+  [\\/]* | ?:[\\/]*)
+  ac_cv_path_ac_pt_WC="$ac_pt_WC" # Let the user override the test with a path.
+  ;;
+  *)
+  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
+for as_dir in $PATH
+do
+  IFS=$as_save_IFS
+  test -z "$as_dir" && as_dir=.
+    for ac_exec_ext in '' $ac_executable_extensions; do
+  if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
+    ac_cv_path_ac_pt_WC="$as_dir/$ac_word$ac_exec_ext"
+    $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5
+    break 2
+  fi
+done
+  done
+IFS=$as_save_IFS
+
+  ;;
+esac
+fi
+ac_pt_WC=$ac_cv_path_ac_pt_WC
+if test -n "$ac_pt_WC"; then
+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_WC" >&5
+$as_echo "$ac_pt_WC" >&6; }
+else
+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+fi
+
+  if test "x$ac_pt_WC" = x; then
+    WC=""
+  else
+    case $cross_compiling:$ac_tool_warned in
+yes:)
+{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5
+$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;}
+ac_tool_warned=yes ;;
+esac
+    WC=$ac_pt_WC
+  fi
+else
+  WC="$ac_cv_path_WC"
+fi
+
+if test -n "$ac_tool_prefix"; then
+  # Extract the first word of "${ac_tool_prefix}sort", so it can be a program name with args.
+set dummy ${ac_tool_prefix}sort; ac_word=$2
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
+$as_echo_n "checking for $ac_word... " >&6; }
+if ${ac_cv_path_SORT+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  case $SORT in
+  [\\/]* | ?:[\\/]*)
+  ac_cv_path_SORT="$SORT" # Let the user override the test with a path.
+  ;;
+  *)
+  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
+for as_dir in $PATH
+do
+  IFS=$as_save_IFS
+  test -z "$as_dir" && as_dir=.
+    for ac_exec_ext in '' $ac_executable_extensions; do
+  if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
+    ac_cv_path_SORT="$as_dir/$ac_word$ac_exec_ext"
+    $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5
+    break 2
+  fi
+done
+  done
+IFS=$as_save_IFS
+
+  ;;
+esac
+fi
+SORT=$ac_cv_path_SORT
+if test -n "$SORT"; then
+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: $SORT" >&5
+$as_echo "$SORT" >&6; }
+else
+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+fi
+
+
+fi
+if test -z "$ac_cv_path_SORT"; then
+  ac_pt_SORT=$SORT
+  # Extract the first word of "sort", so it can be a program name with args.
+set dummy sort; ac_word=$2
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
+$as_echo_n "checking for $ac_word... " >&6; }
+if ${ac_cv_path_ac_pt_SORT+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  case $ac_pt_SORT in
+  [\\/]* | ?:[\\/]*)
+  ac_cv_path_ac_pt_SORT="$ac_pt_SORT" # Let the user override the test with a path.
+  ;;
+  *)
+  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
+for as_dir in $PATH
+do
+  IFS=$as_save_IFS
+  test -z "$as_dir" && as_dir=.
+    for ac_exec_ext in '' $ac_executable_extensions; do
+  if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
+    ac_cv_path_ac_pt_SORT="$as_dir/$ac_word$ac_exec_ext"
+    $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5
+    break 2
+  fi
+done
+  done
+IFS=$as_save_IFS
+
+  ;;
+esac
+fi
+ac_pt_SORT=$ac_cv_path_ac_pt_SORT
+if test -n "$ac_pt_SORT"; then
+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_SORT" >&5
+$as_echo "$ac_pt_SORT" >&6; }
+else
+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+fi
+
+  if test "x$ac_pt_SORT" = x; then
+    SORT=""
+  else
+    case $cross_compiling:$ac_tool_warned in
+yes:)
+{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5
+$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;}
+ac_tool_warned=yes ;;
+esac
+    SORT=$ac_pt_SORT
+  fi
+else
+  SORT="$ac_cv_path_SORT"
+fi
+
 
 ################################################################################
 ac_header_dirent=no
@@ -6320,6 +6519,50 @@ _ACEOF
   esac
 
 
+
+
+    { $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_clz" >&5
+$as_echo_n "checking for __builtin_clz... " >&6; }
+if ${ax_cv_have___builtin_clz+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+
+        cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h.  */
+
+int
+main ()
+{
+
+            __builtin_clz(0)
+
+  ;
+  return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+  ax_cv_have___builtin_clz=yes
+else
+  ax_cv_have___builtin_clz=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+    conftest$ac_exeext conftest.$ac_ext
+
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ax_cv_have___builtin_clz" >&5
+$as_echo "$ax_cv_have___builtin_clz" >&6; }
+
+    if test yes = $ax_cv_have___builtin_clz; then :
+
+cat >>confdefs.h <<_ACEOF
+#define HAVE___BUILTIN_CLZ 1
+_ACEOF
+
+fi
+
+
+
+
 ################################################################################
 for ac_func in ftruncate gethostname getpagesize gettimeofday localtime_r \
   memchr memset mkdir mkfifo munmap nl_langinfo realpath rmdir setenv \
@@ -15339,10 +15582,11 @@ LVM_LIBAPI=`echo "$VER" | $AWK -F '[()]' '{print $2}'`
 
 
 
+
 
 
 ################################################################################
-ac_config_files="$ac_config_files Makefile make.tmpl daemons/Makefile daemons/clvmd/Makefile daemons/cmirrord/Makefile daemons/dmeventd/Makefile daemons/dmeventd/libdevmapper-event.pc daemons/dmeventd/plugins/Makefile daemons/dmeventd/plugins/lvm2/Makefile daemons/dmeventd/plugins/raid/Makefile daemons/dmeventd/plugins/mirror/Makefile daemons/dmeventd/plugins/snapshot/Makefile daemons/dmeventd/plugins/thin/Makefile daemons/lvmdbusd/Makefile daemons/lvmdbusd/path.py daemons/lvmetad/Makefile daemons/lvmpolld/Makefile daemons/lvmlockd/Makefile conf/Makefile conf/example.conf conf/lvmlocal.conf conf/command_profile_template.profile conf/metadata_profile_template.profile include/.symlinks include/Makefile lib/Makefile lib/format1/Makefile lib/format_pool/Makefile lib/locking/Makefile lib/mirror/Makefile lib/replicator/Makefile include/lvm-version.h lib/raid/Makefile lib/snapshot/Makefile lib/thin/Makefile lib/cache_segtype/Makefile libdaemon/Makefile libdaemon/client/Makefile libdaemon/server/Makefile libdm/Makefile libdm/libdevmapper.pc liblvm/Makefile liblvm/liblvm2app.pc man/Makefile po/Makefile python/Makefile python/setup.py scripts/blkdeactivate.sh scripts/blk_availability_init_red_hat scripts/blk_availability_systemd_red_hat.service scripts/clvmd_init_red_hat scripts/cmirrord_init_red_hat scripts/com.redhat.lvmdbus1.service scripts/dm_event_systemd_red_hat.service scripts/dm_event_systemd_red_hat.socket scripts/lvm2_cluster_activation_red_hat.sh scripts/lvm2_cluster_activation_systemd_red_hat.service scripts/lvm2_clvmd_systemd_red_hat.service scripts/lvm2_cmirrord_systemd_red_hat.service scripts/lvm2_lvmdbusd_systemd_red_hat.service scripts/lvm2_lvmetad_init_red_hat scripts/lvm2_lvmetad_systemd_red_hat.service scripts/lvm2_lvmetad_systemd_red_hat.socket scripts/lvm2_lvmpolld_init_red_hat scripts/lvm2_lvmpolld_systemd_red_hat.service scripts/lvm2_lvmpolld_systemd_red_hat.socket scripts/lvm2_lvmlockd_systemd_red_hat.service scripts/lvm2_lvmlocking_systemd_red_hat.service scripts/lvm2_monitoring_init_red_hat scripts/lvm2_monitoring_systemd_red_hat.service scripts/lvm2_pvscan_systemd_red_hat@.service scripts/lvm2_tmpfiles_red_hat.conf scripts/Makefile test/Makefile test/api/Makefile test/unit/Makefile tools/Makefile udev/Makefile unit-tests/datastruct/Makefile unit-tests/regex/Makefile unit-tests/mm/Makefile"
+ac_config_files="$ac_config_files Makefile make.tmpl daemons/Makefile daemons/clvmd/Makefile daemons/cmirrord/Makefile daemons/dmeventd/Makefile daemons/dmeventd/libdevmapper-event.pc daemons/dmeventd/plugins/Makefile daemons/dmeventd/plugins/lvm2/Makefile daemons/dmeventd/plugins/raid/Makefile daemons/dmeventd/plugins/mirror/Makefile daemons/dmeventd/plugins/snapshot/Makefile daemons/dmeventd/plugins/thin/Makefile daemons/lvmdbusd/Makefile daemons/lvmdbusd/path.py daemons/lvmetad/Makefile daemons/lvmpolld/Makefile daemons/lvmlockd/Makefile conf/Makefile conf/example.conf conf/lvmlocal.conf conf/command_profile_template.profile conf/metadata_profile_template.profile include/.symlinks include/Makefile lib/Makefile lib/format1/Makefile lib/format_pool/Makefile lib/locking/Makefile lib/mirror/Makefile lib/replicator/Makefile include/lvm-version.h lib/raid/Makefile lib/snapshot/Makefile lib/thin/Makefile lib/cache_segtype/Makefile libdaemon/Makefile libdaemon/client/Makefile libdaemon/server/Makefile libdm/Makefile libdm/libdevmapper.pc liblvm/Makefile liblvm/liblvm2app.pc man/Makefile po/Makefile python/Makefile python/setup.py scripts/blkdeactivate.sh scripts/blk_availability_init_red_hat scripts/blk_availability_systemd_red_hat.service scripts/clvmd_init_red_hat scripts/cmirrord_init_red_hat scripts/com.redhat.lvmdbus1.service scripts/dm_event_systemd_red_hat.service scripts/dm_event_systemd_red_hat.socket scripts/lvm2_cluster_activation_red_hat.sh scripts/lvm2_cluster_activation_systemd_red_hat.service scripts/lvm2_clvmd_systemd_red_hat.service scripts/lvm2_cmirrord_systemd_red_hat.service scripts/lvm2_lvmdbusd_systemd_red_hat.service scripts/lvm2_lvmetad_init_red_hat scripts/lvm2_lvmetad_systemd_red_hat.service scripts/lvm2_lvmetad_systemd_red_hat.socket scripts/lvm2_lvmpolld_init_red_hat scripts/lvm2_lvmpolld_systemd_red_hat.service scripts/lvm2_lvmpolld_systemd_red_hat.socket scripts/lvm2_lvmlockd_systemd_red_hat.service scripts/lvm2_lvmlocking_systemd_red_hat.service scripts/lvm2_monitoring_init_red_hat scripts/lvm2_monitoring_systemd_red_hat.service scripts/lvm2_pvscan_systemd_red_hat@.service scripts/lvm2_tmpfiles_red_hat.conf scripts/lvmdump.sh scripts/Makefile test/Makefile test/api/Makefile test/unit/Makefile tools/Makefile udev/Makefile unit-tests/datastruct/Makefile unit-tests/regex/Makefile unit-tests/mm/Makefile"
 
 cat >confcache <<\_ACEOF
 # This file is a shell script that caches the results of configure
@@ -16109,6 +16353,7 @@ do
     "scripts/lvm2_monitoring_systemd_red_hat.service") CONFIG_FILES="$CONFIG_FILES scripts/lvm2_monitoring_systemd_red_hat.service" ;;
     "scripts/lvm2_pvscan_systemd_red_hat@.service") CONFIG_FILES="$CONFIG_FILES scripts/lvm2_pvscan_systemd_red_hat@.service" ;;
     "scripts/lvm2_tmpfiles_red_hat.conf") CONFIG_FILES="$CONFIG_FILES scripts/lvm2_tmpfiles_red_hat.conf" ;;
+    "scripts/lvmdump.sh") CONFIG_FILES="$CONFIG_FILES scripts/lvmdump.sh" ;;
     "scripts/Makefile") CONFIG_FILES="$CONFIG_FILES scripts/Makefile" ;;
     "test/Makefile") CONFIG_FILES="$CONFIG_FILES test/Makefile" ;;
     "test/api/Makefile") CONFIG_FILES="$CONFIG_FILES test/api/Makefile" ;;

configure.in

@@ -86,6 +86,8 @@ AC_PROG_RANLIB
 AC_PATH_TOOL(CFLOW_CMD, cflow)
 AC_PATH_TOOL(CSCOPE_CMD, cscope)
 AC_PATH_TOOL(CHMOD, chmod)
+AC_PATH_TOOL(WC, wc)
+AC_PATH_TOOL(SORT, sort)
 
 ################################################################################
 dnl -- Check for header files.
@@ -134,6 +136,7 @@ AC_TYPE_UINT8_T
 AC_TYPE_UINT16_T
 AC_TYPE_UINT32_T
 AC_TYPE_UINT64_T
+AX_GCC_BUILTIN([__builtin_clz])
 
 ################################################################################
 dnl -- Check for functions
@@ -2068,6 +2071,7 @@ AC_SUBST(MIRRORS)
 AC_SUBST(MSGFMT)
 AC_SUBST(OCF)
 AC_SUBST(OCFDIR)
+AC_SUBST(ODIRECT)
 AC_SUBST(PKGCONFIG)
 AC_SUBST(POOL)
 AC_SUBST(M_LIBS)
@@ -2213,6 +2217,7 @@ scripts/lvm2_monitoring_init_red_hat
 scripts/lvm2_monitoring_systemd_red_hat.service
 scripts/lvm2_pvscan_systemd_red_hat@.service
 scripts/lvm2_tmpfiles_red_hat.conf
+scripts/lvmdump.sh
 scripts/Makefile
 test/Makefile
 test/api/Makefile

daemons/Makefile.in

@@ -41,11 +41,11 @@ ifeq ("@BUILD_LVMPOLLD@", "yes")
 endif
 
 ifeq ("@BUILD_LVMLOCKD@", "yes")
-  SUBDIRS += lvmlockd 
+  SUBDIRS += lvmlockd
 endif
 
 ifeq ("@BUILD_LVMDBUSD@", "yes")
-  SUBDIRS += lvmdbusd 
+  SUBDIRS += lvmdbusd
 endif
 
 ifeq ($(MAKECMDGOALS),distclean)

daemons/clvmd/clvmd-corosync.c

@@ -532,6 +532,7 @@ static int _cluster_fd_callback(struct local_client *fd, char *buf, int len,
 static int _cluster_send_message(const void *buf, int msglen, const char *csid,
 				 const char *errtext)
 {
+	static pthread_mutex_t _mutex = PTHREAD_MUTEX_INITIALIZER;
 	struct iovec iov[2];
 	cs_error_t err;
 	int target_node;
@@ -546,7 +547,10 @@ static int _cluster_send_message(const void *buf, int msglen, const char *csid,
 	iov[1].iov_base = (char *)buf;
 	iov[1].iov_len = msglen;
 
+	pthread_mutex_lock(&_mutex);
 	err = cpg_mcast_joined(cpg_handle, CPG_TYPE_AGREED, iov, 2);
+	pthread_mutex_unlock(&_mutex);
+
 	return cs_to_errno(err);
 }
 

daemons/clvmd/clvmd.c

@@ -517,7 +517,7 @@ int main(int argc, char *argv[])
 	/* Initialise the LVM thread variables */
 	dm_list_init(&lvm_cmd_head);
 	if (pthread_attr_init(&stack_attr) ||
-	    pthread_attr_setstacksize(&stack_attr, STACK_SIZE)) {
+	    pthread_attr_setstacksize(&stack_attr, STACK_SIZE + getpagesize())) {
 		log_sys_error("pthread_attr_init", "");
 		exit(1);
 	}

daemons/cmirrord/functions.c

@@ -12,6 +12,7 @@
 #include "logging.h"
 #include "functions.h"
 
+#include <sys/sysmacros.h>
 #include <dirent.h>
 #include <errno.h>
 #include <fcntl.h>

daemons/dmeventd/dmeventd.c

@@ -99,13 +99,28 @@ static time_t _idle_since = 0;
 static char **_initial_registrations = 0;
 
 /* FIXME Make configurable at runtime */
-__attribute__((format(printf, 4, 5)))
-static void _dmeventd_log(int level, const char *file, int line,
-			  const char *format, ...)
+
+/* All libdm messages */
+__attribute__((format(printf, 5, 6)))
+static void _libdm_log(int level, const char *file, int line,
+		       int dm_errno_or_class, const char *format, ...)
 {
 	va_list ap;
 	va_start(ap, format);
-	dm_event_log("dm", level, file, line, 0, format, ap);
+	dm_event_log("#dm", level, file, line, dm_errno_or_class, format, ap);
+	va_end(ap);
+}
+
+/* All dmeventd messages */
+#undef LOG_MESG
+#define LOG_MESG(l, f, ln, e, x...) _dmeventd_log(l, f, ln, e, ## x)
+__attribute__((format(printf, 5, 6)))
+static void _dmeventd_log(int level, const char *file, int line,
+			  int dm_errno_or_class, const char *format, ...)
+{
+	va_list ap;
+	va_start(ap, format);
+	dm_event_log("dmeventd", level, file, line, dm_errno_or_class, format, ap);
 	va_end(ap);
 }
 
@@ -453,7 +468,7 @@ static int _pthread_create_smallstack(pthread_t *t, void *(*fun)(void *), void *
 	/*
 	 * We use a smaller stack since it gets preallocated in its entirety
 	 */
-	pthread_attr_setstacksize(&attr, THREAD_STACK_SIZE);
+	pthread_attr_setstacksize(&attr, THREAD_STACK_SIZE + getpagesize());
 
 	/*
 	 * If no-one will be waiting, we need to detach.
@@ -829,17 +844,6 @@ static void _print_sigset(const char *prefix, const sigset_t *sigset)
 }
 #endif
 
-static sigset_t _unblock_sigalrm(void)
-{
-	sigset_t set, old;
-
-	sigemptyset(&set);
-	sigaddset(&set, SIGALRM);
-	pthread_sigmask(SIG_UNBLOCK, &set, &old);
-
-	return old;
-}
-
 enum {
 	DM_WAIT_RETRY,
 	DM_WAIT_INTR,
@@ -849,7 +853,7 @@ enum {
 /* Wait on a device until an event occurs. */
 static int _event_wait(struct thread_status *thread)
 {
-	sigset_t set;
+	sigset_t set, old;
 	int ret = DM_WAIT_RETRY;
 	struct dm_info info;
 
@@ -859,7 +863,12 @@ static int _event_wait(struct thread_status *thread)
 	 * This is so that you can break out of waiting on an event,
 	 * either for a timeout event, or to cancel the thread.
 	 */
-	set = _unblock_sigalrm();
+	sigemptyset(&set);
+	sigaddset(&set, SIGALRM);
+	if (pthread_sigmask(SIG_UNBLOCK, &set, &old) != 0) {
+		log_sys_error("pthread_sigmask", "unblock alarm");
+		return ret; /* What better */
+	}
 
 	if (dm_task_run(thread->wait_task)) {
 		thread->current_events |= DM_EVENT_DEVICE_ERROR;
@@ -883,10 +892,11 @@ static int _event_wait(struct thread_status *thread)
 		}
 	}
 
-	pthread_sigmask(SIG_SETMASK, &set, NULL);
+	if (pthread_sigmask(SIG_SETMASK, &old, NULL) != 0)
+		log_sys_error("pthread_sigmask", "block alarm");
 
 #ifdef DEBUG_SIGNALS
-	_print_sigset("dmeventd blocking ", &set);
+	_print_sigset("dmeventd blocking ", &old);
 #endif
 	DEBUGLOG("Completed waitevent task for %s.", thread->device.name);
 
@@ -2196,7 +2206,7 @@ int main(int argc, char *argv[])
 		openlog("dmeventd", LOG_PID, LOG_DAEMON);
 
 	dm_event_log_set(_debug_level, _use_syslog);
-	dm_log_init(_dmeventd_log);
+	dm_log_with_errno_init(_libdm_log);
 
 	(void) dm_prepare_selinux_context(DMEVENTD_PIDFILE, S_IFREG);
 	if (dm_create_lockfile(DMEVENTD_PIDFILE) == 0)

daemons/dmeventd/libdevmapper-event.c

@@ -865,28 +865,38 @@ void dm_event_log(const char *subsys, int level, const char *file,
 		  int line, int dm_errno_or_class,
 		  const char *format, va_list ap)
 {
+	static int _abort_on_internal_errors = -1;
 	static pthread_mutex_t _log_mutex = PTHREAD_MUTEX_INITIALIZER;
 	static time_t start = 0;
 	const char *indent = "";
-	FILE *stream = stdout;
+	FILE *stream = log_stderr(level) ? stderr : stdout;
 	int prio;
 	time_t now;
+	int log_with_debug = 0;
 
-	switch (level & ~(_LOG_STDERR | _LOG_ONCE)) {
+	if (subsys[0] == '#') {
+		/* Subsystems starting with '#' are logged
+		 * only when debugging is enabled. */
+		log_with_debug++;
+		subsys++;
+	}
+
+	switch (log_level(level)) {
 	case _LOG_DEBUG:
+		/* Never shown without -ddd */
 		if (_debug_level < 3)
 			return;
 		prio = LOG_DEBUG;
 		indent = "      ";
 		break;
 	case _LOG_INFO:
-		if (_debug_level < 2)
+		if (log_with_debug && _debug_level < 2)
 			return;
 		prio = LOG_INFO;
 		indent = "    ";
 		break;
 	case _LOG_NOTICE:
-		if (_debug_level < 1)
+		if (log_with_debug && _debug_level < 1)
 			return;
 		prio = LOG_NOTICE;
 		indent = "  ";
@@ -912,12 +922,13 @@ void dm_event_log(const char *subsys, int level, const char *file,
 		if (!start)
 			start = now;
 		now -= start;
-		fprintf(stream, "[%2d:%02d] %8x:%-6s%s",
-			(int)now / 60, (int)now % 60,
-			// TODO: Maybe use shorter ID
-			// ((int)(pthread_self()) >> 6) & 0xffff,
-			(int)pthread_self(), subsys,
-			(_debug_level > 3) ? "" : indent);
+		if (_debug_level)
+			fprintf(stream, "[%2d:%02d] %8x:%-6s%s",
+				(int)now / 60, (int)now % 60,
+				// TODO: Maybe use shorter ID
+				// ((int)(pthread_self()) >> 6) & 0xffff,
+				(int)pthread_self(), subsys,
+				(_debug_level > 3) ? "" : indent);
 		if (_debug_level > 3)
 			fprintf(stream, "%28s:%4d %s", file, line, indent);
 		vfprintf(stream, _(format), ap);
@@ -926,6 +937,15 @@ void dm_event_log(const char *subsys, int level, const char *file,
 	}
 
 	pthread_mutex_unlock(&_log_mutex);
+
+	if (_abort_on_internal_errors < 0)
+		/* Set when env DM_ABORT_ON_INTERNAL_ERRORS is not "0" */
+		_abort_on_internal_errors =
+			strcmp(getenv("DM_ABORT_ON_INTERNAL_ERRORS") ? : "0", "0");
+
+	if (_abort_on_internal_errors &&
+	    !strncmp(format, INTERNAL_ERROR, sizeof(INTERNAL_ERROR) - 1))
+		abort();
 }
 
 #if 0				/* left out for now */

daemons/dmeventd/plugins/lvm2/dmeventd_lvm.c

@@ -32,7 +32,7 @@ static int _register_count = 0;
 static struct dm_pool *_mem_pool = NULL;
 static void *_lvm_handle = NULL;
 
-DM_EVENT_LOG_FN("lvm")
+DM_EVENT_LOG_FN("#lvm")
 
 static void _lvm2_print_log(int level, const char *file, int line,
 			    int dm_errno_or_class, const char *msg)
@@ -121,6 +121,7 @@ int dmeventd_lvm2_run(const char *cmdline)
 int dmeventd_lvm2_command(struct dm_pool *mem, char *buffer, size_t size,
 			  const char *cmd, const char *device)
 {
+	static char _internal_prefix[] =  "_dmeventd_";
 	char *vg = NULL, *lv = NULL, *layer;
 	int r;
 
@@ -135,6 +136,21 @@ int dmeventd_lvm2_command(struct dm_pool *mem, char *buffer, size_t size,
 	    (layer = strstr(lv, "_mlog")))
 		*layer = '\0';
 
+	if (!strncmp(cmd, _internal_prefix, sizeof(_internal_prefix) - 1)) {
+		dmeventd_lvm2_lock();
+		/* output of internal command passed via env var */
+		if (!dmeventd_lvm2_run(cmd))
+			cmd = NULL;
+		else if ((cmd = getenv(cmd)))
+			cmd = dm_pool_strdup(mem, cmd); /* copy with lock */
+		dmeventd_lvm2_unlock();
+
+		if (!cmd) {
+			log_error("Unable to find configured command.");
+			return 0;
+		}
+	}
+
 	r = dm_snprintf(buffer, size, "%s %s/%s", cmd, vg, lv);
 
 	dm_pool_free(mem, vg);

daemons/dmeventd/plugins/mirror/dmeventd_mirror.c

@@ -73,8 +73,10 @@ static int _get_mirror_event(struct dso_state *state, char *params)
 	unsigned i;
 	struct dm_status_mirror *ms;
 
-	if (!dm_get_status_mirror(state->mem, params, &ms))
-		goto_out;
+	if (!dm_get_status_mirror(state->mem, params, &ms)) {
+		log_error("Unable to parse mirror status string.");
+		return ME_IGNORE;
+	}
 
 	/* Check for bad mirror devices */
 	for (i = 0; i < ms->dev_count; ++i)
@@ -95,27 +97,23 @@ static int _get_mirror_event(struct dso_state *state, char *params)
 	dm_pool_free(state->mem, ms);
 
 	return r;
-
-out:
-	log_error("Unable to parse mirror status string.");
-
-	return ME_IGNORE;
 }
 
-static int _remove_failed_devices(const char *cmd_lvscan, const char *cmd_lvconvert)
+static int _remove_failed_devices(const char *cmd_lvscan, const char *cmd_lvconvert,
+				  const char *device)
 {
-	int r;
-
 	if (!dmeventd_lvm2_run_with_lock(cmd_lvscan))
-		log_info("Re-scan of mirrored device failed.");
+		log_warn("WARNING: Re-scan of mirrored device %s failed.", device);
 
 	/* if repair goes OK, report success even if lvscan has failed */
-	r = dmeventd_lvm2_run_with_lock(cmd_lvconvert);
+	if (!dmeventd_lvm2_run_with_lock(cmd_lvconvert)) {
+		log_error("Repair of mirrored device %s failed.", device);
+		return 0;
+	}
 
-	log_info("Repair of mirrored device %s.",
-		 (r) ? "finished successfully" : "failed");
+	log_info("Repair of mirrored device %s finished successfully.", device);
 
-	return r;
+	return 1;
 }
 
 void process_event(struct dm_task *dmt,
@@ -154,7 +152,8 @@ void process_event(struct dm_task *dmt,
 		case ME_FAILURE:
 			log_error("Device failure in %s.", device);
 			if (!_remove_failed_devices(state->cmd_lvscan,
-						    state->cmd_lvconvert))
+						    state->cmd_lvconvert,
+						    device))
 				/* FIXME Why are all the error return codes unused? Get rid of them? */
 				log_error("Failed to remove faulty devices in %s.",
 					  device);
@@ -168,7 +167,7 @@ void process_event(struct dm_task *dmt,
 			break;
 		default:
 			/* FIXME Provide value then! */
-			log_info("Unknown event received.");
+			log_warn("WARNING: %s received unknown event.", device);
 		}
 	} while (next);
 }
@@ -185,16 +184,12 @@ int register_device(const char *device,
 		goto_bad;
 
 	if (!dmeventd_lvm2_command(state->mem, state->cmd_lvscan, sizeof(state->cmd_lvscan),
-				   "lvscan --cache", device)) {
-		dmeventd_lvm2_exit_with_pool(state);
+				   "lvscan --cache", device))
 		goto_bad;
-	}
 
 	if (!dmeventd_lvm2_command(state->mem, state->cmd_lvconvert, sizeof(state->cmd_lvconvert),
-				   "lvconvert --repair --use-policies", device)) {
-		dmeventd_lvm2_exit_with_pool(state);
+				   "lvconvert --repair --use-policies", device))
 		goto_bad;
-	}
 
 	*user = state;
 
@@ -204,6 +199,9 @@ int register_device(const char *device,
 bad:
 	log_error("Failed to monitor mirror %s.", device);
 
+	if (state)
+		dmeventd_lvm2_exit_with_pool(state);
+
 	return 0;
 }
 

daemons/dmeventd/plugins/raid/dmeventd_raid.c

@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2005-2015 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2005-2016 Red Hat, Inc. All rights reserved.
  *
  * This file is part of LVM2.
  *
@@ -13,14 +13,20 @@
  */
 
 #include "lib.h"
+#include "defaults.h"
 #include "dmeventd_lvm.h"
 #include "libdevmapper-event.h"
 
+/* Hold enough elements for the mximum number of RAID images */
+#define	RAID_DEVS_ELEMS	((DEFAULT_RAID_MAX_IMAGES + 63) / 64)
+
 struct dso_state {
 	struct dm_pool *mem;
 	char cmd_lvscan[512];
 	char cmd_lvconvert[512];
+	uint64_t raid_devs[RAID_DEVS_ELEMS];
 	int failed;
+	int warned;
 };
 
 DM_EVENT_LOG_FN("raid")
@@ -31,29 +37,50 @@ static int _process_raid_event(struct dso_state *state, char *params, const char
 {
 	struct dm_status_raid *status;
 	const char *d;
+	int dead = 0, r = 1;
+	uint32_t dev;
 
 	if (!dm_get_status_raid(state->mem, params, &status)) {
 		log_error("Failed to process status line for %s.", device);
 		return 0;
 	}
 
-	if ((d = strchr(status->dev_health, 'D'))) {
+	d = status->dev_health;
+	while ((d = strchr(d, 'D'))) {
+		dev = (uint32_t)(d - status->dev_health);
+
+		if (!(state->raid_devs[dev / 64] & (UINT64_C(1) << (dev % 64)))) {
+			state->raid_devs[dev / 64] |= (UINT64_C(1) << (dev % 64));
+			log_warn("WARNING: Device #%u of %s array, %s, has failed.",
+				 dev, status->raid_type, device);
+		}
+
+		d++;
+		dead = 1;
+	}
+
+	if (dead) {
+		if (status->insync_regions < status->total_regions) {
+			if (!state->warned) {
+				state->warned = 1;
+				log_warn("WARNING: waiting for resynchronization to finish "
+					 "before initiating repair on RAID device %s.", device);
+			}
+
+			goto out; /* Not yet done syncing with accessible devices */
+		}
+
 		if (state->failed)
 			goto out; /* already reported */
 
-		log_error("Device #%d of %s array, %s, has failed.",
-			  (int)(d - status->dev_health),
-			  status->raid_type, device);
-
 		state->failed = 1;
 		if (!dmeventd_lvm2_run_with_lock(state->cmd_lvscan))
 			log_warn("WARNING: Re-scan of RAID device %s failed.", device);
 
 		/* if repair goes OK, report success even if lvscan has failed */
 		if (!dmeventd_lvm2_run_with_lock(state->cmd_lvconvert)) {
-			log_info("Repair of RAID device %s failed.", device);
-			dm_pool_free(state->mem, status);
-			return 0;
+			log_error("Repair of RAID device %s failed.", device);
+			r = 0;
 		}
 	} else {
 		state->failed = 0;
@@ -64,7 +91,7 @@ static int _process_raid_event(struct dso_state *state, char *params, const char
 out:
 	dm_pool_free(state->mem, status);
 
-	return 1;
+	return r;
 }
 
 void process_event(struct dm_task *dmt,
@@ -113,10 +140,8 @@ int register_device(const char *device,
 				   "lvscan --cache", device) ||
 	    !dmeventd_lvm2_command(state->mem, state->cmd_lvconvert, sizeof(state->cmd_lvconvert),
 				   "lvconvert --config devices{ignore_suspended_devices=1} "
-				   "--repair --use-policies", device)) {
-		dmeventd_lvm2_exit_with_pool(state);
+				   "--repair --use-policies", device))
 		goto_bad;
-	}
 
 	*user = state;
 
@@ -126,6 +151,9 @@ int register_device(const char *device,
 bad:
 	log_error("Failed to monitor RAID %s.", device);
 
+	if (state)
+		dmeventd_lvm2_exit_with_pool(state);
+
 	return 0;
 }
 

daemons/dmeventd/plugins/snapshot/dmeventd_snapshot.c

@@ -16,6 +16,7 @@
 #include "dmeventd_lvm.h"
 #include "libdevmapper-event.h"
 
+#include <sys/sysmacros.h>
 #include <sys/wait.h>
 #include <stdarg.h>
 #include <pthread.h>
@@ -253,10 +254,8 @@ int register_device(const char *device,
 
 	if (!dmeventd_lvm2_command(state->mem, state->cmd_lvextend,
 				   sizeof(state->cmd_lvextend),
-				   "lvextend --use-policies", device)) {
-		dmeventd_lvm2_exit_with_pool(state);
+				   "lvextend --use-policies", device))
 		goto_bad;
-	}
 
 	state->percent_check = CHECK_MINIMUM;
 	*user = state;
@@ -267,6 +266,9 @@ int register_device(const char *device,
 bad:
 	log_error("Failed to monitor snapshot %s.", device);
 
+	if (state)
+		dmeventd_lvm2_exit_with_pool(state);
+
 	return 0;
 }
 

daemons/dmeventd/plugins/thin/dmeventd_thin.c

@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2011-2016 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2011-2017 Red Hat, Inc. All rights reserved.
  *
  * This file is part of LVM2.
  *
@@ -18,7 +18,6 @@
 
 #include <sys/wait.h>
 #include <stdarg.h>
-#include <pthread.h>
 
 /* TODO - move this mountinfo code into library to be reusable */
 #ifdef __linux__
@@ -40,277 +39,122 @@
 
 #define UMOUNT_COMMAND "/bin/umount"
 
-#define MAX_FAILS	(10)
+#define MAX_FAILS	(256)  /* ~42 mins between cmd call retry with 10s delay */
 
 #define THIN_DEBUG 0
 
 struct dso_state {
 	struct dm_pool *mem;
 	int metadata_percent_check;
+	int metadata_percent;
+	int metadata_warn_once;
 	int data_percent_check;
+	int data_percent;
+	int data_warn_once;
 	uint64_t known_metadata_size;
 	uint64_t known_data_size;
 	unsigned fails;
-	char cmd_str[1024];
+	unsigned max_fails;
+	int restore_sigset;
+	sigset_t old_sigset;
+	pid_t pid;
+	char *argv[3];
+	char *cmd_str;
 };
 
 DM_EVENT_LOG_FN("thin")
 
 #define UUID_PREFIX "LVM-"
 
-/* Figure out device UUID has LVM- prefix and is OPEN */
-static int _has_unmountable_prefix(int major, int minor)
+static int _run_command(struct dso_state *state)
 {
-	struct dm_task *dmt;
-	struct dm_info info;
-	const char *uuid;
-	int r = 0;
+	char val[3][36];
+	char *env[] = { val[0], val[1], val[2], NULL };
+	int i;
 
-	if (!(dmt = dm_task_create(DM_DEVICE_INFO)))
-		return_0;
+	/* Mark for possible lvm2 command we are running from dmeventd
+	 * lvm2 will not try to talk back to dmeventd while processing it */
+	(void) dm_snprintf(val[0], sizeof(val[0]), "LVM_RUN_BY_DMEVENTD=1");
 
-	if (!dm_task_set_major_minor(dmt, major, minor, 1))
-		goto_out;
+	if (state->data_percent) {
+		/* Prepare some known data to env vars for easy use */
+		(void) dm_snprintf(val[1], sizeof(val[1]), "DMEVENTD_THIN_POOL_DATA=%d",
+				   state->data_percent / DM_PERCENT_1);
+		(void) dm_snprintf(val[2], sizeof(val[2]), "DMEVENTD_THIN_POOL_METADATA=%d",
+				   state->metadata_percent / DM_PERCENT_1);
+	} else {
+		/* For an error event it's for a user to check status and decide */
+		env[1] = NULL;
+		log_debug("Error event processing.");
+	}
 
-	if (!dm_task_no_flush(dmt))
-		stack;
+	log_verbose("Executing command: %s", state->cmd_str);
 
-	if (!dm_task_run(dmt))
-		goto out;
-
-	if (!dm_task_get_info(dmt, &info))
-		goto out;
-
-	if (!info.exists || !info.open_count)
-		goto out; /* Not open -> not mounted */
-
-	if (!(uuid = dm_task_get_uuid(dmt)))
-		goto out;
-
-	/* Check it's public mountable LV
-	 * has prefix  LVM-  and UUID size is 68 chars */
-	if (memcmp(uuid, UUID_PREFIX, sizeof(UUID_PREFIX) - 1) ||
-	    strlen(uuid) != 68)
-		goto out;
-
-#if THIN_DEBUG
-	log_debug("Found logical volume %s (%u:%u).", uuid, major, minor);
-#endif
-	r = 1;
-out:
-	dm_task_destroy(dmt);
-
-	return r;
-}
-
-/* Get dependencies for device, and try to find matching device */
-static int _has_deps(const char *name, int tp_major, int tp_minor, int *dev_minor)
-{
-	struct dm_task *dmt;
-	const struct dm_deps *deps;
-	struct dm_info info;
-	int major, minor;
-	int r = 0;
-
-	if (!(dmt = dm_task_create(DM_DEVICE_DEPS)))
+	/* TODO:
+	 *   Support parallel run of 'task' and it's waitpid maintainence
+	 *   ATM we can't handle signaling of  SIGALRM
+	 *   as signalling is not allowed while 'process_event()' is running
+	 */
+	if (!(state->pid = fork())) {
+		/* child */
+		(void) close(0);
+		for (i = 3; i < 255; ++i) (void) close(i);
+		execve(state->argv[0], state->argv, env);
+		_exit(errno);
+	} else if (state->pid == -1) {
+		log_error("Can't fork command %s.", state->cmd_str);
+		state->fails = 1;
 		return 0;
-
-	if (!dm_task_set_name(dmt, name))
-		goto out;
-
-	if (!dm_task_no_open_count(dmt))
-		goto out;
-
-	if (!dm_task_run(dmt))
-		goto out;
-
-	if (!dm_task_get_info(dmt, &info))
-		goto out;
-
-	if (!(deps = dm_task_get_deps(dmt)))
-		goto out;
-
-	if (!info.exists || deps->count != 1)
-		goto out;
-
-	major = (int) MAJOR(deps->device[0]);
-	minor = (int) MINOR(deps->device[0]);
-	if ((major != tp_major) || (minor != tp_minor))
-		goto out;
-
-	*dev_minor = info.minor;
-
-	if (!_has_unmountable_prefix(major, info.minor))
-		goto out;
-
-#if THIN_DEBUG
-	{
-		char dev_name[PATH_MAX];
-		if (dm_device_get_name(major, minor, 0, dev_name, sizeof(dev_name)))
-			log_debug("Found %s (%u:%u) depends on %s.",
-				  name, major, *dev_minor, dev_name);
-	}
-#endif
-	r = 1;
-out:
-	dm_task_destroy(dmt);
-
-	return r;
-}
-
-/* Get all active devices */
-static int _find_all_devs(dm_bitset_t bs, int tp_major, int tp_minor)
-{
-	struct dm_task *dmt;
-	struct dm_names *names;
-	unsigned next = 0;
-	int minor, r = 1;
-
-	if (!(dmt = dm_task_create(DM_DEVICE_LIST)))
-		return 0;
-
-	if (!dm_task_run(dmt)) {
-		r = 0;
-		goto out;
-	}
-
-	if (!(names = dm_task_get_names(dmt))) {
-		r = 0;
-		goto out;
-	}
-
-	if (!names->dev)
-		goto out;
-
-	do {
-		names = (struct dm_names *)((char *) names + next);
-		if (_has_deps(names->name, tp_major, tp_minor, &minor))
-			dm_bit_set(bs, minor);
-		next = names->next;
-	} while (next);
-
-out:
-	dm_task_destroy(dmt);
-
-	return r;
-}
-
-static int _run(const char *cmd, ...)
-{
-	va_list ap;
-	int argc = 1; /* for argv[0], i.e. cmd */
-	int i = 0;
-	const char **argv;
-	pid_t pid = fork();
-	int status;
-
-	if (pid == 0) { /* child */
-		va_start(ap, cmd);
-		while (va_arg(ap, const char *))
-			++argc;
-		va_end(ap);
-
-		/* + 1 for the terminating NULL */
-		argv = alloca(sizeof(const char *) * (argc + 1));
-
-		argv[0] = cmd;
-		va_start(ap, cmd);
-		while ((argv[++i] = va_arg(ap, const char *)));
-		va_end(ap);
-
-		execvp(cmd, (char **)argv);
-		log_sys_error("exec", cmd);
-		exit(127);
-	}
-
-	if (pid > 0) { /* parent */
-		if (waitpid(pid, &status, 0) != pid)
-			return 0; /* waitpid failed */
-		if (!WIFEXITED(status) || WEXITSTATUS(status))
-			return 0; /* the child failed */
-	}
-
-	if (pid < 0)
-		return 0; /* fork failed */
-
-	return 1; /* all good */
-}
-
-struct mountinfo_s {
-	const char *device;
-	struct dm_info info;
-	dm_bitset_t minors; /* Bitset for active thin pool minors */
-};
-
-static int _umount_device(char *buffer, unsigned major, unsigned minor,
-			  char *target, void *cb_data)
-{
-	struct mountinfo_s *data = cb_data;
-	char *words[10];
-
-	if ((major == data->info.major) && dm_bit(data->minors, minor)) {
-		if (dm_split_words(buffer, DM_ARRAY_SIZE(words), 0, words) < DM_ARRAY_SIZE(words))
-			words[9] = NULL; /* just don't show device name */
-		log_info("Unmounting thin %s (%d:%d) of thin-pool %s (%u:%u) from mount point \"%s\".",
-			 words[9] ? : "", major, minor, data->device,
-			 data->info.major, data->info.minor,
-			 target);
-		if (!_run(UMOUNT_COMMAND, "-fl", target, NULL))
-			log_error("Failed to lazy umount thin %s (%d:%d) from %s: %s.",
-				  words[9], major, minor, target, strerror(errno));
 	}
 
 	return 1;
 }
 
-/*
- * Find all thin pool LV users and try to umount them.
- * TODO: work with read-only thin pool support
- */
-static void _umount(struct dm_task *dmt)
-{
-	/* TODO: Convert to use hash to reduce memory usage */
-	static const size_t MINORS = (1U << 20); /* 20 bit */
-	struct mountinfo_s data = { NULL };
-
-	if (!dm_task_get_info(dmt, &data.info))
-		return;
-
-	data.device = dm_task_get_name(dmt);
-
-	if (!(data.minors = dm_bitset_create(NULL, MINORS))) {
-		log_error("Failed to allocate bitset. Not unmounting %s.", data.device);
-		goto out;
-	}
-
-	if (!_find_all_devs(data.minors, data.info.major, data.info.minor)) {
-		log_error("Failed to detect mounted volumes for %s.", data.device);
-		goto out;
-	}
-
-	if (!dm_mountinfo_read(_umount_device, &data)) {
-		log_error("Could not parse mountinfo file.");
-		goto out;
-	}
-
-out:
-	if (data.minors)
-		dm_bitset_destroy(data.minors);
-}
-
 static int _use_policy(struct dm_task *dmt, struct dso_state *state)
 {
 #if THIN_DEBUG
-	log_info("dmeventd executes: %s.", state->cmd_str);
+	log_debug("dmeventd executes: %s.", state->cmd_str);
 #endif
+	if (state->argv[0])
+		return _run_command(state);
+
 	if (!dmeventd_lvm2_run_with_lock(state->cmd_str)) {
-		log_error("Failed to extend thin pool %s.",
-			  dm_task_get_name(dmt));
-		state->fails++;
+		log_error("Failed command for %s.", dm_task_get_name(dmt));
+		state->fails = 1;
 		return 0;
 	}
 
 	state->fails = 0;
+
+	return 1;
+}
+
+/* Check if executed command has finished
+ * Only 1 command may run */
+static int _wait_for_pid(struct dso_state *state)
+{
+	int status = 0;
+
+	if (state->pid == -1)
+		return 1;
+
+	if (!waitpid(state->pid, &status, WNOHANG))
+		return 0;
+
+	/* Wait for finish */
+	if (WIFEXITED(status)) {
+		log_verbose("Child %d exited with status %d.",
+			    state->pid, WEXITSTATUS(status));
+		state->fails = WEXITSTATUS(status) ? 1 : 0;
+	} else {
+		if (WIFSIGNALED(status))
+			log_verbose("Child %d was terminated with status %d.",
+				    state->pid, WTERMSIG(status));
+		state->fails = 1;
+	}
+
+	state->pid = -1;
+
 	return 1;
 }
 
@@ -319,7 +163,6 @@ void process_event(struct dm_task *dmt,
 		   void **user)
 {
 	const char *device = dm_task_get_name(dmt);
-	int percent;
 	struct dso_state *state = *user;
 	struct dm_status_thin_pool *tps = NULL;
 	void *next = NULL;
@@ -327,25 +170,48 @@ void process_event(struct dm_task *dmt,
 	char *target_type = NULL;
 	char *params;
 	int needs_policy = 0;
-	int needs_umount = 0;
+	struct dm_task *new_dmt = NULL;
 
 #if THIN_DEBUG
 	log_debug("Watch for tp-data:%.2f%%  tp-metadata:%.2f%%.",
 		  dm_percent_to_float(state->data_percent_check),
 		  dm_percent_to_float(state->metadata_percent_check));
 #endif
-
-#if 0
-	/* No longer monitoring, waiting for remove */
-	if (!state->meta_percent_check && !state->data_percent_check)
+	if (!_wait_for_pid(state)) {
+		log_warn("WARNING: Skipping event, child %d is still running (%s).",
+			 state->pid, state->cmd_str);
 		return;
-#endif
+	}
+
 	if (event & DM_EVENT_DEVICE_ERROR) {
 		/* Error -> no need to check and do instant resize */
+		state->data_percent = state->metadata_percent = 0;
 		if (_use_policy(dmt, state))
 			goto out;
 
 		stack;
+
+		/*
+		 * Rather update oldish status
+		 * since after 'command' processing
+		 * percentage info could have changed a lot.
+		 * If we would get above UMOUNT_THRESH
+		 * we would wait for next sigalarm.
+		 */
+		if (!(new_dmt = dm_task_create(DM_DEVICE_STATUS)))
+			goto_out;
+
+		if (!dm_task_set_uuid(new_dmt, dm_task_get_uuid(dmt)))
+			goto_out;
+
+		/* Non-blocking status read */
+		if (!dm_task_no_flush(new_dmt))
+			log_warn("WARNING: Can't set no_flush for dm status.");
+
+		if (!dm_task_run(new_dmt))
+			goto_out;
+
+		dmt = new_dmt;
 	}
 
 	dm_get_next_target(dmt, next, &start, &length, &target_type, &params);
@@ -357,7 +223,6 @@ void process_event(struct dm_task *dmt,
 
 	if (!dm_get_status_thin_pool(state->mem, params, &tps)) {
 		log_error("Failed to parse status.");
-		needs_umount = 1;
 		goto out;
 	}
 
@@ -372,67 +237,112 @@ void process_event(struct dm_task *dmt,
 	if (state->known_metadata_size != tps->total_metadata_blocks) {
 		state->metadata_percent_check = CHECK_MINIMUM;
 		state->known_metadata_size = tps->total_metadata_blocks;
+		state->fails = 0;
 	}
 
 	if (state->known_data_size != tps->total_data_blocks) {
 		state->data_percent_check = CHECK_MINIMUM;
 		state->known_data_size = tps->total_data_blocks;
+		state->fails = 0;
 	}
 
-	percent = dm_make_percent(tps->used_metadata_blocks, tps->total_metadata_blocks);
-	if (percent >= state->metadata_percent_check) {
-		/*
-		 * Usage has raised more than CHECK_STEP since the last
-		 * time. Run actions.
-		 */
-		state->metadata_percent_check = (percent / CHECK_STEP) * CHECK_STEP + CHECK_STEP;
+	/*
+	 * Trigger action when threshold boundary is exceeded.
+	 * Report 80% threshold warning when it's used above 80%.
+	 * Only 100% is exception as it cannot be surpased so policy
+	 * action is called for:  >50%, >55% ... >95%, 100%
+	 */
+	state->metadata_percent = dm_make_percent(tps->used_metadata_blocks, tps->total_metadata_blocks);
+	if (state->metadata_percent <= WARNING_THRESH)
+		state->metadata_warn_once = 0; /* Dropped bellow threshold, reset warn once */
+	else if (!state->metadata_warn_once++) /* Warn once when raised above threshold */
+		log_warn("WARNING: Thin pool %s metadata is now %.2f%% full.",
+			 device, dm_percent_to_float(state->metadata_percent));
+	if (state->metadata_percent > CHECK_MINIMUM) {
+		/* Run action when usage raised more than CHECK_STEP since the last time */
+		if (state->metadata_percent > state->metadata_percent_check)
+			needs_policy = 1;
+		state->metadata_percent_check = (state->metadata_percent / CHECK_STEP + 1) * CHECK_STEP;
+		if (state->metadata_percent_check == DM_PERCENT_100)
+			state->metadata_percent_check--; /* Can't get bigger then 100% */
+	} else
+		state->metadata_percent_check = CHECK_MINIMUM;
 
-		/* FIXME: extension of metadata needs to be written! */
-		if (percent >= WARNING_THRESH) /* Print a warning to syslog. */
-			log_warn("WARNING: Thin pool %s metadata is now %.2f%% full.",
-				 device, dm_percent_to_float(percent));
-		needs_policy = 1;
+	state->data_percent = dm_make_percent(tps->used_data_blocks, tps->total_data_blocks);
+	if (state->data_percent <= WARNING_THRESH)
+		state->data_warn_once = 0;
+	else if (!state->data_warn_once++)
+		log_warn("WARNING: Thin pool %s data is now %.2f%% full.",
+			 device, dm_percent_to_float(state->data_percent));
+	if (state->data_percent > CHECK_MINIMUM) {
+		/* Run action when usage raised more than CHECK_STEP since the last time */
+		if (state->data_percent > state->data_percent_check)
+			needs_policy = 1;
+		state->data_percent_check = (state->data_percent / CHECK_STEP + 1) * CHECK_STEP;
+		if (state->data_percent_check == DM_PERCENT_100)
+			state->data_percent_check--; /* Can't get bigger then 100% */
+	} else
+		state->data_percent_check = CHECK_MINIMUM;
 
-		if (percent >= UMOUNT_THRESH)
-			needs_umount = 1;
-	}
+	/* Reduce number of _use_policy() calls by power-of-2 factor till frequency of MAX_FAILS is reached.
+	 * Avoids too high number of error retries, yet shows some status messages in log regularly.
+	 * i.e. PV could have been pvmoved and VG/LV was locked for a while...
+	 */
+	if (state->fails) {
+		if (state->fails++ <= state->max_fails) {
+			log_debug("Postponing frequently failing policy (%u <= %u).",
+				  state->fails - 1, state->max_fails);
+			return;
+		}
+		if (state->max_fails < MAX_FAILS)
+			state->max_fails <<= 1;
+		state->fails = needs_policy = 1; /* Retry failing command */
+	} else
+		state->max_fails = 1; /* Reset on success */
 
-	percent = dm_make_percent(tps->used_data_blocks, tps->total_data_blocks);
-	if (percent >= state->data_percent_check) {
-		/*
-		 * Usage has raised more than CHECK_STEP since
-		 * the last time. Run actions.
-		 */
-		state->data_percent_check = (percent / CHECK_STEP) * CHECK_STEP + CHECK_STEP;
-
-		if (percent >= WARNING_THRESH) /* Print a warning to syslog. */
-			log_warn("WARNING: Thin pool %s data is now %.2f%% full.",
-				 device, dm_percent_to_float(percent));
-		needs_policy = 1;
-
-		if (percent >= UMOUNT_THRESH)
-			needs_umount = 1;
-	}
-
-	if (needs_policy &&
-	    _use_policy(dmt, state))
-		needs_umount = 0; /* No umount when command was successful */
+	if (needs_policy)
+		_use_policy(dmt, state);
 out:
-	if (needs_umount) {
-		_umount(dmt);
-		/* Until something changes, do not retry any more actions */
-		state->data_percent_check = state->metadata_percent_check = (DM_PERCENT_1 * 101);
-	}
-
 	if (tps)
 		dm_pool_free(state->mem, tps);
 
-	if (state->fails >= MAX_FAILS) {
-		log_warn("WARNING: Dropping monitoring of %s. "
-			 "lvm2 command fails too often (%u times in raw).",
-			 device, state->fails);
-		pthread_kill(pthread_self(), SIGALRM);
-	}
+	if (new_dmt)
+		dm_task_destroy(new_dmt);
+}
+
+/* Handle SIGCHLD for a thread */
+static void _sig_child(int signum __attribute__((unused)))
+{
+	/* empty SIG_IGN */;
+}
+
+/* Setup handler for SIGCHLD when executing external command
+ * to get quick 'waitpid()' reaction
+ * It will interrupt syscall just like SIGALRM and
+ * invoke process_event().
+ */
+static void _init_thread_signals(struct dso_state *state)
+{
+	struct sigaction act = { .sa_handler = _sig_child };
+	sigset_t my_sigset;
+
+	sigemptyset(&my_sigset);
+
+	if (sigaction(SIGCHLD, &act, NULL))
+		log_warn("WARNING: Failed to set SIGCHLD action.");
+	else if (sigaddset(&my_sigset, SIGCHLD))
+		log_warn("WARNING: Failed to add SIGCHLD to set.");
+	else if (pthread_sigmask(SIG_UNBLOCK, &my_sigset, &state->old_sigset))
+		log_warn("WARNING: Failed to unblock SIGCHLD.");
+	else
+		state->restore_sigset = 1;
+}
+
+static void _restore_thread_signals(struct dso_state *state)
+{
+	if (state->restore_sigset &&
+	    pthread_sigmask(SIG_SETMASK, &state->old_sigset, NULL))
+		log_warn("WARNING: Failed to block SIGCHLD.");
 }
 
 int register_device(const char *device,
@@ -442,27 +352,55 @@ int register_device(const char *device,
 		    void **user)
 {
 	struct dso_state *state;
+	char *str;
+	char cmd_str[PATH_MAX + 128 + 2]; /* cmd ' ' vg/lv \0 */
 
 	if (!dmeventd_lvm2_init_with_pool("thin_pool_state", state))
 		goto_bad;
 
-	if (!dmeventd_lvm2_command(state->mem, state->cmd_str,
-				   sizeof(state->cmd_str),
-				   "lvextend --use-policies",
-				   device)) {
-		dmeventd_lvm2_exit_with_pool(state);
+	if (!dmeventd_lvm2_command(state->mem, cmd_str, sizeof(cmd_str),
+				   "_dmeventd_thin_command", device))
 		goto_bad;
-	}
 
-	state->metadata_percent_check = CHECK_MINIMUM;
-	state->data_percent_check = CHECK_MINIMUM;
+	if (strncmp(cmd_str, "lvm ", 4) == 0) {
+		if (!(state->cmd_str = dm_pool_strdup(state->mem, cmd_str + 4))) {
+			log_error("Failed to copy lvm command.");
+			goto bad;
+		}
+	} else if (cmd_str[0] == '/') {
+		if (!(state->cmd_str = dm_pool_strdup(state->mem, cmd_str))) {
+			log_error("Failed to copy thin command.");
+			goto bad;
+		}
+
+		/* Find last space before 'vg/lv' */
+		if (!(str = strrchr(state->cmd_str, ' ')))
+			goto inval;
+
+		if (!(state->argv[0] = dm_pool_strndup(state->mem, state->cmd_str,
+						       str - state->cmd_str))) {
+			log_error("Failed to copy command.");
+			goto bad;
+		}
+
+		state->argv[1] = str + 1;  /* 1 argument - vg/lv */
+		_init_thread_signals(state);
+	} else /* Unuspported command format */
+		goto inval;
+
+	state->pid = -1;
 	*user = state;
 
-	log_info("Monitoring thin %s.", device);
+	log_info("Monitoring thin pool %s.", device);
 
 	return 1;
+inval:
+	log_error("Invalid command for monitoring: %s.", cmd_str);
 bad:
-	log_error("Failed to monitor thin %s.", device);
+	log_error("Failed to monitor thin pool %s.", device);
+
+	if (state)
+		dmeventd_lvm2_exit_with_pool(state);
 
 	return 0;
 }
@@ -474,9 +412,31 @@ int unregister_device(const char *device,
 		      void **user)
 {
 	struct dso_state *state = *user;
+	int i;
+
+	for (i = 0; !_wait_for_pid(state) && (i < 6); ++i) {
+		if (i == 0)
+			/* Give it 2 seconds, then try to terminate & kill it */
+			log_verbose("Child %d still not finished (%s) waiting.",
+				    state->pid, state->cmd_str);
+		else if (i == 3) {
+			log_warn("WARNING: Terminating child %d.", state->pid);
+			kill(state->pid, SIGINT);
+			kill(state->pid, SIGTERM);
+		} else if (i == 5) {
+			log_warn("WARNING: Killing child %d.", state->pid);
+			kill(state->pid, SIGKILL);
+		}
+		sleep(1);
+	}
+
+	if (state->pid != -1)
+		log_warn("WARNING: Cannot kill child %d!", state->pid);
+
+	_restore_thread_signals(state);
 
 	dmeventd_lvm2_exit_with_pool(state);
-	log_info("No longer monitoring thin %s.", device);
+	log_info("No longer monitoring thin pool %s.", device);
 
 	return 1;
 }

daemons/lvmdbusd/.gitignore

@@ -0,0 +1 @@
+path.py

daemons/lvmdbusd/Makefile.in

@@ -33,7 +33,6 @@ LVMDBUS_SRCDIR_FILES = \
 	manager.py \
 	objectmanager.py \
 	pv.py \
-	refresh.py \
 	request.py \
 	state.py \
 	udevwatch.py \

daemons/lvmdbusd/automatedproperties.py

@@ -8,6 +8,7 @@
 # along with this program. If not, see <http://www.gnu.org/licenses/>.
 
 import dbus
+import dbus.service
 from . import cfg
 from .utils import get_properties, add_properties, get_object_property_diff, \
 	log_debug
@@ -158,10 +159,7 @@ class AutomatedProperties(dbus.service.Object):
 			cfg.om.lookup_update(self, new_id[0], new_id[1])
 
 		# Grab the properties values, then replace the state of the object
-		# and retrieve the new values
-		# TODO: We need to add locking to prevent concurrent access to the
-		# properties so that a client is not accessing while we are
-		# replacing.
+		# and retrieve the new values.
 		o_prop = get_properties(self)
 		self.state = new_state
 		n_prop = get_properties(self)

daemons/lvmdbusd/background.py

@@ -7,17 +7,13 @@
 # You should have received a copy of the GNU General Public License
 # along with this program. If not, see <http://www.gnu.org/licenses/>.
 
-import threading
 import subprocess
 from . import cfg
-import time
 from .cmdhandler import options_to_cli_args
 import dbus
 from .utils import pv_range_append, pv_dest_ranges, log_error, log_debug
-import traceback
-
-_rlock = threading.RLock()
-_thread_list = list()
+import os
+import threading
 
 
 def pv_move_lv_cmd(move_options, lv_full_name,
@@ -41,15 +37,40 @@ def lv_merge_cmd(merge_options, lv_full_name):
 	return cmd
 
 
-def _move_merge(interface_name, cmd, job_state):
-	add(cmd, job_state)
+def _move_merge(interface_name, command, job_state):
+	# We need to execute these command stand alone by forking & exec'ing
+	# the command always as we will be getting periodic output from them on
+	# the status of the long running operation.
+	command.insert(0, cfg.LVM_CMD)
+	process = subprocess.Popen(command, stdout=subprocess.PIPE,
+								env=os.environ,
+								stderr=subprocess.PIPE, close_fds=True)
 
-	done = job_state.Wait(-1)
-	if not done:
-		ec, err_msg = job_state.GetError
+	log_debug("Background process for %s is %d" %
+				(str(command), process.pid))
+
+	lines_iterator = iter(process.stdout.readline, b"")
+	for line in lines_iterator:
+		line_str = line.decode("utf-8")
+
+		# Check to see if the line has the correct number of separators
+		try:
+			if line_str.count(':') == 2:
+				(device, ignore, percentage) = line_str.split(':')
+				job_state.Percent = round(
+					float(percentage.strip()[:-1]), 1)
+		except ValueError:
+			log_error("Trying to parse percentage which failed for %s" %
+				line_str)
+
+	out = process.communicate()
+
+	if process.returncode == 0:
+		job_state.Percent = 100
+	else:
 		raise dbus.exceptions.DBusException(
 			interface_name,
-			'Exit code %s, stderr = %s' % (str(ec), err_msg))
+			'Exit code %s, stderr = %s' % (str(process.returncode), out[1]))
 
 	cfg.load()
 	return '/'
@@ -84,8 +105,6 @@ def move(interface_name, lv_name, pv_src_obj, pv_source_range,
 
 				pv_dests.append((pv_dbus_obj.lvm_id, pr[1], pr[2]))
 
-		# Generate the command line for this command, but don't
-		# execute it.
 		cmd = pv_move_lv_cmd(move_options,
 								lv_name,
 								pv_src.lvm_id,
@@ -110,92 +129,14 @@ def merge(interface_name, lv_uuid, lv_name, merge_options, job_state):
 			'LV with uuid %s and name %s not present!' % (lv_uuid, lv_name))
 
 
-def background_reaper():
-	while cfg.run.value != 0:
-		with _rlock:
-			num_threads = len(_thread_list) - 1
-			if num_threads >= 0:
-				for i in range(num_threads, -1, -1):
-					_thread_list[i].join(0)
-					if not _thread_list[i].is_alive():
-						log_debug("Removing thread: %s" % _thread_list[i].name)
-						_thread_list.pop(i)
-
-		time.sleep(3)
+def _run_cmd(req):
+	log_debug(
+		"_run_cmd: Running method: %s with args %s" %
+		(str(req.method), str(req.arguments)))
+	req.run_cmd()
+	log_debug("_run_cmd: complete!")
 
 
-def background_execute(command, background_job):
-
-	# Wrap this whole operation in an exception handler, otherwise if we
-	# hit a code bug we will silently exit this thread without anyone being
-	# the wiser.
-	try:
-		process = subprocess.Popen(command, stdout=subprocess.PIPE,
-									stderr=subprocess.PIPE, close_fds=True)
-		lines_iterator = iter(process.stdout.readline, b"")
-		for line in lines_iterator:
-			line_str = line.decode("utf-8")
-
-			# Check to see if the line has the correct number of separators
-			try:
-				if line_str.count(':') == 2:
-					(device, ignore, percentage) = line_str.split(':')
-					background_job.Percent = \
-						round(float(percentage.strip()[:-1]), 1)
-			except ValueError:
-				log_error("Trying to parse percentage which failed for %s" %
-					line_str)
-
-		out = process.communicate()
-
-		if process.returncode == 0:
-			background_job.Percent = 100
-
-		background_job.set_result(process.returncode, out[1])
-
-	except Exception:
-		# In the unlikely event that we blow up, we need to unblock caller which
-		# is waiting on an answer.
-		st = traceback.format_exc()
-		error = "Exception in background thread: \n%s" % st
-		log_error(error)
-		background_job.set_result(1, error)
-
-
-def add(command, reporting_job):
-	# Create the thread, get it running and then add it to the list
-	t = threading.Thread(
-		target=background_execute,
-		name="thread: " + ' '.join(command),
-		args=(command, reporting_job))
+def cmd_runner(request):
+	t = threading.Thread(target=_run_cmd, args=(request,))
 	t.start()
-
-	with _rlock:
-		_thread_list.append(t)
-
-
-def wait_thread(job, timeout, cb, cbe):
-	# We need to put the wait on it's own thread, so that we don't block the
-	# entire dbus queue processing thread
-	try:
-		cb(job.state.Wait(timeout))
-	except Exception as e:
-		cbe("Wait exception: %s" % str(e))
-	return 0
-
-
-def add_wait(job, timeout, cb, cbe):
-
-	if timeout == 0:
-		# Users are basically polling, do not create thread
-		cb(job.Complete)
-	else:
-		t = threading.Thread(
-			target=wait_thread,
-			name="thread job.Wait: %s" % job.dbus_object_path(),
-			args=(job, timeout, cb, cbe)
-		)
-
-		t.start()
-		with _rlock:
-			_thread_list.append(t)

daemons/lvmdbusd/cfg.py

@@ -11,10 +11,8 @@ import os
 import multiprocessing
 import queue
 import itertools
-try:
-	from . import path
-except SystemError:
-	import path
+
+from lvmdbusd import path
 
 LVM_CMD = os.getenv('LVM_BINARY', path.LVM_BINARY)
 
@@ -24,24 +22,28 @@ om = None
 # This is the global bus connection
 bus = None
 
+# Command line args
+args = None
+
+# Set to true if we are depending on external events for updates
+ee = False
+
 # Shared state variable across all processes
 run = multiprocessing.Value('i', 1)
 
-# Debug
-DEBUG = True
-
-# Use lvm shell
-USE_SHELL = False
+# If this is set to true, the current setup support lvm shell and we are
+# running in that mode of operation
+SHELL_IN_USE = None
 
 # Lock used by pprint
 stdout_lock = multiprocessing.Lock()
 
-kick_q = multiprocessing.Queue()
 worker_q = queue.Queue()
 
 # Main event loop
 loop = None
 
+BUS_NAME = os.getenv('LVM_DBUS_NAME', 'com.redhat.lvmdbus1')
 BASE_INTERFACE = 'com.redhat.lvmdbus1'
 PV_INTERFACE = BASE_INTERFACE + '.Pv'
 VG_INTERFACE = BASE_INTERFACE + '.Vg'
@@ -75,6 +77,7 @@ hidden_lv = itertools.count()
 
 # Used to prevent circular imports...
 load = None
+event = None
 
 # Global cached state
 db = None

daemons/lvmdbusd/cmdhandler.py

@@ -12,15 +12,12 @@ import time
 import threading
 from itertools import chain
 import collections
+import traceback
+import os
 
-try:
-	from . import cfg
-	from .utils import pv_dest_ranges, log_debug, log_error
-	from .lvm_shell_proxy import LVMShellProxy
-except SystemError:
-	import cfg
-	from utils import pv_dest_ranges, log_debug, log_error
-	from lvm_shell_proxy import LVMShellProxy
+from lvmdbusd import cfg
+from lvmdbusd.utils import pv_dest_ranges, log_debug, log_error
+from lvmdbusd.lvm_shell_proxy import LVMShellProxy
 
 try:
 	import simplejson as json
@@ -36,10 +33,6 @@ total_count = 0
 # at the same time.
 cmd_lock = threading.RLock()
 
-# The actual method which gets called to invoke the lvm command, can vary
-# from forking a new process to using lvm shell
-_t_call = None
-
 
 class LvmExecutionMeta(object):
 
@@ -62,18 +55,19 @@ class LvmExecutionMeta(object):
 
 class LvmFlightRecorder(object):
 
-	def __init__(self):
-		self.queue = collections.deque(maxlen=16)
+	def __init__(self, size=16):
+		self.queue = collections.deque(maxlen=size)
 
 	def add(self, lvm_exec_meta):
 		self.queue.append(lvm_exec_meta)
 
 	def dump(self):
 		with cmd_lock:
-			log_error("LVM dbus flight recorder START")
-			for c in self.queue:
-				log_error(str(c))
-			log_error("LVM dbus flight recorder END")
+			if len(self.queue):
+				log_error("LVM dbus flight recorder START")
+				for c in self.queue:
+					log_error(str(c))
+				log_error("LVM dbus flight recorder END")
 
 
 cfg.blackbox = LvmFlightRecorder()
@@ -100,7 +94,8 @@ def call_lvm(command, debug=False):
 	# in different locations on the same box
 	command.insert(0, cfg.LVM_CMD)
 
-	process = Popen(command, stdout=PIPE, stderr=PIPE, close_fds=True)
+	process = Popen(command, stdout=PIPE, stderr=PIPE, close_fds=True,
+					env=os.environ)
 	out = process.communicate()
 
 	stdout_text = bytes(out[0]).decode("utf-8")
@@ -109,37 +104,48 @@ def call_lvm(command, debug=False):
 	if debug or process.returncode != 0:
 		_debug_c(command, process.returncode, (stdout_text, stderr_text))
 
-	if process.returncode == 0:
-		if cfg.DEBUG and out[1] and len(out[1]) and 'help' not in command:
-			log_error('WARNING: lvm is out-putting text to STDERR on success!')
-			_debug_c(command, process.returncode, (stdout_text, stderr_text))
-
 	return process.returncode, stdout_text, stderr_text
 
+# The actual method which gets called to invoke the lvm command, can vary
+# from forking a new process to using lvm shell
+_t_call = call_lvm
+
 
 def _shell_cfg():
 	global _t_call
-	log_debug('Using lvm shell!')
-	lvm_shell = LVMShellProxy()
-	_t_call = lvm_shell.call_lvm
-
-
-if cfg.USE_SHELL:
-	_shell_cfg()
-else:
-	_t_call = call_lvm
+	# noinspection PyBroadException
+	try:
+		lvm_shell = LVMShellProxy()
+		_t_call = lvm_shell.call_lvm
+		cfg.SHELL_IN_USE = lvm_shell
+		return True
+	except Exception:
+		_t_call = call_lvm
+		cfg.SHELL_IN_USE = None
+		log_error(traceback.format_exc())
+		log_error("Unable to utilize lvm shell, dropping back to fork & exec")
+		return False
 
 
 def set_execution(shell):
 	global _t_call
 	with cmd_lock:
-		_t_call = None
-		if shell:
-			log_debug('Using lvm shell!')
-			lvm_shell = LVMShellProxy()
-			_t_call = lvm_shell.call_lvm
+		# If the user requested lvm shell and we are currently setup that
+		# way, just return
+		if cfg.SHELL_IN_USE and shell:
+			return True
 		else:
-			_t_call = call_lvm
+			if not shell and cfg.SHELL_IN_USE:
+				cfg.SHELL_IN_USE.exit_shell()
+				cfg.SHELL_IN_USE = None
+
+		_t_call = call_lvm
+		if shell:
+			if cfg.args.use_json:
+				return _shell_cfg()
+			else:
+				return False
+		return True
 
 
 def time_wrapper(command, debug=False):
@@ -219,6 +225,10 @@ def pv_remove(device, remove_options):
 	return call(cmd)
 
 
+def _qt(tag_name):
+	return '@%s' % tag_name
+
+
 def _tag(operation, what, add, rm, tag_options):
 	cmd = [operation]
 	cmd.extend(options_to_cli_args(tag_options))
@@ -229,9 +239,11 @@ def _tag(operation, what, add, rm, tag_options):
 		cmd.append(what)
 
 	if add:
-		cmd.extend(list(chain.from_iterable(('--addtag', x) for x in add)))
+		cmd.extend(list(chain.from_iterable(
+			('--addtag', _qt(x)) for x in add)))
 	if rm:
-		cmd.extend(list(chain.from_iterable(('--deltag', x) for x in rm)))
+		cmd.extend(list(chain.from_iterable(
+			('--deltag', _qt(x)) for x in rm)))
 
 	return call(cmd, False)
 
@@ -283,7 +295,7 @@ def vg_lv_snapshot(vg_name, snapshot_options, name, size_bytes):
 	return call(cmd)
 
 
-def vg_lv_create_linear(vg_name, create_options, name, size_bytes, thin_pool):
+def _vg_lv_create_common_cmd(create_options, size_bytes, thin_pool):
 	cmd = ['lvcreate']
 	cmd.extend(options_to_cli_args(create_options))
 
@@ -291,20 +303,18 @@ def vg_lv_create_linear(vg_name, create_options, name, size_bytes, thin_pool):
 		cmd.extend(['--size', str(size_bytes) + 'B'])
 	else:
 		cmd.extend(['--thin', '--size', str(size_bytes) + 'B'])
+	return cmd
+
+
+def vg_lv_create_linear(vg_name, create_options, name, size_bytes, thin_pool):
+	cmd = _vg_lv_create_common_cmd(create_options, size_bytes, thin_pool)
 	cmd.extend(['--name', name, vg_name])
 	return call(cmd)
 
 
 def vg_lv_create_striped(vg_name, create_options, name, size_bytes,
 							num_stripes, stripe_size_kb, thin_pool):
-	cmd = ['lvcreate']
-	cmd.extend(options_to_cli_args(create_options))
-
-	if not thin_pool:
-		cmd.extend(['--size', str(size_bytes) + 'B'])
-	else:
-		cmd.extend(['--thin', '--size', str(size_bytes) + 'B'])
-
+	cmd = _vg_lv_create_common_cmd(create_options, size_bytes, thin_pool)
 	cmd.extend(['--stripes', str(num_stripes)])
 
 	if stripe_size_kb != 0:
@@ -342,7 +352,8 @@ def vg_lv_create_raid(vg_name, create_options, name, raid_type, size_bytes,
 								size_bytes, num_stripes, stripe_size_kb)
 
 
-def vg_lv_create_mirror(vg_name, create_options, name, size_bytes, num_copies):
+def vg_lv_create_mirror(
+		vg_name, create_options, name, size_bytes, num_copies):
 	cmd = ['lvcreate']
 	cmd.extend(options_to_cli_args(create_options))
 
@@ -435,8 +446,11 @@ def supports_json():
 	cmd = ['help']
 	rc, out, err = call(cmd)
 	if rc == 0:
-		if 'fullreport' in err:
+		if cfg.SHELL_IN_USE:
 			return True
+		else:
+			if 'fullreport' in err:
+				return True
 	return False
 
 
@@ -445,7 +459,7 @@ def lvm_full_report_json():
 					'pv_used', 'dev_size', 'pv_mda_size', 'pv_mda_free',
 					'pv_ba_start', 'pv_ba_size', 'pe_start', 'pv_pe_count',
 					'pv_pe_alloc_count', 'pv_attr', 'pv_tags', 'vg_name',
-					'vg_uuid']
+					'vg_uuid', 'pv_missing']
 
 	pv_seg_columns = ['pvseg_start', 'pvseg_size', 'segtype',
 						'pv_uuid', 'lv_uuid', 'pv_name']
@@ -461,7 +475,9 @@ def lvm_full_report_json():
 				'vg_name', 'pool_lv_uuid', 'pool_lv', 'origin_uuid',
 				'origin', 'data_percent',
 				'lv_attr', 'lv_tags', 'vg_uuid', 'lv_active', 'data_lv',
-				'metadata_lv', 'lv_parent', 'lv_role', 'lv_layout']
+				'metadata_lv', 'lv_parent', 'lv_role', 'lv_layout',
+				'snap_percent', 'metadata_percent', 'copy_percent',
+				'sync_percent', 'lv_metadata_size', 'move_pv', 'move_pv_uuid']
 
 	lv_seg_columns = ['seg_pe_ranges', 'segtype', 'lv_uuid']
 
@@ -477,7 +493,14 @@ def lvm_full_report_json():
 
 	rc, out, err = call(cmd)
 	if rc == 0:
-		return json.loads(out)
+		# With the current implementation, if we are using the shell then we
+		# are using JSON and JSON is returned back to us as it was parsed to
+		# figure out if we completed OK or not
+		if cfg.SHELL_IN_USE:
+			assert(type(out) == dict)
+			return out
+		else:
+			return json.loads(out)
 	return None
 
 
@@ -491,7 +514,7 @@ def pv_retrieve_with_segs(device=None):
 				'pv_used', 'dev_size', 'pv_mda_size', 'pv_mda_free',
 				'pv_ba_start', 'pv_ba_size', 'pe_start', 'pv_pe_count',
 				'pv_pe_alloc_count', 'pv_attr', 'pv_tags', 'vg_name',
-				'vg_uuid', 'pvseg_start', 'pvseg_size', 'segtype']
+				'vg_uuid', 'pvseg_start', 'pvseg_size', 'segtype', 'pv_missing']
 
 	# Lvm has some issues where it returns failure when querying pvs when other
 	# operations are in process, see:
@@ -704,7 +727,9 @@ def lv_retrieve_with_segments():
 				'origin', 'data_percent',
 				'lv_attr', 'lv_tags', 'vg_uuid', 'lv_active', 'data_lv',
 				'metadata_lv', 'seg_pe_ranges', 'segtype', 'lv_parent',
-				'lv_role', 'lv_layout']
+				'lv_role', 'lv_layout',
+				'snap_percent', 'metadata_percent', 'copy_percent',
+				'sync_percent', 'lv_metadata_size', 'move_pv', 'move_pv_uuid']
 
 	cmd = _dc('lvs', ['-a', '-o', ','.join(columns)])
 	rc, out, err = call(cmd)
@@ -721,4 +746,4 @@ if __name__ == '__main__':
 	pv_data = pv_retrieve_with_segs()
 
 	for p in pv_data:
-		log_debug(str(p))
+		print(str(p))

daemons/lvmdbusd/fetch.py

@@ -11,20 +11,151 @@ from .pv import load_pvs
 from .vg import load_vgs
 from .lv import load_lvs
 from . import cfg
+from .utils import MThreadRunner, log_debug, log_error
+import threading
+import queue
+import traceback
 
 
-def load(refresh=True, emit_signal=True, cache_refresh=True, log=True):
+def _main_thread_load(refresh=True, emit_signal=True):
 	num_total_changes = 0
 
+	num_total_changes += load_pvs(
+		refresh=refresh,
+		emit_signal=emit_signal,
+		cache_refresh=False)[1]
+	num_total_changes += load_vgs(
+		refresh=refresh,
+		emit_signal=emit_signal,
+		cache_refresh=False)[1]
+	num_total_changes += load_lvs(
+		refresh=refresh,
+		emit_signal=emit_signal,
+		cache_refresh=False)[1]
+
+	return num_total_changes
+
+
+def load(refresh=True, emit_signal=True, cache_refresh=True, log=True,
+			need_main_thread=True):
 	# Go through and load all the PVs, VGs and LVs
 	if cache_refresh:
 		cfg.db.refresh(log)
 
-	num_total_changes += load_pvs(refresh=refresh, emit_signal=emit_signal,
-									cache_refresh=False)[1]
-	num_total_changes += load_vgs(refresh=refresh, emit_signal=emit_signal,
-									cache_refresh=False)[1]
-	num_total_changes += load_lvs(refresh=refresh, emit_signal=emit_signal,
-									cache_refresh=False)[1]
+	if need_main_thread:
+		rc = MThreadRunner(_main_thread_load, refresh, emit_signal).done()
+	else:
+		rc = _main_thread_load(refresh, emit_signal)
 
-	return num_total_changes
+	return rc
+
+
+# Even though lvm can handle multiple changes concurrently it really doesn't
+# make sense to make a 1-1 fetch of data for each change of lvm because when
+# we fetch the data once all previous changes are reflected.
+class StateUpdate(object):
+
+	class UpdateRequest(object):
+
+		def __init__(self, refresh, emit_signal, cache_refresh, log,
+						need_main_thread):
+			self.is_done = False
+			self.refresh = refresh
+			self.emit_signal = emit_signal
+			self.cache_refresh = cache_refresh
+			self.log = log
+			self.need_main_thread = need_main_thread
+			self.result = None
+			self.cond = threading.Condition(threading.Lock())
+
+		def done(self):
+			with self.cond:
+				if not self.is_done:
+					self.cond.wait()
+			return self.result
+
+		def set_result(self, result):
+			with self.cond:
+				self.result = result
+				self.is_done = True
+				self.cond.notify_all()
+
+	@staticmethod
+	def update_thread(obj):
+		while cfg.run.value != 0:
+			# noinspection PyBroadException
+			try:
+				queued_requests = []
+				refresh = True
+				emit_signal = True
+				cache_refresh = True
+				log = True
+				need_main_thread = True
+
+				with obj.lock:
+					wait = not obj.deferred
+					obj.deferred = False
+
+				if wait:
+					queued_requests.append(obj.queue.get(True, 2))
+
+				# Ok we have one or the deferred queue has some,
+				# check if any others
+				try:
+					while True:
+						queued_requests.append(obj.queue.get(False))
+
+				except queue.Empty:
+					pass
+
+				if len(queued_requests) > 1:
+					log_debug("Processing %d updates!" % len(queued_requests),
+							'bg_black', 'fg_light_green')
+
+				# We have what we can, run the update with the needed options
+				for i in queued_requests:
+					if not i.refresh:
+						refresh = False
+					if not i.emit_signal:
+						emit_signal = False
+					if not i.cache_refresh:
+						cache_refresh = False
+					if not i.log:
+						log = False
+					if not i.need_main_thread:
+						need_main_thread = False
+
+				num_changes = load(refresh, emit_signal, cache_refresh, log,
+									need_main_thread)
+				# Update is done, let everyone know!
+				for i in queued_requests:
+					i.set_result(num_changes)
+
+			except queue.Empty:
+				pass
+			except Exception:
+				st = traceback.format_exc()
+				log_error("update_thread exception: \n%s" % st)
+
+	def __init__(self):
+		self.lock = threading.RLock()
+		self.queue = queue.Queue()
+		self.deferred = False
+
+		# Do initial load
+		load(refresh=False, emit_signal=False, need_main_thread=False)
+
+		self.thread = threading.Thread(target=StateUpdate.update_thread,
+										args=(self,))
+
+	def load(self, refresh=True, emit_signal=True, cache_refresh=True,
+					log=True, need_main_thread=True):
+		# Place this request on the queue and wait for it to be completed
+		req = StateUpdate.UpdateRequest(refresh, emit_signal, cache_refresh,
+										log, need_main_thread)
+		self.queue.put(req)
+		return req.done()
+
+	def event(self):
+		with self.lock:
+			self.deferred = True

daemons/lvmdbusd/job.py

@@ -8,12 +8,55 @@
 # along with this program. If not, see <http://www.gnu.org/licenses/>.
 
 from .automatedproperties import AutomatedProperties
-from .utils import job_obj_path_generate
+from .utils import job_obj_path_generate, mt_async_result, mt_run_no_wait
 from . import cfg
 from .cfg import JOB_INTERFACE
 import dbus
 import threading
-from . import background
+# noinspection PyUnresolvedReferences
+from gi.repository import GLib
+
+
+# Class that handles a client waiting for something to be complete.  We either
+# get a timeout or the operation is done.
+class WaitingClient(object):
+
+	# A timeout occurred
+	@staticmethod
+	def _timeout(wc):
+		with wc.rlock:
+			if wc.in_use:
+				wc.in_use = False
+				# Remove ourselves from waiting client
+				wc.job_state.remove_waiting_client(wc)
+				wc.timer_id = -1
+				mt_async_result(wc.cb, wc.job_state.Complete)
+				wc.job_state = None
+
+	def __init__(self, job_state, tmo, cb, cbe):
+		self.rlock = threading.RLock()
+		self.job_state = job_state
+		self.cb = cb
+		self.cbe = cbe
+		self.in_use = True		# Indicates if object is in play
+		self.timer_id = -1
+		if tmo > 0:
+			self.timer_id = GLib.timeout_add_seconds(
+				tmo, WaitingClient._timeout, self)
+
+	# The job finished before the timer popped and we are being notified that
+	# it's done
+	def notify(self):
+		with self.rlock:
+			if self.in_use:
+				self.in_use = False
+				# Clear timer
+				if self.timer_id != -1:
+					GLib.source_remove(self.timer_id)
+					self.timer_id = -1
+
+				mt_async_result(self.cb, self.job_state.Complete)
+				self.job_state = None
 
 
 # noinspection PyPep8Naming
@@ -24,9 +67,9 @@ class JobState(object):
 		self._percent = 0
 		self._complete = False
 		self._request = request
-		self._cond = threading.Condition(self.rlock)
 		self._ec = 0
 		self._stderr = ''
+		self._waiting_clients = []
 
 		# This is an lvm command that is just taking too long and doesn't
 		# support background operation
@@ -49,8 +92,6 @@ class JobState(object):
 		with self.rlock:
 			if self._request:
 				self._complete = self._request.is_done()
-				if self._complete:
-					self._percent = 100
 
 			return self._complete
 
@@ -58,7 +99,8 @@ class JobState(object):
 	def Complete(self, value):
 		with self.rlock:
 			self._complete = value
-			self._cond.notify_all()
+			self._percent = 100
+			self.notify_waiting_clients()
 
 	@property
 	def GetError(self):
@@ -72,29 +114,10 @@ class JobState(object):
 			else:
 				return (-1, 'Job is not complete!')
 
-	def set_result(self, ec, msg):
-		with self.rlock:
-			self.Complete = True
-			self._ec = ec
-			self._stderr = msg
-
 	def dtor(self):
 		with self.rlock:
 			self._request = None
 
-	def Wait(self, timeout):
-		try:
-			with self._cond:
-				# Check to see if we are done, before we wait
-				if not self.Complete:
-					if timeout != -1:
-						self._cond.wait(timeout)
-					else:
-						self._cond.wait()
-				return self.Complete
-		except RuntimeError:
-			return False
-
 	@property
 	def Result(self):
 		with self.rlock:
@@ -102,10 +125,40 @@ class JobState(object):
 				return self._request.result()
 			return '/'
 
+	def add_waiting_client(self, client):
+		with self.rlock:
+			# Avoid race condition where it goes complete before we get added
+			# to the list of waiting clients
+			if self.Complete:
+				client.notify()
+			else:
+				self._waiting_clients.append(client)
+
+	def remove_waiting_client(self, client):
+		# If a waiting client timer pops before the job is done we will allow
+		# the client to remove themselves from the list.  As we have a lock
+		# here and a lock in the waiting client too, and they can be obtained
+		# in different orders, a dead lock can occur.
+		# As this remove is really optional, we will try to acquire the lock
+		# and remove.  If we are unsuccessful it's not fatal, we just delay
+		# the time when the objects can be garbage collected by python
+		if self.rlock.acquire(False):
+			try:
+				self._waiting_clients.remove(client)
+			finally:
+				self.rlock.release()
+
+	def notify_waiting_clients(self):
+		with self.rlock:
+			for c in self._waiting_clients:
+				c.notify()
+
+			self._waiting_clients = []
+
 
 # noinspection PyPep8Naming
 class Job(AutomatedProperties):
-	_Percent_meta = ('y', JOB_INTERFACE)
+	_Percent_meta = ('d', JOB_INTERFACE)
 	_Complete_meta = ('b', JOB_INTERFACE)
 	_Result_meta = ('o', JOB_INTERFACE)
 	_GetError_meta = ('(is)', JOB_INTERFACE)
@@ -121,26 +174,25 @@ class Job(AutomatedProperties):
 
 	@property
 	def Percent(self):
-		return self.state.Percent
-
-	@Percent.setter
-	def Percent(self, value):
-		self.state.Percent = value
+		return dbus.Double(float(self.state.Percent))
 
 	@property
 	def Complete(self):
-		return self.state.Complete
+		return dbus.Boolean(self.state.Complete)
+
+	@staticmethod
+	def _signal_complete(obj):
+		obj.PropertiesChanged(
+			JOB_INTERFACE, dict(Complete=dbus.Boolean(obj.state.Complete)), [])
 
 	@Complete.setter
 	def Complete(self, value):
 		self.state.Complete = value
+		mt_run_no_wait(Job._signal_complete, self)
 
 	@property
 	def GetError(self):
-		return self.state.GetError
-
-	def set_result(self, ec, msg):
-		self.state.set_result(ec, msg)
+		return dbus.Struct(self.state.GetError, signature="(is)")
 
 	@dbus.service.method(dbus_interface=JOB_INTERFACE)
 	def Remove(self):
@@ -156,11 +208,15 @@ class Job(AutomatedProperties):
 							out_signature='b',
 							async_callbacks=('cb', 'cbe'))
 	def Wait(self, timeout, cb, cbe):
-		background.add_wait(self, timeout, cb, cbe)
+		if timeout == 0 or self.state.Complete:
+			cb(dbus.Boolean(self.state.Complete))
+		else:
+			self.state.add_waiting_client(
+				WaitingClient(self.state, timeout, cb, cbe))
 
 	@property
 	def Result(self):
-		return self.state.Result
+		return dbus.ObjectPath(self.state.Result)
 
 	@property
 	def lvm_id(self):

daemons/lvmdbusd/lv.py

@@ -21,7 +21,7 @@ from .utils import n, n32
 from .loader import common
 from .state import State
 from . import background
-from .utils import round_size
+from .utils import round_size, mt_remove_dbus_objects
 from .job import JobState
 
 
@@ -81,7 +81,14 @@ def lvs_state_retrieve(selection, cache_refresh=True):
 			n32(l['data_percent']), l['lv_attr'],
 			l['lv_tags'], l['lv_active'], l['data_lv'],
 			l['metadata_lv'], l['segtype'], l['lv_role'],
-			l['lv_layout']))
+			l['lv_layout'],
+			n32(l['snap_percent']),
+			n32(l['metadata_percent']),
+			n32(l['copy_percent']),
+			n32(l['sync_percent']),
+			n(l['lv_metadata_size']),
+			l['move_pv'],
+			l['move_pv_uuid']))
 	return rc
 
 
@@ -101,9 +108,15 @@ class LvState(State):
 		rc = []
 		for pv in sorted(cfg.db.lv_contained_pv(uuid)):
 			(pv_uuid, pv_name, pv_segs) = pv
-			pv_obj = cfg.om.get_object_path_by_uuid_lvm_id(
-				pv_uuid, pv_name, gen_new=False)
-			rc.append((pv_obj, pv_segs))
+			pv_obj = cfg.om.get_object_path_by_uuid_lvm_id(pv_uuid, pv_name)
+
+			segs_decorate = []
+			for i in pv_segs:
+				segs_decorate.append((dbus.UInt64(i[0]),
+									dbus.UInt64(i[1]),
+									dbus.String(i[2])))
+
+			rc.append((dbus.ObjectPath(pv_obj), segs_decorate))
 
 		return dbus.Array(rc, signature="(oa(tts))")
 
@@ -124,25 +137,26 @@ class LvState(State):
 
 		for l in cfg.db.hidden_lvs(self.Uuid):
 			full_name = "%s/%s" % (vg_name, l[1])
-			op = cfg.om.get_object_path_by_uuid_lvm_id(
-				l[0], full_name, gen_new=False)
+			op = cfg.om.get_object_path_by_uuid_lvm_id(l[0], full_name)
 			assert op
-			rc.append(op)
+			rc.append(dbus.ObjectPath(op))
 		return rc
 
 	def __init__(self, Uuid, Name, Path, SizeBytes,
 			vg_name, vg_uuid, pool_lv_uuid, PoolLv,
 			origin_uuid, OriginLv, DataPercent, Attr, Tags, active,
-			data_lv, metadata_lv, segtypes, role, layout):
+			data_lv, metadata_lv, segtypes, role, layout, SnapPercent,
+			MetaDataPercent, CopyPercent, SyncPercent, MetaDataSizeBytes,
+			move_pv, move_pv_uuid):
 		utils.init_class_from_arguments(self)
 
 		# The segtypes is possibly an array with potentially dupes or a single
 		# value
 		self._segs = dbus.Array([], signature='s')
 		if not isinstance(segtypes, list):
-			self._segs.append(segtypes)
+			self._segs.append(dbus.String(segtypes))
 		else:
-			self._segs.extend(set(segtypes))
+			self._segs.extend([dbus.String(x) for x in set(segtypes)])
 
 		self.Vg = cfg.om.get_object_path_by_uuid_lvm_id(
 			vg_uuid, vg_name, vg_obj_path_generate)
@@ -153,8 +167,7 @@ class LvState(State):
 			gen = utils.lv_object_path_method(Name, (Attr, layout, role))
 
 			self.PoolLv = cfg.om.get_object_path_by_uuid_lvm_id(
-				pool_lv_uuid, '%s/%s' % (vg_name, PoolLv),
-				gen)
+				pool_lv_uuid, '%s/%s' % (vg_name, PoolLv), gen)
 		else:
 			self.PoolLv = '/'
 
@@ -210,13 +223,20 @@ class LvState(State):
 @utils.dbus_property(LV_COMMON_INTERFACE, 'Name', 's')
 @utils.dbus_property(LV_COMMON_INTERFACE, 'Path', 's')
 @utils.dbus_property(LV_COMMON_INTERFACE, 'SizeBytes', 't')
-@utils.dbus_property(LV_COMMON_INTERFACE, 'DataPercent', 'u')
 @utils.dbus_property(LV_COMMON_INTERFACE, 'SegType', 'as')
 @utils.dbus_property(LV_COMMON_INTERFACE, 'Vg', 'o')
 @utils.dbus_property(LV_COMMON_INTERFACE, 'OriginLv', 'o')
 @utils.dbus_property(LV_COMMON_INTERFACE, 'PoolLv', 'o')
 @utils.dbus_property(LV_COMMON_INTERFACE, 'Devices', "a(oa(tts))")
 @utils.dbus_property(LV_COMMON_INTERFACE, 'HiddenLvs', "ao")
+@utils.dbus_property(LV_COMMON_INTERFACE, 'Attr', 's')
+@utils.dbus_property(LV_COMMON_INTERFACE, 'DataPercent', 'u')
+@utils.dbus_property(LV_COMMON_INTERFACE, 'SnapPercent', 'u')
+@utils.dbus_property(LV_COMMON_INTERFACE, 'DataPercent', 'u')
+@utils.dbus_property(LV_COMMON_INTERFACE, 'MetaDataPercent', 'u')
+@utils.dbus_property(LV_COMMON_INTERFACE, 'CopyPercent', 'u')
+@utils.dbus_property(LV_COMMON_INTERFACE, 'SyncPercent', 'u')
+@utils.dbus_property(LV_COMMON_INTERFACE, 'MetaDataSizeBytes', 't')
 class LvCommon(AutomatedProperties):
 	_Tags_meta = ("as", LV_COMMON_INTERFACE)
 	_Roles_meta = ("as", LV_COMMON_INTERFACE)
@@ -232,12 +252,45 @@ class LvCommon(AutomatedProperties):
 	_FixedMinor_meta = ('b', LV_COMMON_INTERFACE)
 	_ZeroBlocks_meta = ('b', LV_COMMON_INTERFACE)
 	_SkipActivation_meta = ('b', LV_COMMON_INTERFACE)
+	_MovePv_meta = ('o', LV_COMMON_INTERFACE)
+
+	def _get_move_pv(self):
+		path = None
+
+		# It's likely that the move_pv is empty
+		if self.state.move_pv_uuid and self.state.move_pv:
+			path = cfg.om.get_object_path_by_uuid_lvm_id(
+				self.state.move_pv_uuid, self.state.move_pv)
+		if not path:
+			path = '/'
+		return path
 
 	# noinspection PyUnusedLocal,PyPep8Naming
 	def __init__(self, object_path, object_state):
 		super(LvCommon, self).__init__(object_path, lvs_state_retrieve)
 		self.set_interface(LV_COMMON_INTERFACE)
 		self.state = object_state
+		self._move_pv = self._get_move_pv()
+
+	@staticmethod
+	def handle_execute(rc, out, err):
+		if rc == 0:
+			cfg.load()
+		else:
+			# Need to work on error handling, need consistent
+			raise dbus.exceptions.DBusException(
+				LV_INTERFACE,
+				'Exit code %s, stderr = %s' % (str(rc), err))
+
+	@staticmethod
+	def validate_dbus_object(lv_uuid, lv_name):
+		dbo = cfg.om.get_object_by_uuid_lvm_id(lv_uuid, lv_name)
+		if not dbo:
+			raise dbus.exceptions.DBusException(
+				LV_INTERFACE,
+				'LV with uuid %s and name %s not present!' %
+				(lv_uuid, lv_name))
+		return dbo
 
 	@property
 	def VolumeType(self):
@@ -252,14 +305,16 @@ class LvCommon(AutomatedProperties):
 					'V': 'thin Volume', 't': 'thin pool', 'T': 'Thin pool data',
 					'e': 'raid or pool metadata or pool metadata spare',
 					'-': 'Unspecified'}
-		return (self.state.Attr[0], type_map[self.state.Attr[0]])
+		return dbus.Struct((self.state.Attr[0], type_map[self.state.Attr[0]]),
+						signature="as")
 
 	@property
 	def Permissions(self):
 		type_map = {'w': 'writable', 'r': 'read-only',
 					'R': 'Read-only activation of non-read-only volume',
 					'-': 'Unspecified'}
-		return (self.state.Attr[1], type_map[self.state.Attr[1]])
+		return dbus.Struct((self.state.Attr[1], type_map[self.state.Attr[1]]),
+						signature="(ss)")
 
 	@property
 	def AllocationPolicy(self):
@@ -268,11 +323,12 @@ class LvCommon(AutomatedProperties):
 					'i': 'inherited', 'I': 'inherited locked',
 					'l': 'cling', 'L': 'cling locked',
 					'n': 'normal', 'N': 'normal locked', '-': 'Unspecified'}
-		return (self.state.Attr[2], type_map[self.state.Attr[2]])
+		return dbus.Struct((self.state.Attr[2], type_map[self.state.Attr[2]]),
+						signature="(ss)")
 
 	@property
 	def FixedMinor(self):
-		return self.state.Attr[3] == 'm'
+		return dbus.Boolean(self.state.Attr[3] == 'm')
 
 	@property
 	def State(self):
@@ -283,29 +339,32 @@ class LvCommon(AutomatedProperties):
 					'd': 'mapped device present without  tables',
 					'i': 'mapped device present with inactive table',
 					'X': 'unknown', '-': 'Unspecified'}
-		return (self.state.Attr[4], type_map[self.state.Attr[4]])
+		return dbus.Struct((self.state.Attr[4], type_map[self.state.Attr[4]]),
+						signature="(ss)")
 
 	@property
 	def TargetType(self):
 		type_map = {'C': 'Cache', 'm': 'mirror', 'r': 'raid',
 					's': 'snapshot', 't': 'thin', 'u': 'unknown',
 					'v': 'virtual', '-': 'Unspecified'}
-		return (self.state.Attr[6], type_map[self.state.Attr[6]])
+		return dbus.Struct((self.state.Attr[6], type_map[self.state.Attr[6]]),
+						signature="(ss)")
 
 	@property
 	def ZeroBlocks(self):
-		return self.state.Attr[7] == 'z'
+		return dbus.Boolean(self.state.Attr[7] == 'z')
 
 	@property
 	def Health(self):
 		type_map = {'p': 'partial', 'r': 'refresh',
 					'm': 'mismatches', 'w': 'writemostly',
 					'X': 'X unknown', '-': 'Unspecified'}
-		return (self.state.Attr[8], type_map[self.state.Attr[8]])
+		return dbus.Struct((self.state.Attr[8], type_map[self.state.Attr[8]]),
+					signature="(ss)")
 
 	@property
 	def SkipActivation(self):
-		return self.state.Attr[9] == 'k'
+		return dbus.Boolean(self.state.Attr[9] == 'k')
 
 	def vg_name_lookup(self):
 		return self.state.vg_name_lookup()
@@ -331,22 +390,19 @@ class LvCommon(AutomatedProperties):
 
 	@property
 	def IsThinVolume(self):
-		return self.state.Attr[0] == 'V'
+		return dbus.Boolean(self.state.Attr[0] == 'V')
 
 	@property
 	def IsThinPool(self):
-		return self.state.Attr[0] == 't'
+		return dbus.Boolean(self.state.Attr[0] == 't')
 
 	@property
 	def Active(self):
-		return self.state.active == "active"
+		return dbus.Boolean(self.state.active == "active")
 
-	@dbus.service.method(
-		dbus_interface=LV_COMMON_INTERFACE,
-		in_signature='ia{sv}',
-		out_signature='o')
-	def _Future(self, tmo, open_options):
-		raise dbus.exceptions.DBusException(LV_COMMON_INTERFACE, 'Do not use!')
+	@property
+	def MovePv(self):
+		return dbus.ObjectPath(self._move_pv)
 
 
 # noinspection PyPep8Naming
@@ -372,25 +428,10 @@ class Lv(LvCommon):
 	@staticmethod
 	def _remove(lv_uuid, lv_name, remove_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(lv_uuid, lv_name)
-
-		if dbo:
-			# Remove the LV, if successful then remove from the model
-			rc, out, err = cmdhandler.lv_remove(lv_name, remove_options)
-
-			if rc == 0:
-				cfg.om.remove_object(dbo, True)
-				cfg.load()
-			else:
-				# Need to work on error handling, need consistent
-				raise dbus.exceptions.DBusException(
-					LV_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				LV_INTERFACE,
-				'LV with uuid %s and name %s not present!' %
-				(lv_uuid, lv_name))
+		LvCommon.validate_dbus_object(lv_uuid, lv_name)
+		# Remove the LV, if successful then remove from the model
+		rc, out, err = cmdhandler.lv_remove(lv_name, remove_options)
+		LvCommon.handle_execute(rc, out, err)
 		return '/'
 
 	@dbus.service.method(
@@ -408,24 +449,11 @@ class Lv(LvCommon):
 	@staticmethod
 	def _rename(lv_uuid, lv_name, new_name, rename_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(lv_uuid, lv_name)
-
-		if dbo:
-			# Rename the logical volume
-			rc, out, err = cmdhandler.lv_rename(lv_name, new_name,
-												rename_options)
-			if rc == 0:
-				cfg.load()
-			else:
-				# Need to work on error handling, need consistent
-				raise dbus.exceptions.DBusException(
-					LV_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				LV_INTERFACE,
-				'LV with uuid %s and name %s not present!' %
-				(lv_uuid, lv_name))
+		LvCommon.validate_dbus_object(lv_uuid, lv_name)
+		# Rename the logical volume
+		rc, out, err = cmdhandler.lv_rename(lv_name, new_name,
+											rename_options)
+		LvCommon.handle_execute(rc, out, err)
 		return '/'
 
 	@dbus.service.method(
@@ -459,44 +487,27 @@ class Lv(LvCommon):
 				pv_dests_and_ranges, move_options, job_state), cb, cbe, False,
 				job_state)
 
-		cfg.worker_q.put(r)
+		background.cmd_runner(r)
 
 	@staticmethod
 	def _snap_shot(lv_uuid, lv_name, name, optional_size,
 			snapshot_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(lv_uuid, lv_name)
+		dbo = LvCommon.validate_dbus_object(lv_uuid, lv_name)
+		# If you specify a size you get a 'thick' snapshot even if
+		# it is a thin lv
+		if not dbo.IsThinVolume:
+			if optional_size == 0:
+				space = dbo.SizeBytes / 80
+				remainder = space % 512
+				optional_size = space + 512 - remainder
 
-		if dbo:
-			# If you specify a size you get a 'thick' snapshot even if
-			# it is a thin lv
-			if not dbo.IsThinVolume:
-				if optional_size == 0:
-					space = dbo.SizeBytes / 80
-					remainder = space % 512
-					optional_size = space + 512 - remainder
+		rc, out, err = cmdhandler.vg_lv_snapshot(
+			lv_name, snapshot_options, name, optional_size)
+		LvCommon.handle_execute(rc, out, err)
+		full_name = "%s/%s" % (dbo.vg_name_lookup(), name)
+		return cfg.om.get_object_path_by_lvm_id(full_name)
 
-			rc, out, err = cmdhandler.vg_lv_snapshot(
-				lv_name, snapshot_options, name, optional_size)
-			if rc == 0:
-				return_path = '/'
-				full_name = "%s/%s" % (dbo.vg_name_lookup(), name)
-				lvs = load_lvs([full_name], emit_signal=True)[0]
-				for l in lvs:
-					return_path = l.dbus_object_path()
-
-				# Refresh self and all included PVs
-				cfg.load(cache_refresh=False)
-				return return_path
-			else:
-				raise dbus.exceptions.DBusException(
-					LV_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				LV_INTERFACE,
-				'LV with uuid %s and name %s not present!' %
-				(lv_uuid, lv_name))
 
 	@dbus.service.method(
 		dbus_interface=LV_INTERFACE,
@@ -519,38 +530,24 @@ class Lv(LvCommon):
 				resize_options):
 		# Make sure we have a dbus object representing it
 		pv_dests = []
-		dbo = cfg.om.get_object_by_uuid_lvm_id(lv_uuid, lv_name)
+		dbo = LvCommon.validate_dbus_object(lv_uuid, lv_name)
 
-		if dbo:
-			# If we have PVs, verify them
-			if len(pv_dests_and_ranges):
-				for pr in pv_dests_and_ranges:
-					pv_dbus_obj = cfg.om.get_object_by_path(pr[0])
-					if not pv_dbus_obj:
-						raise dbus.exceptions.DBusException(
-							LV_INTERFACE,
-							'PV Destination (%s) not found' % pr[0])
+		# If we have PVs, verify them
+		if len(pv_dests_and_ranges):
+			for pr in pv_dests_and_ranges:
+				pv_dbus_obj = cfg.om.get_object_by_path(pr[0])
+				if not pv_dbus_obj:
+					raise dbus.exceptions.DBusException(
+						LV_INTERFACE,
+						'PV Destination (%s) not found' % pr[0])
 
-					pv_dests.append((pv_dbus_obj.lvm_id, pr[1], pr[2]))
+				pv_dests.append((pv_dbus_obj.lvm_id, pr[1], pr[2]))
 
-			size_change = new_size_bytes - dbo.SizeBytes
-
-			rc, out, err = cmdhandler.lv_resize(dbo.lvm_id, size_change,
-												pv_dests, resize_options)
-
-			if rc == 0:
-				# Refresh what's changed
-				cfg.load()
-				return "/"
-			else:
-				raise dbus.exceptions.DBusException(
-					LV_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				LV_INTERFACE,
-				'LV with uuid %s and name %s not present!' %
-				(lv_uuid, lv_name))
+		size_change = new_size_bytes - dbo.SizeBytes
+		rc, out, err = cmdhandler.lv_resize(dbo.lvm_id, size_change,
+											pv_dests, resize_options)
+		LvCommon.handle_execute(rc, out, err)
+		return "/"
 
 	@dbus.service.method(
 		dbus_interface=LV_INTERFACE,
@@ -583,23 +580,11 @@ class Lv(LvCommon):
 	def _lv_activate_deactivate(uuid, lv_name, activate, control_flags,
 								options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, lv_name)
-
-		if dbo:
-			rc, out, err = cmdhandler.activate_deactivate(
-				'lvchange', lv_name, activate, control_flags, options)
-			if rc == 0:
-				dbo.refresh()
-				return '/'
-			else:
-				raise dbus.exceptions.DBusException(
-					LV_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				LV_INTERFACE,
-				'LV with uuid %s and name %s not present!' %
-				(uuid, lv_name))
+		LvCommon.validate_dbus_object(uuid, lv_name)
+		rc, out, err = cmdhandler.activate_deactivate(
+			'lvchange', lv_name, activate, control_flags, options)
+		LvCommon.handle_execute(rc, out, err)
+		return '/'
 
 	@dbus.service.method(
 		dbus_interface=LV_INTERFACE,
@@ -631,25 +616,11 @@ class Lv(LvCommon):
 	@staticmethod
 	def _add_rm_tags(uuid, lv_name, tags_add, tags_del, tag_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, lv_name)
-
-		if dbo:
-
-			rc, out, err = cmdhandler.lv_tag(
-				lv_name, tags_add, tags_del, tag_options)
-			if rc == 0:
-				dbo.refresh()
-				return '/'
-			else:
-				raise dbus.exceptions.DBusException(
-					LV_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-
-		else:
-			raise dbus.exceptions.DBusException(
-				LV_INTERFACE,
-				'LV with uuid %s and name %s not present!' %
-				(uuid, lv_name))
+		LvCommon.validate_dbus_object(uuid, lv_name)
+		rc, out, err = cmdhandler.lv_tag(
+			lv_name, tags_add, tags_del, tag_options)
+		LvCommon.handle_execute(rc, out, err)
+		return '/'
 
 	@dbus.service.method(
 		dbus_interface=LV_INTERFACE,
@@ -698,37 +669,22 @@ class LvThinPool(Lv):
 
 	@property
 	def DataLv(self):
-		return self._data_lv
+		return dbus.ObjectPath(self._data_lv)
 
 	@property
 	def MetaDataLv(self):
-		return self._metadata_lv
+		return dbus.ObjectPath(self._metadata_lv)
 
 	@staticmethod
 	def _lv_create(lv_uuid, lv_name, name, size_bytes, create_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(lv_uuid, lv_name)
+		dbo = LvCommon.validate_dbus_object(lv_uuid, lv_name)
 
-		lv_created = '/'
-
-		if dbo:
-			rc, out, err = cmdhandler.lv_lv_create(
-				lv_name, create_options, name, size_bytes)
-			if rc == 0:
-				full_name = "%s/%s" % (dbo.vg_name_lookup(), name)
-				lvs = load_lvs([full_name], emit_signal=True)[0]
-				for l in lvs:
-					lv_created = l.dbus_object_path()
-			else:
-				raise dbus.exceptions.DBusException(
-					LV_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				LV_INTERFACE,
-				'LV with uuid %s and name %s not present!' %
-				(lv_uuid, lv_name))
-		return lv_created
+		rc, out, err = cmdhandler.lv_lv_create(
+			lv_name, create_options, name, size_bytes)
+		LvCommon.handle_execute(rc, out, err)
+		full_name = "%s/%s" % (dbo.vg_name_lookup(), name)
+		return cfg.om.get_object_path_by_lvm_id(full_name)
 
 	@dbus.service.method(
 		dbus_interface=THIN_POOL_INTERFACE,
@@ -757,22 +713,21 @@ class LvCachePool(Lv):
 
 	@property
 	def DataLv(self):
-		return self._data_lv
+		return dbus.ObjectPath(self._data_lv)
 
 	@property
 	def MetaDataLv(self):
-		return self._metadata_lv
+		return dbus.ObjectPath(self._metadata_lv)
 
 	@staticmethod
 	def _cache_lv(lv_uuid, lv_name, lv_object_path, cache_options):
-
 		# Make sure we have a dbus object representing cache pool
-		dbo = cfg.om.get_object_by_uuid_lvm_id(lv_uuid, lv_name)
+		dbo = LvCommon.validate_dbus_object(lv_uuid, lv_name)
 
 		# Make sure we have dbus object representing lv to cache
 		lv_to_cache = cfg.om.get_object_by_path(lv_object_path)
 
-		if dbo and lv_to_cache:
+		if lv_to_cache:
 			fcn = lv_to_cache.lv_full_name()
 			rc, out, err = cmdhandler.lv_cache_lv(
 				dbo.lv_full_name(), fcn, cache_options)
@@ -780,27 +735,18 @@ class LvCachePool(Lv):
 				# When we cache an LV, the cache pool and the lv that is getting
 				# cached need to be removed from the object manager and
 				# re-created as their interfaces have changed!
-				cfg.om.remove_object(dbo, emit_signal=True)
-				cfg.om.remove_object(lv_to_cache, emit_signal=True)
+				mt_remove_dbus_objects((dbo, lv_to_cache))
 				cfg.load()
 
 				lv_converted = cfg.om.get_object_path_by_lvm_id(fcn)
-
 			else:
 				raise dbus.exceptions.DBusException(
 					LV_INTERFACE,
 					'Exit code %s, stderr = %s' % (str(rc), err))
 		else:
-			msg = ""
-			if not dbo:
-				dbo += 'CachePool LV with uuid %s and name %s not present!' % \
-					(lv_uuid, lv_name)
-
-			if not lv_to_cache:
-				dbo += 'LV to cache with object path %s not present!' % \
-					(lv_object_path)
-
-			raise dbus.exceptions.DBusException(LV_INTERFACE, msg)
+			raise dbus.exceptions.DBusException(
+				LV_INTERFACE, 'LV to cache with object path %s not present!' %
+				lv_object_path)
 		return lv_converted
 
 	@dbus.service.method(
@@ -826,37 +772,30 @@ class LvCacheLv(Lv):
 
 	@property
 	def CachePool(self):
-		return self.state.PoolLv
+		return dbus.ObjectPath(self.state.PoolLv)
 
 	@staticmethod
 	def _detach_lv(lv_uuid, lv_name, detach_options, destroy_cache):
 		# Make sure we have a dbus object representing cache pool
-		dbo = cfg.om.get_object_by_uuid_lvm_id(lv_uuid, lv_name)
+		dbo = LvCommon.validate_dbus_object(lv_uuid, lv_name)
 
-		if dbo:
+		# Get current cache name
+		cache_pool = cfg.om.get_object_by_path(dbo.CachePool)
 
-			# Get current cache name
-			cache_pool = cfg.om.get_object_by_path(dbo.CachePool)
+		rc, out, err = cmdhandler.lv_detach_cache(
+			dbo.lv_full_name(), detach_options, destroy_cache)
+		if rc == 0:
+			# The cache pool gets removed as hidden and put back to
+			# visible, so lets delete
+			mt_remove_dbus_objects((cache_pool, dbo))
+			cfg.load()
 
-			rc, out, err = cmdhandler.lv_detach_cache(
-				dbo.lv_full_name(), detach_options, destroy_cache)
-			if rc == 0:
-				# The cache pool gets removed as hidden and put back to
-				# visible, so lets delete
-				cfg.om.remove_object(cache_pool, emit_signal=True)
-				cfg.om.remove_object(dbo, emit_signal=True)
-				cfg.load()
-
-				uncached_lv_path = cfg.om.get_object_path_by_lvm_id(lv_name)
-			else:
-				raise dbus.exceptions.DBusException(
-					LV_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
+			uncached_lv_path = cfg.om.get_object_path_by_lvm_id(lv_name)
 		else:
 			raise dbus.exceptions.DBusException(
 				LV_INTERFACE,
-				'LV with uuid %s and name %s not present!' %
-				(lv_uuid, lv_name))
+				'Exit code %s, stderr = %s' % (str(rc), err))
+
 		return uncached_lv_path
 
 	@dbus.service.method(
@@ -890,4 +829,4 @@ class LvSnapShot(Lv):
 							(SNAPSHOT_INTERFACE, self.Uuid, self.lvm_id,
 							merge_options, job_state), cb, cbe, False,
 							job_state)
-		cfg.worker_q.put(r)
+		background.cmd_runner(r)

daemons/lvmdbusd/lvm_shell_proxy.py

@@ -14,17 +14,22 @@
 import subprocess
 import shlex
 from fcntl import fcntl, F_GETFL, F_SETFL
-from os import O_NONBLOCK
+import os
 import traceback
 import sys
-import re
+import tempfile
+import time
+import select
+import copy
 
 try:
-	from .cfg import LVM_CMD
-	from .utils import log_debug, log_error
-except:
-	from cfg import LVM_CMD
-	from utils import log_debug, log_error
+	import simplejson as json
+except ImportError:
+	import json
+
+
+from lvmdbusd.cfg import LVM_CMD
+from lvmdbusd.utils import log_debug, log_error
 
 SHELL_PROMPT = "lvm> "
 
@@ -37,43 +42,81 @@ def _quote_arg(arg):
 
 
 class LVMShellProxy(object):
-	def _read_until_prompt(self):
-		prev_ec = None
+
+	@staticmethod
+	def _read(stream):
+		tmp = stream.read()
+		if tmp:
+			return tmp.decode("utf-8")
+		return ''
+
+	# Read until we get prompt back and a result
+	# @param: no_output	Caller expects no output to report FD
+	# Returns stdout, report, stderr (report is JSON!)
+	def _read_until_prompt(self, no_output=False):
 		stdout = ""
-		while not stdout.endswith(SHELL_PROMPT):
+		report = ""
+		stderr = ""
+		keep_reading = True
+		extra_passes = 3
+		report_json = {}
+		prev_report_len = 0
+
+		# Try reading from all FDs to prevent one from filling up and causing
+		# a hang.  Keep reading until we get the prompt back and the report
+		# FD does not contain valid JSON
+		while keep_reading:
 			try:
-				tmp = self.lvm_shell.stdout.read()
-				if tmp:
-					stdout += tmp.decode("utf-8")
-			except IOError:
-				# nothing written yet
+				rd_fd = [
+					self.lvm_shell.stdout.fileno(),
+					self.report_stream.fileno(),
+					self.lvm_shell.stderr.fileno()]
+				ready = select.select(rd_fd, [], [], 2)
+
+				for r in ready[0]:
+					if r == self.lvm_shell.stdout.fileno():
+						stdout += LVMShellProxy._read(self.lvm_shell.stdout)
+					elif r == self.report_stream.fileno():
+						report += LVMShellProxy._read(self.report_stream)
+					elif r == self.lvm_shell.stderr.fileno():
+						stderr += LVMShellProxy._read(self.lvm_shell.stderr)
+
+				# Check to see if the lvm process died on us
+				if self.lvm_shell.poll():
+					raise Exception(self.lvm_shell.returncode, "%s" % stderr)
+
+				if stdout.endswith(SHELL_PROMPT):
+					if no_output:
+						keep_reading = False
+					else:
+						cur_report_len = len(report)
+						if cur_report_len != 0:
+							# Only bother to parse if we have more data
+							if prev_report_len != cur_report_len:
+								prev_report_len = cur_report_len
+								# Parse the JSON if it's good we are done,
+								# if not we will try to read some more.
+								try:
+									report_json = json.loads(report)
+									keep_reading = False
+								except ValueError:
+									pass
+
+						if keep_reading:
+							extra_passes -= 1
+							if extra_passes <= 0:
+								if len(report):
+									raise ValueError("Invalid json: %s" %
+														report)
+								else:
+									raise ValueError(
+										"lvm returned no JSON output!")
+
+			except IOError as ioe:
+				log_debug(str(ioe))
 				pass
 
-		# strip the prompt from the STDOUT before returning and grab the exit
-		# code if it's available
-		m = self.re.match(stdout)
-		if m:
-			prev_ec = int(m.group(2))
-			strip_idx = -1 * len(m.group(1))
-		else:
-			strip_idx = -1 * len(SHELL_PROMPT)
-
-		return stdout[:strip_idx], prev_ec
-
-	def _read_line(self):
-		while True:
-			try:
-				tmp = self.lvm_shell.stdout.readline()
-				if tmp:
-					return tmp.decode("utf-8")
-			except IOError:
-				pass
-
-	def _discard_echo(self, expected):
-		line = ""
-		while line != expected:
-			# GNU readline inserts some interesting characters at times...
-			line += self._read_line().replace(' \r', '')
+		return stdout, report_json, stderr
 
 	def _write_cmd(self, cmd):
 		cmd_bytes = bytes(cmd, "utf-8")
@@ -81,39 +124,88 @@ class LVMShellProxy(object):
 		assert (num_written == len(cmd_bytes))
 		self.lvm_shell.stdin.flush()
 
-	def _lvm_echos(self):
-		echo = False
-		cmd = "version\n"
-		self._write_cmd(cmd)
-		line = self._read_line()
-
-		if line == cmd:
-			echo = True
-
-		self._read_until_prompt()
-
-		return echo
+	@staticmethod
+	def _make_non_block(stream):
+		flags = fcntl(stream, F_GETFL)
+		fcntl(stream, F_SETFL, flags | os.O_NONBLOCK)
 
 	def __init__(self):
-		self.re = re.compile(".*(\[(-?[0-9]+)\] lvm> $)", re.DOTALL)
+
+		# Create a temp directory
+		tmp_dir = tempfile.mkdtemp(prefix="lvmdbus_")
+		tmp_file = "%s/lvmdbus_report" % (tmp_dir)
+
+		try:
+			# Lets create fifo for the report output
+			os.mkfifo(tmp_file, 0o600)
+		except FileExistsError:
+			pass
+
+		# We have to open non-blocking as the other side isn't open until
+		# we actually fork the process.
+		self.report_fd = os.open(tmp_file, os.O_NONBLOCK)
+		self.report_stream = os.fdopen(self.report_fd, 'rb', 0)
+
+		# Setup the environment for using our own socket for reporting
+		local_env = copy.deepcopy(os.environ)
+		local_env["LVM_REPORT_FD"] = "32"
+		local_env["LVM_COMMAND_PROFILE"] = "lvmdbusd"
+
+		# Disable the abort logic if lvm logs too much, which easily happens
+		# when utilizing the lvm shell.
+		local_env["LVM_LOG_FILE_MAX_LINES"] = "0"
 
 		# run the lvm shell
 		self.lvm_shell = subprocess.Popen(
-			[LVM_CMD], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
-			stderr=subprocess.PIPE, close_fds=True)
-		flags = fcntl(self.lvm_shell.stdout, F_GETFL)
-		fcntl(self.lvm_shell.stdout, F_SETFL, flags | O_NONBLOCK)
-		flags = fcntl(self.lvm_shell.stderr, F_GETFL)
-		fcntl(self.lvm_shell.stderr, F_SETFL, flags | O_NONBLOCK)
+			[LVM_CMD + " 32>%s" % tmp_file],
+			stdin=subprocess.PIPE, stdout=subprocess.PIPE, env=local_env,
+			stderr=subprocess.PIPE, close_fds=True, shell=True)
 
-		# wait for the first prompt
-		self._read_until_prompt()
+		try:
+			LVMShellProxy._make_non_block(self.lvm_shell.stdout)
+			LVMShellProxy._make_non_block(self.lvm_shell.stderr)
 
-		# Check to see if the version of LVM we are using is running with
-		# gnu readline which will echo our writes from stdin to stdout
-		self.echo = self._lvm_echos()
+			# wait for the first prompt
+			errors = self._read_until_prompt(no_output=True)[2]
+			if errors and len(errors):
+				raise RuntimeError(errors)
+		except:
+			raise
+		finally:
+			# These will get deleted when the FD count goes to zero so we
+			# can be sure to clean up correctly no matter how we finish
+			os.unlink(tmp_file)
+			os.rmdir(tmp_dir)
+
+	def get_error_msg(self):
+		# We got an error, lets go fetch the error message
+		self._write_cmd('lastlog\n')
+
+		# read everything from the STDOUT to the next prompt
+		stdout, report_json, stderr = self._read_until_prompt()
+		if 'log' in report_json:
+			error_msg = ""
+			# Walk the entire log array and build an error string
+			for log_entry in report_json['log']:
+				if log_entry['log_type'] == "error":
+					if error_msg:
+						error_msg += ', ' + log_entry['log_message']
+					else:
+						error_msg = log_entry['log_message']
+
+			return error_msg
+
+		return 'No error reason provided! (missing "log" section)'
 
 	def call_lvm(self, argv, debug=False):
+		rc = 1
+		error_msg = ""
+
+		if self.lvm_shell.poll():
+			raise Exception(
+				self.lvm_shell.returncode,
+				"Underlying lvm shell process is not present!")
+
 		# create the command string
 		cmd = " ".join(_quote_arg(arg) for arg in argv)
 		cmd += "\n"
@@ -121,46 +213,34 @@ class LVMShellProxy(object):
 		# run the command by writing it to the shell's STDIN
 		self._write_cmd(cmd)
 
-		# If lvm is utilizing gnu readline, it echos stdin to stdout
-		if self.echo:
-			self._discard_echo(cmd)
-
 		# read everything from the STDOUT to the next prompt
-		stdout, exit_code = self._read_until_prompt()
+		stdout, report_json, stderr = self._read_until_prompt()
 
-		# read everything from STDERR if there's something (we waited for the
-		# prompt on STDOUT so there should be all or nothing at this point on
-		# STDERR)
-		stderr = None
-		try:
-			t_error = self.lvm_shell.stderr.read()
-			if t_error:
-				stderr = t_error.decode("utf-8")
-		except IOError:
-			# nothing on STDERR
-			pass
-
-		if exit_code is not None:
-			rc = exit_code
-		else:
-			# LVM does write to stderr even when it did complete successfully,
-			# so without having the exit code in the prompt we can never be
-			# sure.
-			if stderr:
-				rc = 1
-			else:
+		# Parse the report to see what happened
+		if 'log' in report_json:
+			if report_json['log'][-1:][0]['log_ret_code'] == '1':
 				rc = 0
+			else:
+				error_msg = self.get_error_msg()
 
 		if debug or rc != 0:
 			log_error(('CMD: %s' % cmd))
 			log_error(("EC = %d" % rc))
-			log_error(("STDOUT=\n %s\n" % stdout))
-			log_error(("STDERR=\n %s\n" % stderr))
+			log_error(("ERROR_MSG=\n %s\n" % error_msg))
 
-		return (rc, stdout, stderr)
+		return rc, report_json, error_msg
+
+	def exit_shell(self):
+		try:
+			self._write_cmd('exit\n')
+		except Exception as e:
+			log_error(str(e))
 
 	def __del__(self):
-		self.lvm_shell.terminate()
+		try:
+			self.lvm_shell.terminate()
+		except:
+			pass
 
 
 if __name__ == "__main__":
@@ -170,15 +250,18 @@ if __name__ == "__main__":
 		while in_line:
 			in_line = input("lvm> ")
 			if in_line:
-				ret, out, err, = shell.call_lvm(in_line.split())
-				print(("RET: %d" % ret))
+				start = time.time()
+				ret, out, err = shell.call_lvm(in_line.split())
+				end = time.time()
+
+				print(("RC: %d" % ret))
 				print(("OUT:\n%s" % out))
 				print(("ERR:\n%s" % err))
+
+				print("Command     = %f seconds" % (end - start))
 	except KeyboardInterrupt:
 		pass
 	except EOFError:
 		pass
 	except Exception:
 		traceback.print_exc(file=sys.stdout)
-	finally:
-		print()

daemons/lvmdbusd/lvmdb.py

@@ -15,16 +15,12 @@ import pprint as prettyprint
 import os
 import sys
 
-try:
-	from . import cmdhandler
-	from .utils import log_debug, log_error
-except SystemError:
-	import cmdhandler
-	from utils import log_debug
+from lvmdbusd import cmdhandler
+from lvmdbusd.utils import log_debug, log_error
 
 
 class DataStore(object):
-	def __init__(self, usejson=None):
+	def __init__(self, usejson=True):
 		self.pvs = {}
 		self.vgs = {}
 		self.lvs = {}
@@ -42,7 +38,7 @@ class DataStore(object):
 		# self.refresh()
 		self.num_refreshes = 0
 
-		if usejson is None:
+		if usejson:
 			self.json = cmdhandler.supports_json()
 		else:
 			self.json = usejson
@@ -73,18 +69,7 @@ class DataStore(object):
 			table[key] = record
 
 	@staticmethod
-	def _parse_pvs(_pvs):
-		pvs = sorted(_pvs, key=lambda pk: pk['pv_name'])
-
-		c_pvs = OrderedDict()
-		c_lookup = {}
-		c_pvs_in_vgs = {}
-
-		for p in pvs:
-			DataStore._insert_record(
-				c_pvs, p['pv_uuid'], p,
-				['pvseg_start', 'pvseg_size', 'segtype'])
-
+	def _pvs_parse_common(c_pvs, c_pvs_in_vgs, c_lookup):
 		for p in c_pvs.values():
 			# Capture which PVs are associated with which VG
 			if p['vg_uuid'] not in c_pvs_in_vgs:
@@ -97,6 +82,20 @@ class DataStore(object):
 			# Lookup for translating between /dev/<name> and pv uuid
 			c_lookup[p['pv_name']] = p['pv_uuid']
 
+	@staticmethod
+	def _parse_pvs(_pvs):
+		pvs = sorted(_pvs, key=lambda pk: pk['pv_name'])
+
+		c_pvs = OrderedDict()
+		c_lookup = {}
+		c_pvs_in_vgs = {}
+
+		for p in pvs:
+			DataStore._insert_record(
+				c_pvs, p['pv_uuid'], p,
+				['pvseg_start', 'pvseg_size', 'segtype'])
+
+		DataStore._pvs_parse_common(c_pvs, c_pvs_in_vgs, c_lookup)
 		return c_pvs, c_lookup, c_pvs_in_vgs
 
 	@staticmethod
@@ -136,17 +135,7 @@ class DataStore(object):
 						i['pvseg_size'] = i['pv_pe_count']
 						i['segtype'] = 'free'
 
-		for p in c_pvs.values():
-			# Capture which PVs are associated with which VG
-			if p['vg_uuid'] not in c_pvs_in_vgs:
-				c_pvs_in_vgs[p['vg_uuid']] = []
-
-			if p['vg_name']:
-				c_pvs_in_vgs[p['vg_uuid']].append(
-					(p['pv_name'], p['pv_uuid']))
-
-			# Lookup for translating between /dev/<name> and pv uuid
-			c_lookup[p['pv_name']] = p['pv_uuid']
+		DataStore._pvs_parse_common(c_pvs, c_pvs_in_vgs, c_lookup)
 
 		return c_pvs, c_lookup, c_pvs_in_vgs
 
@@ -437,6 +426,12 @@ class DataStore(object):
 				rc.append(self.pvs[self.pv_path_to_uuid[s]])
 			return rc
 
+	def pv_missing(self, pv_uuid):
+		if pv_uuid in self.pvs:
+			if self.pvs[pv_uuid]['pv_missing'] == '':
+				return False
+		return True
+
 	def fetch_vgs(self, vg_name):
 		if not vg_name:
 			return self.vgs.values()
@@ -520,6 +515,7 @@ if __name__ == "__main__":
 	print("PVS")
 	for v in ds.pvs.values():
 		pp.pprint(v)
+		print('PV missing is %s' % ds.pv_missing(v['pv_uuid']))
 
 	print("VGS")
 	for v in ds.vgs.values():

daemons/lvmdbusd/lvmdbusd

@@ -10,7 +10,7 @@
 # along with this program. If not, see <http://www.gnu.org/licenses/>.
 
 import sys
-import lvmdbusd
+from lvmdbusd import main
 
 if __name__ == '__main__':
-	sys.exit(lvmdbusd.main())
+	sys.exit(main())

daemons/lvmdbusd/main.py

@@ -10,25 +10,26 @@
 from . import cfg
 from . import objectmanager
 from . import utils
-from .cfg import BASE_INTERFACE, BASE_OBJ_PATH, MANAGER_OBJ_PATH
+from .cfg import BUS_NAME, BASE_INTERFACE, BASE_OBJ_PATH, MANAGER_OBJ_PATH
 import threading
 from . import cmdhandler
 import time
 import signal
 import dbus
+import dbus.mainloop.glib
 from . import lvmdb
 # noinspection PyUnresolvedReferences
 from gi.repository import GLib
-from .fetch import load
+from .fetch import StateUpdate
 from .manager import Manager
-from .background import background_reaper
 import traceback
 import queue
 from . import udevwatch
-from .utils import log_debug
+from .utils import log_debug, log_error
 import argparse
 import os
-from .refresh import handle_external_event, event_complete
+import sys
+from .cmdhandler import LvmFlightRecorder
 
 
 class Lvm(objectmanager.ObjectManager):
@@ -36,54 +37,16 @@ class Lvm(objectmanager.ObjectManager):
 		super(Lvm, self).__init__(object_path, BASE_INTERFACE)
 
 
-def _discard_pending_refreshes():
-	# We just handled a refresh, if we have any in the queue they can be
-	# removed because by definition they are older than the refresh we just did.
-	# As we limit the number of refreshes getting into the queue
-	# we should only ever have one to remove.
-	requests = []
-	while not cfg.worker_q.empty():
-		try:
-			r = cfg.worker_q.get(block=False)
-			if r.method != handle_external_event:
-				requests.append(r)
-			else:
-				# Make sure we make this event complete even though it didn't
-				# run, otherwise no other events will get processed
-				event_complete()
-				break
-		except queue.Empty:
-			break
-
-	# Any requests we removed, but did not discard need to be re-queued
-	for r in requests:
-		cfg.worker_q.put(r)
-
-
 def process_request():
 	while cfg.run.value != 0:
+		# noinspection PyBroadException
 		try:
 			req = cfg.worker_q.get(True, 5)
-
-			start = cfg.db.num_refreshes
-
 			log_debug(
 				"Running method: %s with args %s" %
 				(str(req.method), str(req.arguments)))
 			req.run_cmd()
-
-			end = cfg.db.num_refreshes
-
-			num_refreshes = end - start
-
-			if num_refreshes > 0:
-				_discard_pending_refreshes()
-
-				if num_refreshes > 1:
-					log_debug(
-						"Inspect method %s for too many refreshes" %
-						(str(req.method)))
-			log_debug("Complete ")
+			log_debug("Method complete ")
 		except queue.Empty:
 			pass
 		except Exception:
@@ -91,34 +54,62 @@ def process_request():
 			utils.log_error("process_request exception: \n%s" % st)
 
 
+def check_bb_size(value):
+	v = int(value)
+	if v < 0:
+		raise argparse.ArgumentTypeError(
+			"positive integers only ('%s' invalid)" % value)
+	return v
+
+
 def main():
+	start = time.time()
 	# Add simple command line handling
 	parser = argparse.ArgumentParser()
-	parser.add_argument("--udev", action='store_true',
-						help="Use udev for updating state", default=False,
-						dest='use_udev')
-	parser.add_argument("--debug", action='store_true',
-						help="Dump debug messages", default=False,
-						dest='debug')
-
-	parser.add_argument("--nojson", action='store_false',
-						help="Do not use LVM JSON output", default=None,
-						dest='use_json')
+	parser.add_argument(
+		"--udev", action='store_true',
+		help="Use udev for updating state",
+		default=False,
+		dest='use_udev')
+	parser.add_argument(
+		"--debug", action='store_true',
+		help="Dump debug messages", default=False,
+		dest='debug')
+	parser.add_argument(
+		"--nojson", action='store_false',
+		help="Do not use LVM JSON output (disables lvmshell)", default=True,
+		dest='use_json')
+	parser.add_argument(
+		"--lvmshell", action='store_true',
+		help="Use the lvm shell, not fork & exec lvm",
+		default=False,
+		dest='use_lvm_shell')
+	parser.add_argument(
+		"--blackboxsize",
+		help="Size of the black box flight recorder, 0 to disable",
+		default=10,
+		type=check_bb_size,
+		dest='bb_size')
 
 	use_session = os.getenv('LVMDBUSD_USE_SESSION', False)
 
 	# Ensure that we get consistent output for parsing stdout/stderr
 	os.environ["LC_ALL"] = "C"
 
-	args = parser.parse_args()
+	cfg.args = parser.parse_args()
 
-	cfg.DEBUG = args.debug
+	# We create a flight recorder in cmdhandler too, but we replace it here
+	# as the user may be specifying a different size.  The default one in
+	# cmdhandler is for when we are running other code with a different main.
+	cfg.blackbox = LvmFlightRecorder(cfg.args.bb_size)
+
+	if cfg.args.use_lvm_shell and not cfg.args.use_json:
+		log_error("You cannot specify --lvmshell and --nojson")
+		sys.exit(1)
 
 	# List of threads that we start up
 	thread_list = []
 
-	start = time.time()
-
 	# Install signal handlers
 	for s in [signal.SIGHUP, signal.SIGINT]:
 		try:
@@ -129,54 +120,60 @@ def main():
 	dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
 	dbus.mainloop.glib.threads_init()
 
+	cmdhandler.set_execution(cfg.args.use_lvm_shell)
+
 	if use_session:
 		cfg.bus = dbus.SessionBus()
 	else:
 		cfg.bus = dbus.SystemBus()
 	# The base name variable needs to exist for things to work.
 	# noinspection PyUnusedLocal
-	base_name = dbus.service.BusName(BASE_INTERFACE, cfg.bus)
+	base_name = dbus.service.BusName(BUS_NAME, cfg.bus)
 	cfg.om = Lvm(BASE_OBJ_PATH)
 	cfg.om.register_object(Manager(MANAGER_OBJ_PATH))
 
-	cfg.load = load
+	cfg.db = lvmdb.DataStore(cfg.args.use_json)
 
-	cfg.db = lvmdb.DataStore(args.use_json)
-
-	# Start up thread to monitor pv moves
-	thread_list.append(
-		threading.Thread(target=background_reaper, name="pv_move_reaper"))
-
-	# Using a thread to process requests.
+	# Using a thread to process requests, we cannot hang the dbus library
+	# thread that is handling the dbus interface
 	thread_list.append(threading.Thread(target=process_request))
 
-	cfg.load(refresh=False, emit_signal=False)
+	# Have a single thread handling updating lvm and the dbus model so we
+	# don't have multiple threads doing this as the same time
+	updater = StateUpdate()
+	thread_list.append(updater.thread)
+
+	cfg.load = updater.load
+	cfg.event = updater.event
+
 	cfg.loop = GLib.MainLoop()
 
-	for process in thread_list:
-		process.damon = True
-		process.start()
+	for thread in thread_list:
+		thread.damon = True
+		thread.start()
+
+	# Add udev watching
+	if cfg.args.use_udev:
+		log_debug('Utilizing udev to trigger updates')
+
+	# In all cases we are going to monitor for udev until we get an
+	# ExternalEvent.  In the case where we get an external event and the user
+	# didn't specify --udev we will stop monitoring udev
+	udevwatch.add()
 
 	end = time.time()
 	log_debug(
-		'Service ready! total time= %.2f, lvm time= %.2f count= %d' %
+		'Service ready! total time= %.4f, lvm time= %.4f count= %d' %
 		(end - start, cmdhandler.total_time, cmdhandler.total_count),
 		'bg_black', 'fg_light_green')
 
-	# Add udev watching
-	if args.use_udev:
-		log_debug('Utilizing udev to trigger updates')
-		udevwatch.add()
-
 	try:
 		if cfg.run.value != 0:
 			cfg.loop.run()
+			udevwatch.remove()
 
-			if args.use_udev:
-				udevwatch.remove()
-
-			for process in thread_list:
-				process.join()
+			for thread in thread_list:
+				thread.join()
 	except KeyboardInterrupt:
 		utils.handler(signal.SIGINT, None)
 	return 0

daemons/lvmdbusd/manager.py

@@ -14,14 +14,13 @@ from .cfg import MANAGER_INTERFACE
 import dbus
 from . import cfg
 from . import cmdhandler
-from .fetch import load_pvs, load_vgs
 from .request import RequestEntry
-from .refresh import event_add
+from . import udevwatch
 
 
 # noinspection PyPep8Naming
 class Manager(AutomatedProperties):
-	_Version_meta = ("t", MANAGER_INTERFACE)
+	_Version_meta = ("s", MANAGER_INTERFACE)
 
 	def __init__(self, object_path):
 		super(Manager, self).__init__(object_path)
@@ -29,31 +28,31 @@ class Manager(AutomatedProperties):
 
 	@property
 	def Version(self):
-		return '1.0.0'
+		return dbus.String('1.0.0')
+
+	@staticmethod
+	def handle_execute(rc, out, err):
+		if rc == 0:
+			cfg.load()
+		else:
+			# Need to work on error handling, need consistent
+			raise dbus.exceptions.DBusException(
+				MANAGER_INTERFACE,
+				'Exit code %s, stderr = %s' % (str(rc), err))
 
 	@staticmethod
 	def _pv_create(device, create_options):
 
 		# Check to see if we are already trying to create a PV for an existing
 		# PV
-		pv = cfg.om.get_object_path_by_uuid_lvm_id(
-			device, device, None, False)
+		pv = cfg.om.get_object_path_by_uuid_lvm_id(device, device)
 		if pv:
 			raise dbus.exceptions.DBusException(
 				MANAGER_INTERFACE, "PV Already exists!")
 
-		created_pv = []
 		rc, out, err = cmdhandler.pv_create(create_options, [device])
-		if rc == 0:
-			pvs = load_pvs([device], emit_signal=True)[0]
-			for p in pvs:
-				created_pv = p.dbus_object_path()
-		else:
-			raise dbus.exceptions.DBusException(
-				MANAGER_INTERFACE,
-				'Exit code %s, stderr = %s' % (str(rc), err))
-
-		return created_pv
+		Manager.handle_execute(rc, out, err)
+		return cfg.om.get_object_path_by_lvm_id(device)
 
 	@dbus.service.method(
 		dbus_interface=MANAGER_INTERFACE,
@@ -80,20 +79,8 @@ class Manager(AutomatedProperties):
 					MANAGER_INTERFACE, 'object path = %s not found' % p)
 
 		rc, out, err = cmdhandler.vg_create(create_options, pv_devices, name)
-		created_vg = "/"
-
-		if rc == 0:
-			vgs = load_vgs([name], emit_signal=True)[0]
-			for v in vgs:
-				created_vg = v.dbus_object_path()
-
-			# Update the PVS
-			load_pvs(refresh=True, emit_signal=True, cache_refresh=False)
-		else:
-			raise dbus.exceptions.DBusException(
-				MANAGER_INTERFACE,
-				'Exit code %s, stderr = %s' % (str(rc), err))
-		return created_vg
+		Manager.handle_execute(rc, out, err)
+		return cfg.om.get_object_path_by_lvm_id(name)
 
 	@dbus.service.method(
 		dbus_interface=MANAGER_INTERFACE,
@@ -114,6 +101,10 @@ class Manager(AutomatedProperties):
 
 		# This is a diagnostic and should not be run in normal operation, so
 		# lets remove the log entries for refresh as it's implied.
+
+		# Run an internal diagnostic on the object manager look up tables
+		lc = cfg.om.validate_lookups()
+
 		rc = cfg.load(log=False)
 
 		if rc != 0:
@@ -121,7 +112,7 @@ class Manager(AutomatedProperties):
 							'bg_black', 'fg_light_red')
 		else:
 			utils.log_debug('Manager.Refresh - exit %d' % (rc))
-		return rc
+		return rc + lc
 
 	@dbus.service.method(
 		dbus_interface=MANAGER_INTERFACE,
@@ -159,29 +150,44 @@ class Manager(AutomatedProperties):
 		:param key: The lookup value
 		:return: Return the object path.  If object not found you will get '/'
 		"""
-		p = cfg.om.get_object_path_by_uuid_lvm_id(
-			key, key, gen_new=False)
+		p = cfg.om.get_object_path_by_uuid_lvm_id(key, key)
 		if p:
 			return p
 		return '/'
 
+	@staticmethod
+	def _use_lvm_shell(yes_no):
+		return dbus.Boolean(cmdhandler.set_execution(yes_no))
+
 	@dbus.service.method(
 		dbus_interface=MANAGER_INTERFACE,
-		in_signature='b')
-	def UseLvmShell(self, yes_no):
+		in_signature='b', out_signature='b',
+		async_callbacks=('cb', 'cbe'))
+	def UseLvmShell(self, yes_no, cb, cbe):
 		"""
 		Allow the client to enable/disable lvm shell, used for testing
 		:param yes_no:
+		:param cb:	dbus python call back parameter, not client visible
+		:param cbe:	dbus python error call back parameter, not client visible
 		:return: Nothing
 		"""
-		cmdhandler.set_execution(yes_no)
+		r = RequestEntry(-1, Manager._use_lvm_shell, (yes_no,), cb, cbe, False)
+		cfg.worker_q.put(r)
 
 	@dbus.service.method(
 		dbus_interface=MANAGER_INTERFACE,
 		in_signature='s', out_signature='i')
 	def ExternalEvent(self, command):
 
-		event_add((command,))
+		# If a user didn't explicitly specify udev, we will turn it off now.
+		if not cfg.args.use_udev:
+			if udevwatch.remove():
+				utils.log_debug("ExternalEvent received, disabling "
+								"udev monitoring")
+				# We are dependent on external events now to stay current!
+				cfg.ee = True
+		utils.log_debug("ExternalEvent %s" % command)
+		cfg.event()
 		return dbus.Int32(0)
 
 	@staticmethod
@@ -191,15 +197,8 @@ class Manager(AutomatedProperties):
 			activate, cache, device_path,
 			major_minor, scan_options)
 
-		if rc == 0:
-			# This could potentially change the state quite a bit, so lets
-			# update everything to be safe
-			cfg.load()
-			return '/'
-		else:
-			raise dbus.exceptions.DBusException(
-				MANAGER_INTERFACE,
-				'Exit code %s, stderr = %s' % (str(rc), err))
+		Manager.handle_execute(rc, out, err)
+		return '/'
 
 	@dbus.service.method(
 		dbus_interface=MANAGER_INTERFACE,

daemons/lvmdbusd/objectmanager.py

@@ -12,8 +12,9 @@ import threading
 import traceback
 import dbus
 import os
+import copy
 from . import cfg
-from .utils import log_debug
+from .utils import log_debug, pv_obj_path_generate, log_error
 from .automatedproperties import AutomatedProperties
 
 
@@ -70,12 +71,37 @@ class ObjectManager(AutomatedProperties):
 		log_debug(('SIGNAL: InterfacesRemoved(%s, %s)' %
 			(str(object_path), str(interface_list))))
 
+	def validate_lookups(self):
+		with self.rlock:
+			tmp_lookups = copy.deepcopy(self._id_to_object_path)
+
+			# iterate over all we know, removing from the copy.  If all is well
+			# we will have zero items left over
+			for path, md in self._objects.items():
+				obj, lvm_id, uuid = md
+
+				if lvm_id:
+					assert path == tmp_lookups[lvm_id]
+					del tmp_lookups[lvm_id]
+
+				if uuid:
+					assert path == tmp_lookups[uuid]
+					del tmp_lookups[uuid]
+
+			rc = len(tmp_lookups)
+			if rc:
+				# Error condition
+				log_error("_id_to_object_path has extraneous lookups!")
+				for key, path in tmp_lookups.items():
+					log_error("Key= %s, path= %s" % (key, path))
+		return rc
+
 	def _lookup_add(self, obj, path, lvm_id, uuid):
 		"""
 		Store information about what we added to the caches so that we
 		can remove it cleanly
 		:param obj:     The dbus object we are storing
-		:param lvm_id:  The user name for the asset
+		:param lvm_id:  The lvm id for the asset
 		:param uuid:    The uuid for the asset
 		:return:
 		"""
@@ -85,7 +111,12 @@ class ObjectManager(AutomatedProperties):
 		self._lookup_remove(path)
 
 		self._objects[path] = (obj, lvm_id, uuid)
-		self._id_to_object_path[lvm_id] = path
+
+		# Make sure we have one or the other
+		assert lvm_id or uuid
+
+		if lvm_id:
+			self._id_to_object_path[lvm_id] = path
 
 		if uuid:
 			self._id_to_object_path[uuid] = path
@@ -94,8 +125,13 @@ class ObjectManager(AutomatedProperties):
 		# Note: Only called internally, lock implied
 		if obj_path in self._objects:
 			(obj, lvm_id, uuid) = self._objects[obj_path]
-			del self._id_to_object_path[lvm_id]
-			del self._id_to_object_path[uuid]
+
+			if lvm_id in self._id_to_object_path:
+				del self._id_to_object_path[lvm_id]
+
+			if uuid in self._id_to_object_path:
+				del self._id_to_object_path[uuid]
+
 			del self._objects[obj_path]
 
 	def lookup_update(self, dbus_obj, new_uuid, new_lvm_id):
@@ -125,7 +161,7 @@ class ObjectManager(AutomatedProperties):
 			path, props = dbus_object.emit_data()
 
 			# print('Registering object path %s for %s' %
-			#		(path, dbus_object.lvm_id))
+			# (path, dbus_object.lvm_id))
 
 			# We want fast access to the object by a number of different ways
 			# so we use multiple hashs with different keys
@@ -173,7 +209,7 @@ class ObjectManager(AutomatedProperties):
 	def get_object_by_uuid_lvm_id(self, uuid, lvm_id):
 		with self.rlock:
 			return self.get_object_by_path(
-				self.get_object_path_by_uuid_lvm_id(uuid, lvm_id, None, False))
+				self.get_object_path_by_uuid_lvm_id(uuid, lvm_id))
 
 	def get_object_by_lvm_id(self, lvm_id):
 		"""
@@ -206,78 +242,111 @@ class ObjectManager(AutomatedProperties):
 		:return: None
 		"""
 		# This gets called when we found an object based on lvm_id, ensure
-		# uuid is correct too, as they can change
+		# uuid is correct too, as they can change. There is no durable
+		# non-changeable name in lvm
 		if lvm_id != uuid:
-			if uuid not in self._id_to_object_path:
+			if uuid and uuid not in self._id_to_object_path:
 				obj = self.get_object_by_path(path)
 				self._lookup_add(obj, path, lvm_id, uuid)
 
-	def _return_lookup(self, uuid, lvm_identifier):
+	def _lvm_id_verify(self, path, uuid, lvm_id):
 		"""
-		We found an identifier, so lets return the path to the found object
-		:param uuid:	The lvm uuid
-		:param lvm_identifier: The lvm_id used to find object
-		:return:
+		Ensure lvm_id is present for a successful uuid lookup
+		NOTE: Internal call, assumes under object manager lock
+		:param path: 		Path to object we looked up
+		:param uuid: 		uuid used to find object
+		:param lvm_id:		lvm_id to verify
+		:return: None
 		"""
-		path = self._id_to_object_path[lvm_identifier]
-		self._uuid_verify(path, uuid, lvm_identifier)
+		# This gets called when we found an object based on uuid, ensure
+		# lvm_id is correct too, as they can change.  There is no durable
+		# non-changeable name in lvm
+		if lvm_id != uuid:
+			if lvm_id and lvm_id not in self._id_to_object_path:
+				obj = self.get_object_by_path(path)
+				self._lookup_add(obj, path, lvm_id, uuid)
+
+	def _id_lookup(self, the_id):
+		path = None
+
+		if the_id:
+			# The _id_to_object_path contains hash keys for everything, so
+			# uuid and lvm_id
+			if the_id in self._id_to_object_path:
+				path = self._id_to_object_path[the_id]
+			else:
+				if "/" in the_id:
+					if the_id.startswith('/'):
+						# We could have a pv device path lookup that failed,
+						# lets try canonical form and try again.
+						canonical = os.path.realpath(the_id)
+						if canonical in self._id_to_object_path:
+							path = self._id_to_object_path[canonical]
+					else:
+						vg, lv = the_id.split("/", 1)
+						int_lvm_id = vg + "/" + ("[%s]" % lv)
+						if int_lvm_id in self._id_to_object_path:
+							path = self._id_to_object_path[int_lvm_id]
 		return path
 
-	def get_object_path_by_uuid_lvm_id(self, uuid, lvm_id, path_create=None,
-										gen_new=True):
+	def get_object_path_by_uuid_lvm_id(self, uuid, lvm_id, path_create=None):
 		"""
-		For a given lvm asset return the dbus object registered to it.  If the
-		object is not found and gen_new == True and path_create is a valid
-		function we will create a new path, register it and return it.
-		:param uuid: The uuid for the lvm object
-		:param lvm_id: The lvm name
-		:param path_create: If true create an object path if not found
-		:param gen_new: The function used to create the new path
+		For a given lvm asset return the dbus object path registered for it.
+		This method first looks up by uuid and then by lvm_id.  You
+		can search by just one by setting uuid == lvm_id (uuid or lvm_id).
+		If the object is not found and path_create is a not None, the
+		path_create function will be called to create a new object path and
+		register it with the object manager for the specified uuid & lvm_id.
+		Note: If path create is not None, uuid and lvm_id cannot be equal
+		:param uuid: The uuid for the lvm object we are searching for
+		:param lvm_id: The lvm name (eg. pv device path, vg name, lv full name)
+		:param path_create: If not None, create the path using this function if
+				we fail to find the object by uuid or lvm_id.
+		:returns None if lvm asset not found and path_create == None otherwise
+				a valid dbus object path
 		"""
 		with self.rlock:
 			assert lvm_id
 			assert uuid
 
-			if gen_new:
-				assert path_create
+			if path_create:
+				assert uuid != lvm_id
 
-			path = None
-
-			if lvm_id in self._id_to_object_path:
-				self._return_lookup(uuid, lvm_id)
-
-			if "/" in lvm_id:
-				vg, lv = lvm_id.split("/", 1)
-				int_lvm_id = vg + "/" + ("[%s]" % lv)
-				if int_lvm_id in self._id_to_object_path:
-					self._return_lookup(uuid, int_lvm_id)
-				elif lvm_id.startswith('/'):
-					# We could have a pv device path lookup that failed,
-					# lets try canonical form and try again.
-					canonical = os.path.realpath(lvm_id)
-					if canonical in self._id_to_object_path:
-						self._return_lookup(uuid, canonical)
-
-			if uuid and uuid in self._id_to_object_path:
-				# If we get here it indicates that we found the object, but
-				# the lvm_id lookup failed.  In the case of a rename, the uuid
-				# will be correct, but the lvm_id will be wrong and vise versa.
-				# If the lvm_id does not equal the uuid, lets fix up the table
-				# so that lookups will be handled correctly.
-				path = self._id_to_object_path[uuid]
-
-				# In some cases we are looking up by one or the other, don't
-				# update when they are the same.
-				if uuid != lvm_id:
-					obj = self.get_object_by_path(path)
-					self._lookup_add(obj, path, lvm_id, uuid)
+			# Check for Manager.LookUpByLvmId query, we cannot
+			# check/verify/update the uuid and lvm_id lookups so don't!
+			if uuid == lvm_id:
+				path = self._id_lookup(lvm_id)
 			else:
-				if gen_new:
-					path = path_create()
-					self._lookup_add(None, path, lvm_id, uuid)
+				# We have a uuid and a lvm_id we can do sanity checks to ensure
+				# that they are consistent
 
-			# pprint('get_object_path_by_lvm_id(%s, %s, %s, %s: return %s' %
-			#       (uuid, lvm_id, str(path_create), str(gen_new), path))
+				# If a PV is missing it's device path is '[unknown]' or some
+				# other text derivation of unknown.  When we find that a PV is
+				# missing we will clear out the lvm_id as it's likely not unique
+				# and thus not useful and potentially harmful for lookups.
+				if path_create == pv_obj_path_generate and \
+						cfg.db.pv_missing(uuid):
+					lvm_id = None
+
+				# Lets check for the uuid first
+				path = self._id_lookup(uuid)
+				if path:
+					# Verify the lvm_id is sane
+					self._lvm_id_verify(path, uuid, lvm_id)
+				else:
+					# Unable to find by UUID, lets lookup by lvm_id
+					path = self._id_lookup(lvm_id)
+					if path:
+						# Verify the uuid is sane
+						self._uuid_verify(path, uuid, lvm_id)
+					else:
+						# We have exhausted all lookups, let's create if we can
+						if path_create:
+							path = path_create()
+							self._lookup_add(None, path, lvm_id, uuid)
+
+			# print('get_object_path_by_lvm_id(%s, %s, %s, %s: return %s' %
+			# 	   (uuid, lvm_id, str(path_create), str(gen_new), path))
 
 			return path
 

daemons/lvmdbusd/pv.py

@@ -55,9 +55,6 @@ class PvState(State):
 		return self.lvm_path
 
 	def _lv_object_list(self, vg_name):
-
-		# Note we are returning "a(oa(tts))"
-
 		rc = []
 		if vg_name:
 			for lv in sorted(cfg.db.pv_contained_lv(self.lvm_id)):
@@ -69,7 +66,7 @@ class PvState(State):
 					lv_uuid, full_name, path_create)
 
 				rc.append((lv_path, segs))
-		return dbus.Array(rc, signature="(oa(tts))")
+		return rc
 
 	# noinspection PyUnusedLocal,PyPep8Naming
 	def __init__(self, lvm_path, Uuid, Name,
@@ -138,23 +135,30 @@ class Pv(AutomatedProperties):
 	def _remove(pv_uuid, pv_name, remove_options):
 		# Remove the PV, if successful then remove from the model
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(pv_uuid, pv_name)
+		Pv.validate_dbus_object(pv_uuid, pv_name)
+		rc, out, err = cmdhandler.pv_remove(pv_name, remove_options)
+		Pv.handle_execute(rc, out, err)
+		return '/'
 
-		if dbo:
-			rc, out, err = cmdhandler.pv_remove(pv_name, remove_options)
-			if rc == 0:
-				cfg.om.remove_object(dbo, True)
-			else:
-				# Need to work on error handling, need consistent
-				raise dbus.exceptions.DBusException(
-					PV_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
+	@staticmethod
+	def handle_execute(rc, out, err):
+		if rc == 0:
+			cfg.load()
 		else:
+			# Need to work on error handling, need consistent
+			raise dbus.exceptions.DBusException(
+				PV_INTERFACE,
+				'Exit code %s, stderr = %s' % (str(rc), err))
+
+	@staticmethod
+	def validate_dbus_object(pv_uuid, pv_name):
+		dbo = cfg.om.get_object_by_uuid_lvm_id(pv_uuid, pv_name)
+		if not dbo:
 			raise dbus.exceptions.DBusException(
 				PV_INTERFACE,
 				'PV with uuid %s and name %s not present!' %
 				(pv_uuid, pv_name))
-		return '/'
+		return dbo
 
 	@dbus.service.method(
 		dbus_interface=PV_INTERFACE,
@@ -171,22 +175,11 @@ class Pv(AutomatedProperties):
 	@staticmethod
 	def _resize(pv_uuid, pv_name, new_size_bytes, resize_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(pv_uuid, pv_name)
+		Pv.validate_dbus_object(pv_uuid, pv_name)
 
-		if dbo:
-			rc, out, err = cmdhandler.pv_resize(pv_name, new_size_bytes,
+		rc, out, err = cmdhandler.pv_resize(pv_name, new_size_bytes,
 												resize_options)
-			if rc == 0:
-				dbo.refresh()
-			else:
-				raise dbus.exceptions.DBusException(
-					PV_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				PV_INTERFACE,
-				'PV with uuid %s and name %s not present!' %
-				(pv_uuid, pv_name))
+		Pv.handle_execute(rc, out, err)
 		return '/'
 
 	@dbus.service.method(
@@ -204,21 +197,10 @@ class Pv(AutomatedProperties):
 	@staticmethod
 	def _allocation_enabled(pv_uuid, pv_name, yes_no, allocation_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(pv_uuid, pv_name)
-
-		if dbo:
-			rc, out, err = cmdhandler.pv_allocatable(
-				pv_name, yes_no, allocation_options)
-			if rc == 0:
-				cfg.load()
-			else:
-				raise dbus.exceptions.DBusException(
-					PV_INTERFACE, 'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				PV_INTERFACE,
-				'PV with uuid %s and name %s not present!' %
-				(pv_uuid, pv_name))
+		Pv.validate_dbus_object(pv_uuid, pv_name)
+		rc, out, err = cmdhandler.pv_allocatable(
+			pv_name, yes_no, allocation_options)
+		Pv.handle_execute(rc, out, err)
 		return '/'
 
 	@dbus.service.method(
@@ -241,26 +223,20 @@ class Pv(AutomatedProperties):
 	@property
 	def PeSegments(self):
 		if len(self.state.pe_segments):
-			return self.state.pe_segments
+			return dbus.Array(self.state.pe_segments, signature='(tt)')
 		return dbus.Array([], '(tt)')
 
 	@property
 	def Exportable(self):
-		if self.state.attr[1] == 'x':
-			return True
-		return False
+		return dbus.Boolean(self.state.attr[1] == 'x')
 
 	@property
 	def Allocatable(self):
-		if self.state.attr[0] == 'a':
-			return True
-		return False
+		return dbus.Boolean(self.state.attr[0] == 'a')
 
 	@property
 	def Missing(self):
-		if self.state.attr[2] == 'm':
-			return True
-		return False
+		return dbus.Boolean(self.state.attr[2] == 'm')
 
 	def object_path(self):
 		return self._object_path
@@ -275,8 +251,8 @@ class Pv(AutomatedProperties):
 
 	@property
 	def Lv(self):
-		return self.state.lv
+		return dbus.Array(self.state.lv, signature="(oa(tts))")
 
 	@property
 	def Vg(self):
-		return self.state.vg_path
+		return dbus.ObjectPath(self.state.vg_path)

daemons/lvmdbusd/refresh.py

@@ -1,45 +0,0 @@
-# Copyright (C) 2015-2016 Red Hat, Inc. All rights reserved.
-#
-# This copyrighted material is made available to anyone wishing to use,
-# modify, copy, or redistribute it subject to the terms and conditions
-# of the GNU General Public License v.2.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-# Try and minimize the refreshes we do.
-
-import threading
-from .request import RequestEntry
-from . import cfg
-from . import utils
-
-_rlock = threading.RLock()
-_count = 0
-
-
-def handle_external_event(command):
-	utils.log_debug("External event: '%s'" % command)
-	event_complete()
-	cfg.load()
-
-
-def event_add(params):
-	global _rlock
-	global _count
-	with _rlock:
-		if _count == 0:
-			_count += 1
-			r = RequestEntry(
-				-1, handle_external_event,
-				params, None, None, False)
-			cfg.worker_q.put(r)
-
-
-def event_complete():
-	global _rlock
-	global _count
-	with _rlock:
-		if _count > 0:
-			_count -= 1
-		return _count

daemons/lvmdbusd/request.py

@@ -13,13 +13,12 @@ from gi.repository import GLib
 from .job import Job
 from . import cfg
 import traceback
-from .utils import log_error
+from .utils import log_error, mt_async_result
 
 
 class RequestEntry(object):
 	def __init__(self, tmo, method, arguments, cb, cb_error,
 			return_tuple=True, job_state=None):
-		self.tmo = tmo
 		self.method = method
 		self.arguments = arguments
 		self.cb = cb
@@ -35,25 +34,32 @@ class RequestEntry(object):
 		self._return_tuple = return_tuple
 		self._job_state = job_state
 
-		if self.tmo < 0:
+		if tmo < 0:
 			# Client is willing to block forever
 			pass
 		elif tmo == 0:
 			self._return_job()
 		else:
-			self.timer_id = GLib.timeout_add_seconds(
-				tmo, RequestEntry._request_timeout, self)
+			# Note: using 990 instead of 1000 for second to ms conversion to
+			# account for overhead.  Goal is to return just before the
+			# timeout amount has expired.  Better to be a little early than
+			# late.
+			self.timer_id = GLib.timeout_add(
+				tmo * 990, RequestEntry._request_timeout, self)
 
 	@staticmethod
 	def _request_timeout(r):
 		"""
 		Method which gets called when the timer runs out!
 		:param r:  RequestEntry which timed out
-		:return: Nothing
+		:return: Result of timer_expired
 		"""
-		r.timer_expired()
+		return r.timer_expired()
 
 	def _return_job(self):
+		# Return job is only called when we create a request object or when
+		# we pop a timer.  In both cases we are running in the correct context
+		# and do not need to schedule the call back in main context.
 		self._job = Job(self, self._job_state)
 		cfg.om.register_object(self._job, True)
 		if self._return_tuple:
@@ -110,9 +116,9 @@ class RequestEntry(object):
 				if error_rc == 0:
 					if self.cb:
 						if self._return_tuple:
-							self.cb((result, '/'))
+							mt_async_result(self.cb, (result, '/'))
 						else:
-							self.cb(result)
+							mt_async_result(self.cb, result)
 				else:
 					if self.cb_error:
 						if not error_exception:
@@ -123,15 +129,14 @@ class RequestEntry(object):
 							else:
 								error_exception = Exception(error_msg)
 
-						self.cb_error(error_exception)
+						mt_async_result(self.cb_error, error_exception)
 			else:
 				# We have a job and it's complete, indicate that it's done.
-				# TODO: We need to signal the job is done too.
 				self._job.Complete = True
 				self._job = None
 
 	def register_error(self, error_rc, error_message, error_exception):
-		self._reg_ending(None, error_rc, error_message, error_exception)
+		self._reg_ending('/', error_rc, error_message, error_exception)
 
 	def register_result(self, result):
 		self._reg_ending(result)

daemons/lvmdbusd/udevwatch.py

@@ -8,10 +8,11 @@
 # along with this program. If not, see <http://www.gnu.org/licenses/>.
 
 import pyudev
-from .refresh import event_add
+import threading
 from . import cfg
 
 observer = None
+observer_lock = threading.RLock()
 
 
 # noinspection PyUnusedLocal
@@ -36,19 +37,24 @@ def filter_event(action, device):
 		refresh = True
 
 	if refresh:
-		event_add(('udev',))
+		cfg.event()
 
 
 def add():
-	global observer
-	context = pyudev.Context()
-	monitor = pyudev.Monitor.from_netlink(context)
-	monitor.filter_by('block')
-	observer = pyudev.MonitorObserver(monitor, filter_event)
-	observer.start()
+	with observer_lock:
+		global observer
+		context = pyudev.Context()
+		monitor = pyudev.Monitor.from_netlink(context)
+		monitor.filter_by('block')
+		observer = pyudev.MonitorObserver(monitor, filter_event)
+		observer.start()
 
 
 def remove():
-	global observer
-	observer.stop()
-	observer = None
+	with observer_lock:
+		global observer
+		if observer:
+			observer.stop()
+			observer = None
+			return True
+		return False

daemons/lvmdbusd/utils.py

@@ -13,15 +13,14 @@ import inspect
 import ctypes
 import os
 import string
+import datetime
 
 import dbus
-import dbus.service
-import dbus.mainloop.glib
+from lvmdbusd import cfg
+# noinspection PyUnresolvedReferences
+from gi.repository import GLib
+import threading
 
-try:
-	from . import cfg
-except SystemError:
-	import cfg
 
 STDOUT_TTY = os.isatty(sys.stdout.fileno())
 
@@ -149,17 +148,27 @@ def add_properties(xml, interface, props):
 	:param props:       Output from get_properties
 	:return: updated XML string
 	"""
-	root = Et.fromstring(xml)
-
 	if props:
+		root = Et.fromstring(xml)
+		interface_element = None
 
+		# Check to see if interface is present
 		for c in root:
-			# print c.attrib['name']
 			if c.attrib['name'] == interface:
-				for p in props:
-					temp = '<property type="%s" name="%s" access="%s"/>\n' % \
-						(p['p_t'], p['p_name'], p['p_access'])
-					c.append(Et.fromstring(temp))
+				interface_element = c
+				break
+
+		# Interface is not present, lets create it so we have something to
+		# attach the properties too
+		if interface_element is None:
+			interface_element = Et.Element("interface", name=interface)
+			root.append(interface_element)
+
+		# Add the properties
+		for p in props:
+			temp = '<property type="%s" name="%s" access="%s"/>\n' % \
+				(p['p_t'], p['p_name'], p['p_access'])
+			interface_element.append(Et.fromstring(temp))
 
 		return Et.tostring(root, encoding='utf8')
 	return xml
@@ -235,7 +244,7 @@ def parse_tags(tags):
 	if len(tags):
 		if ',' in tags:
 			return tags.split(',')
-		return sorted([tags])
+		return dbus.Array(sorted([tags]), signature='s')
 	return dbus.Array([], signature='s')
 
 
@@ -243,7 +252,13 @@ def _common_log(msg, *attributes):
 	cfg.stdout_lock.acquire()
 	tid = ctypes.CDLL('libc.so.6').syscall(186)
 
-	msg = "%d:%d - %s" % (os.getpid(), tid, msg)
+	if STDOUT_TTY:
+		msg = "%s: %d:%d - %s" % \
+			(datetime.datetime.now().strftime("%b %d %H:%M:%S.%f"),
+			os.getpid(), tid, msg)
+
+	else:
+		msg = "%d:%d - %s" % (os.getpid(), tid, msg)
 
 	if STDOUT_TTY and attributes:
 		print(color(msg, *attributes))
@@ -258,7 +273,7 @@ def _common_log(msg, *attributes):
 # @param msg    Message to output to stdout
 # @return None
 def log_debug(msg, *attributes):
-	if cfg.DEBUG:
+	if cfg.args and cfg.args.debug:
 		_common_log(msg, *attributes)
 
 
@@ -482,3 +497,71 @@ def validate_tag(interface, tag):
 		raise dbus.exceptions.DBusException(
 			interface, 'tag (%s) contains invalid character, allowable set(%s)'
 			% (tag, _ALLOWABLE_TAG_CH))
+
+
+# The methods below which start with mt_* are used to execute the desired code
+# on the the main thread of execution to alleviate any issues the dbus-python
+# library with regards to multi-threaded access.  Essentially, we are trying to
+# ensure all dbus library interaction is done from the same thread!
+
+
+def _async_result(call_back, results):
+	log_debug('Results = %s' % str(results))
+	call_back(results)
+
+
+# Return result in main thread
+def mt_async_result(call_back, results):
+	GLib.idle_add(_async_result, call_back, results)
+
+
+# Take the supplied function and run it on the main thread and not wait for
+# a result!
+def mt_run_no_wait(function, param):
+	GLib.idle_add(function, param)
+
+# Run the supplied function and arguments on the main thread and wait for them
+# to complete while allowing the ability to get the return value too.
+#
+# Example:
+# result = MThreadRunner(foo, arg1, arg2).done()
+#
+class MThreadRunner(object):
+
+	@staticmethod
+	def runner(obj):
+		# noinspection PyProtectedMember
+		obj._run()
+		with obj.cond:
+			obj.function_complete = True
+			obj.cond.notify_all()
+
+	def __init__(self, function, *args):
+		self.f = function
+		self.rc = None
+		self.args = args
+		self.function_complete = False
+		self.cond = threading.Condition(threading.Lock())
+
+	def done(self):
+		GLib.idle_add(MThreadRunner.runner, self)
+		with self.cond:
+			if not self.function_complete:
+				self.cond.wait()
+		return self.rc
+
+	def _run(self):
+		if len(self.args):
+			self.rc = self.f(*self.args)
+		else:
+			self.rc = self.f()
+
+
+def _remove_objects(dbus_objects_rm):
+	for o in dbus_objects_rm:
+		cfg.om.remove_object(o, emit_signal=True)
+
+
+# Remove dbus objects from main thread
+def mt_remove_dbus_objects(objs):
+	MThreadRunner(_remove_objects, objs).done()

daemons/lvmdbusd/vg.py

@@ -19,7 +19,7 @@ from .request import RequestEntry
 from .loader import common
 from .state import State
 from . import background
-from .utils import round_size
+from .utils import round_size, mt_remove_dbus_objects
 from .job import JobState
 
 
@@ -78,7 +78,7 @@ class VgState(State):
 			(pv_name, pv_uuid) = p
 			rc.append(cfg.om.get_object_path_by_uuid_lvm_id(
 				pv_uuid, pv_name, pv_obj_path_generate))
-		return dbus.Array(rc, signature='o')
+		return rc
 
 	def __init__(self, Uuid, Name, Fmt,
 			SizeBytes, FreeBytes, SysId, ExtentSizeBytes,
@@ -145,29 +145,35 @@ class Vg(AutomatedProperties):
 
 	@staticmethod
 	def fetch_new_lv(vg_name, lv_name):
-		cfg.load()
-		return cfg.om.get_object_by_lvm_id("%s/%s" % (vg_name, lv_name))
+		return cfg.om.get_object_path_by_lvm_id("%s/%s" % (vg_name, lv_name))
+
+	@staticmethod
+	def handle_execute(rc, out, err):
+		if rc == 0:
+			cfg.load()
+		else:
+			# Need to work on error handling, need consistent
+			raise dbus.exceptions.DBusException(
+				VG_INTERFACE,
+				'Exit code %s, stderr = %s' % (str(rc), err))
+
+	@staticmethod
+	def validate_dbus_object(vg_uuid, vg_name):
+		dbo = cfg.om.get_object_by_uuid_lvm_id(vg_uuid, vg_name)
+		if not dbo:
+			raise dbus.exceptions.DBusException(
+				VG_INTERFACE,
+				'VG with uuid %s and name %s not present!' %
+				(vg_uuid, vg_name))
+		return dbo
 
 	@staticmethod
 	def _rename(uuid, vg_name, new_name, rename_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
-
-		if dbo:
-			rc, out, err = cmdhandler.vg_rename(vg_name, new_name,
-												rename_options)
-			if rc == 0:
-				cfg.load()
-			else:
-				# Need to work on error handling, need consistent
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
+		Vg.validate_dbus_object(uuid, vg_name)
+		rc, out, err = cmdhandler.vg_rename(
+			vg_name, new_name, rename_options)
+		Vg.handle_execute(rc, out, err)
 		return '/'
 
 	@dbus.service.method(
@@ -184,31 +190,10 @@ class Vg(AutomatedProperties):
 	@staticmethod
 	def _remove(uuid, vg_name, remove_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
-
-		if dbo:
-			# Remove the VG, if successful then remove from the model
-			rc, out, err = cmdhandler.vg_remove(vg_name, remove_options)
-
-			if rc == 0:
-				# Remove the VG
-				cfg.om.remove_object(dbo, True)
-
-				# If an LV has hidden LVs, things can get quite involved,
-				# especially if it's the last thin pool to get removed, so
-				# lets refresh all
-				cfg.load()
-
-			else:
-				# Need to work on error handling, need consistent
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
+		Vg.validate_dbus_object(uuid, vg_name)
+		# Remove the VG, if successful then remove from the model
+		rc, out, err = cmdhandler.vg_remove(vg_name, remove_options)
+		Vg.handle_execute(rc, out, err)
 		return '/'
 
 	@dbus.service.method(
@@ -223,26 +208,9 @@ class Vg(AutomatedProperties):
 
 	@staticmethod
 	def _change(uuid, vg_name, change_options):
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
-
-		if dbo:
-			rc, out, err = cmdhandler.vg_change(change_options, vg_name)
-
-			# To use an example with d-feet (Method input)
-			# {"activate": __import__('gi.repository.GLib', globals(),
-			# locals(), ['Variant']).Variant("s", "n")}
-
-			if rc == 0:
-				cfg.load()
-			else:
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
+		Vg.validate_dbus_object(uuid, vg_name)
+		rc, out, err = cmdhandler.vg_change(change_options, vg_name)
+		Vg.handle_execute(rc, out, err)
 		return '/'
 
 	# TODO: This should be broken into a number of different methods
@@ -263,34 +231,24 @@ class Vg(AutomatedProperties):
 	@staticmethod
 	def _reduce(uuid, vg_name, missing, pv_object_paths, reduce_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
+		Vg.validate_dbus_object(uuid, vg_name)
 
-		if dbo:
-			pv_devices = []
+		pv_devices = []
 
-			# If pv_object_paths is not empty, then get the device paths
-			if pv_object_paths and len(pv_object_paths) > 0:
-				for pv_op in pv_object_paths:
-					pv = cfg.om.get_object_by_path(pv_op)
-					if pv:
-						pv_devices.append(pv.lvm_id)
-					else:
-						raise dbus.exceptions.DBusException(
-							VG_INTERFACE,
-							'PV Object path not found = %s!' % pv_op)
+		# If pv_object_paths is not empty, then get the device paths
+		if pv_object_paths and len(pv_object_paths) > 0:
+			for pv_op in pv_object_paths:
+				pv = cfg.om.get_object_by_path(pv_op)
+				if pv:
+					pv_devices.append(pv.lvm_id)
+				else:
+					raise dbus.exceptions.DBusException(
+						VG_INTERFACE,
+						'PV Object path not found = %s!' % pv_op)
 
-			rc, out, err = cmdhandler.vg_reduce(vg_name, missing, pv_devices,
-												reduce_options)
-			if rc == 0:
-				cfg.load()
-			else:
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE, 'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
+		rc, out, err = cmdhandler.vg_reduce(vg_name, missing, pv_devices,
+											reduce_options)
+		Vg.handle_execute(rc, out, err)
 		return '/'
 
 	@dbus.service.method(
@@ -307,36 +265,26 @@ class Vg(AutomatedProperties):
 	@staticmethod
 	def _extend(uuid, vg_name, pv_object_paths, extend_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
+		Vg.validate_dbus_object(uuid, vg_name)
 
-		if dbo:
-			extend_devices = []
+		extend_devices = []
 
-			for i in pv_object_paths:
-				pv = cfg.om.get_object_by_path(i)
-				if pv:
-					extend_devices.append(pv.lvm_id)
-				else:
-					raise dbus.exceptions.DBusException(
-						VG_INTERFACE, 'PV Object path not found = %s!' % i)
-
-			if len(extend_devices):
-				rc, out, err = cmdhandler.vg_extend(vg_name, extend_devices,
-													extend_options)
-				if rc == 0:
-					cfg.load()
-				else:
-					raise dbus.exceptions.DBusException(
-						VG_INTERFACE,
-						'Exit code %s, stderr = %s' % (str(rc), err))
+		for i in pv_object_paths:
+			pv = cfg.om.get_object_by_path(i)
+			if pv:
+				extend_devices.append(pv.lvm_id)
 			else:
 				raise dbus.exceptions.DBusException(
-					VG_INTERFACE, 'No pv_object_paths provided!')
+					VG_INTERFACE, 'PV Object path not found = %s!' % i)
+
+		if len(extend_devices):
+			rc, out, err = cmdhandler.vg_extend(vg_name, extend_devices,
+												extend_options)
+			Vg.handle_execute(rc, out, err)
 		else:
 			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
+				VG_INTERFACE, 'No pv_object_paths provided!')
+
 		return '/'
 
 	@dbus.service.method(
@@ -373,33 +321,24 @@ class Vg(AutomatedProperties):
 			create_options):
 		# Make sure we have a dbus object representing it
 		pv_dests = []
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
 
-		if dbo:
-			if len(pv_dests_and_ranges):
-				for pr in pv_dests_and_ranges:
-					pv_dbus_obj = cfg.om.get_object_by_path(pr[0])
-					if not pv_dbus_obj:
-						raise dbus.exceptions.DBusException(
-							VG_INTERFACE,
-							'PV Destination (%s) not found' % pr[0])
+		Vg.validate_dbus_object(uuid, vg_name)
 
-					pv_dests.append((pv_dbus_obj.lvm_id, pr[1], pr[2]))
+		if len(pv_dests_and_ranges):
+			for pr in pv_dests_and_ranges:
+				pv_dbus_obj = cfg.om.get_object_by_path(pr[0])
+				if not pv_dbus_obj:
+					raise dbus.exceptions.DBusException(
+						VG_INTERFACE,
+						'PV Destination (%s) not found' % pr[0])
 
-			rc, out, err = cmdhandler.vg_lv_create(
-				vg_name, create_options, name, size_bytes, pv_dests)
+				pv_dests.append((pv_dbus_obj.lvm_id, pr[1], pr[2]))
 
-			if rc == 0:
-				return Vg.fetch_new_lv(vg_name, name)
-			else:
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
+		rc, out, err = cmdhandler.vg_lv_create(
+			vg_name, create_options, name, size_bytes, pv_dests)
+
+		Vg.handle_execute(rc, out, err)
+		return Vg.fetch_new_lv(vg_name, name)
 
 	@dbus.service.method(
 		dbus_interface=VG_INTERFACE,
@@ -435,25 +374,13 @@ class Vg(AutomatedProperties):
 	def _lv_create_linear(uuid, vg_name, name, size_bytes,
 			thin_pool, create_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
+		Vg.validate_dbus_object(uuid, vg_name)
 
-		if dbo:
-			rc, out, err = cmdhandler.vg_lv_create_linear(
-				vg_name, create_options, name, size_bytes, thin_pool)
+		rc, out, err = cmdhandler.vg_lv_create_linear(
+			vg_name, create_options, name, size_bytes, thin_pool)
 
-			if rc == 0:
-				created_lv = Vg.fetch_new_lv(vg_name, name)
-			else:
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
-
-		return created_lv
+		Vg.handle_execute(rc, out, err)
+		return Vg.fetch_new_lv(vg_name, name)
 
 	@dbus.service.method(
 		dbus_interface=VG_INTERFACE,
@@ -473,24 +400,12 @@ class Vg(AutomatedProperties):
 	def _lv_create_striped(uuid, vg_name, name, size_bytes, num_stripes,
 			stripe_size_kb, thin_pool, create_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
-
-		if dbo:
-			rc, out, err = cmdhandler.vg_lv_create_striped(
-				vg_name, create_options, name, size_bytes,
-				num_stripes, stripe_size_kb, thin_pool)
-			if rc == 0:
-				created_lv = Vg.fetch_new_lv(vg_name, name)
-			else:
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE, 'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
-
-		return created_lv
+		Vg.validate_dbus_object(uuid, vg_name)
+		rc, out, err = cmdhandler.vg_lv_create_striped(
+			vg_name, create_options, name, size_bytes,
+			num_stripes, stripe_size_kb, thin_pool)
+		Vg.handle_execute(rc, out, err)
+		return Vg.fetch_new_lv(vg_name, name)
 
 	@dbus.service.method(
 		dbus_interface=VG_INTERFACE,
@@ -513,25 +428,11 @@ class Vg(AutomatedProperties):
 	def _lv_create_mirror(uuid, vg_name, name, size_bytes,
 			num_copies, create_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
-
-		if dbo:
-			rc, out, err = cmdhandler.vg_lv_create_mirror(
-				vg_name, create_options, name, size_bytes, num_copies)
-			if rc == 0:
-				created_lv = Vg.fetch_new_lv(vg_name, name)
-			else:
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
-
-		return created_lv
+		Vg.validate_dbus_object(uuid, vg_name)
+		rc, out, err = cmdhandler.vg_lv_create_mirror(
+			vg_name, create_options, name, size_bytes, num_copies)
+		Vg.handle_execute(rc, out, err)
+		return Vg.fetch_new_lv(vg_name, name)
 
 	@dbus.service.method(
 		dbus_interface=VG_INTERFACE,
@@ -552,26 +453,12 @@ class Vg(AutomatedProperties):
 	def _lv_create_raid(uuid, vg_name, name, raid_type, size_bytes,
 						num_stripes, stripe_size_kb, create_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
-
-		if dbo:
-			rc, out, err = cmdhandler.vg_lv_create_raid(
-				vg_name, create_options, name, raid_type, size_bytes,
-				num_stripes, stripe_size_kb)
-			if rc == 0:
-				created_lv = Vg.fetch_new_lv(vg_name, name)
-			else:
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
-
-		return created_lv
+		Vg.validate_dbus_object(uuid, vg_name)
+		rc, out, err = cmdhandler.vg_lv_create_raid(
+			vg_name, create_options, name, raid_type, size_bytes,
+			num_stripes, stripe_size_kb)
+		Vg.handle_execute(rc, out, err)
+		return Vg.fetch_new_lv(vg_name, name)
 
 	@dbus.service.method(
 		dbus_interface=VG_INTERFACE,
@@ -592,35 +479,27 @@ class Vg(AutomatedProperties):
 	def _create_pool(uuid, vg_name, meta_data_lv, data_lv,
 						create_options, create_method):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
+		Vg.validate_dbus_object(uuid, vg_name)
 
 		# Retrieve the full names for the metadata and data lv
 		md = cfg.om.get_object_by_path(meta_data_lv)
 		data = cfg.om.get_object_by_path(data_lv)
 
-		if dbo and md and data:
+		if md and data:
 
 			new_name = data.Name
 
 			rc, out, err = create_method(
 				md.lv_full_name(), data.lv_full_name(), create_options)
-			if rc == 0:
-				cfg.om.remove_object(md, emit_signal=True)
-				cfg.om.remove_object(data, emit_signal=True)
 
-				cache_pool_lv = Vg.fetch_new_lv(vg_name, new_name)
-			else:
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
+			if rc == 0:
+				mt_remove_dbus_objects((md, data))
+
+			Vg.handle_execute(rc, out, err)
 
 		else:
 			msg = ""
 
-			if not dbo:
-				msg += 'VG with uuid %s and name %s not present!' % \
-					(uuid, vg_name)
-
 			if not md:
 				msg += 'Meta data LV with object path %s not present!' % \
 					(meta_data_lv)
@@ -631,7 +510,7 @@ class Vg(AutomatedProperties):
 
 			raise dbus.exceptions.DBusException(VG_INTERFACE, msg)
 
-		return cache_pool_lv
+		return Vg.fetch_new_lv(vg_name, new_name)
 
 	@dbus.service.method(
 		dbus_interface=VG_INTERFACE,
@@ -665,33 +544,21 @@ class Vg(AutomatedProperties):
 		pv_devices = []
 
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
+		Vg.validate_dbus_object(uuid, vg_name)
 
-		if dbo:
-			# Check for existence of pv object paths
-			for p in pv_object_paths:
-				pv = cfg.om.get_object_by_path(p)
-				if pv:
-					pv_devices.append(pv.Name)
-				else:
-					raise dbus.exceptions.DBusException(
-						VG_INTERFACE, 'PV object path = %s not found' % p)
-
-			rc, out, err = cmdhandler.pv_tag(
-				pv_devices, tags_add, tags_del, tag_options)
-			if rc == 0:
-				cfg.load()
-				return '/'
+		# Check for existence of pv object paths
+		for p in pv_object_paths:
+			pv = cfg.om.get_object_by_path(p)
+			if pv:
+				pv_devices.append(pv.Name)
 			else:
 				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
+					VG_INTERFACE, 'PV object path = %s not found' % p)
 
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
+		rc, out, err = cmdhandler.pv_tag(
+			pv_devices, tags_add, tags_del, tag_options)
+		Vg.handle_execute(rc, out, err)
+		return '/'
 
 	@dbus.service.method(
 		dbus_interface=VG_INTERFACE,
@@ -729,25 +596,12 @@ class Vg(AutomatedProperties):
 	@staticmethod
 	def _vg_add_rm_tags(uuid, vg_name, tags_add, tags_del, tag_options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
+		Vg.validate_dbus_object(uuid, vg_name)
 
-		if dbo:
-
-			rc, out, err = cmdhandler.vg_tag(
-				vg_name, tags_add, tags_del, tag_options)
-			if rc == 0:
-				dbo.refresh()
-				return '/'
-			else:
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
+		rc, out, err = cmdhandler.vg_tag(
+			vg_name, tags_add, tags_del, tag_options)
+		Vg.handle_execute(rc, out, err)
+		return '/'
 
 	@dbus.service.method(
 		dbus_interface=VG_INTERFACE,
@@ -784,23 +638,10 @@ class Vg(AutomatedProperties):
 	@staticmethod
 	def _vg_change_set(uuid, vg_name, method, value, options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
-
-		if dbo:
-			rc, out, err = method(vg_name, value, options)
-			if rc == 0:
-				dbo.refresh()
-				return '/'
-			else:
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
+		Vg.validate_dbus_object(uuid, vg_name)
+		rc, out, err = method(vg_name, value, options)
+		Vg.handle_execute(rc, out, err)
+		return '/'
 
 	@dbus.service.method(
 		dbus_interface=VG_INTERFACE,
@@ -840,9 +681,7 @@ class Vg(AutomatedProperties):
 		cfg.worker_q.put(r)
 
 	def _attribute(self, pos, ch):
-		if self.state.attr[pos] == ch:
-			return True
-		return False
+		return dbus.Boolean(self.state.attr[pos] == ch)
 
 	@dbus.service.method(
 		dbus_interface=VG_INTERFACE,
@@ -860,23 +699,11 @@ class Vg(AutomatedProperties):
 	def _vg_activate_deactivate(uuid, vg_name, activate, control_flags,
 								options):
 		# Make sure we have a dbus object representing it
-		dbo = cfg.om.get_object_by_uuid_lvm_id(uuid, vg_name)
-
-		if dbo:
-			rc, out, err = cmdhandler.activate_deactivate(
-				'vgchange', vg_name, activate, control_flags, options)
-			if rc == 0:
-				cfg.load()
-				return '/'
-			else:
-				raise dbus.exceptions.DBusException(
-					VG_INTERFACE,
-					'Exit code %s, stderr = %s' % (str(rc), err))
-		else:
-			raise dbus.exceptions.DBusException(
-				VG_INTERFACE,
-				'VG with uuid %s and name %s not present!' %
-				(uuid, vg_name))
+		Vg.validate_dbus_object(uuid, vg_name)
+		rc, out, err = cmdhandler.activate_deactivate(
+			'vgchange', vg_name, activate, control_flags, options)
+		Vg.handle_execute(rc, out, err)
+		return '/'
 
 	@dbus.service.method(
 		dbus_interface=VG_INTERFACE,
@@ -908,11 +735,11 @@ class Vg(AutomatedProperties):
 
 	@property
 	def Pvs(self):
-		return self.state.Pvs
+		return dbus.Array(self.state.Pvs, signature='o')
 
 	@property
 	def Lvs(self):
-		return self.state.Lvs
+		return dbus.Array(self.state.Lvs, signature='o')
 
 	@property
 	def lvm_id(self):

daemons/lvmetad/lvmetad-core.c

@@ -599,7 +599,7 @@ static void mark_outdated_pv(lvmetad_state *s, const char *vgid, const char *pvi
 
 	outdated_pvs = dm_hash_lookup(s->vgid_to_outdated_pvs, vgid);
 	if (!outdated_pvs) {
-		if (!(outdated_pvs = dm_config_from_string("outdated_pvs/pv_list = []")) ||
+		if (!(outdated_pvs = config_tree_from_string_without_dup_node_check("outdated_pvs/pv_list = []")) ||
 		    !(cft_vgid = make_text_node(outdated_pvs, "vgid", dm_pool_strdup(outdated_pvs->mem, vgid),
 						outdated_pvs->root, NULL)))
 			abort();
@@ -1249,8 +1249,8 @@ static int _update_metadata(lvmetad_state *s, const char *arg_name, const char *
 	const char *old_vgid = NULL;
 	const char *new_vgid = NULL;
 	const char *new_metadata_vgid;
+	int new_seq;
 	int old_seq = -1;
-	int new_seq = -1;
 	int needs_repair = 0;
 	int abort_daemon = 0;
 	int retval = 0;
@@ -2745,7 +2745,8 @@ static response handler(daemon_state s, client_handle h, request r)
 							   "expected = %s", state->token,
 							   "received = %s", token,
 							   "update_pid = " FMTd64, (int64_t)state->update_pid,
-							   "reason = %s", "another command has populated the cache");
+							   "reason = %s", "another command has populated the cache",
+							   NULL);
 			}
 
 			DEBUGLOG(state, "token_update end len %d pid %d new token %s",
@@ -2778,7 +2779,8 @@ static response handler(daemon_state s, client_handle h, request r)
 					   "expected = %s", state->token,
 					   "received = %s", token,
 					   "update_pid = " FMTd64, (int64_t)state->update_pid,
-					   "reason = %s", "another command has populated the cache");
+					   "reason = %s", "another command has populated the cache",
+					   NULL);
 	}
 
 	/* If a pid doing update was cancelled, ignore its update messages. */
@@ -2793,7 +2795,8 @@ static response handler(daemon_state s, client_handle h, request r)
 					   "expected = %s", state->token,
 					   "received = %s", token,
 					   "update_pid = " FMTd64, (int64_t)state->update_pid,
-					   "reason = %s", "another command has populated the lvmetad cache");
+					   "reason = %s", "another command has populated the lvmetad cache",
+					   NULL);
 	}
 
 	pthread_mutex_unlock(&state->token_lock);

daemons/lvmlockd/lvmlockd-core.c

@@ -4431,7 +4431,7 @@ static void client_recv_action(struct client *cl)
 		return;
 	}
 
-	req.cft = dm_config_from_string(req.buffer.mem);
+	req.cft = config_tree_from_string_without_dup_node_check(req.buffer.mem);
 	if (!req.cft) {
 		log_error("client recv %u config_from_string error", cl->id);
 		buffer_destroy(&req.buffer);

daemons/lvmpolld/lvmpolld-cmd-utils.c

@@ -19,10 +19,12 @@
 
 #define MIN_ARGV_SIZE  8
 
-static const char *const const polling_ops[] = { [PVMOVE] = LVMPD_REQ_PVMOVE,
-						 [CONVERT] = LVMPD_REQ_CONVERT,
-						 [MERGE] = LVMPD_REQ_MERGE,
-						 [MERGE_THIN] = LVMPD_REQ_MERGE_THIN };
+static const char *const polling_ops[] = {
+	[PVMOVE] = LVMPD_REQ_PVMOVE,
+	[CONVERT] = LVMPD_REQ_CONVERT,
+	[MERGE] = LVMPD_REQ_MERGE,
+	[MERGE_THIN] = LVMPD_REQ_MERGE_THIN
+};
 
 const char *polling_op(enum poll_type type)
 {

daemons/lvmpolld/lvmpolld-data-utils.c

@@ -19,6 +19,8 @@
 #include <fcntl.h>
 #include <signal.h>
 
+static const char LVM_SYSTEM_DIR[] = "LVM_SYSTEM_DIR=";
+
 static char *_construct_full_lvname(const char *vgname, const char *lvname)
 {
 	char *name;
@@ -52,7 +54,7 @@ static char *_construct_lvm_system_dir_env(const char *sysdir)
 
 	*env = '\0';
 
-	if (sysdir && dm_snprintf(env, l, "LVM_SYSTEM_DIR=%s", sysdir) < 0) {
+	if (sysdir && dm_snprintf(env, l, "%s%s", LVM_SYSTEM_DIR, sysdir) < 0) {
 		dm_free(env);
 		env = NULL;
 	}
@@ -259,8 +261,8 @@ static void _pdlv_locked_dump(struct buffer *buff, const struct lvmpolld_lv *pdl
 		buffer_append(buff, tmp);
 	if (dm_snprintf(tmp, sizeof(tmp), "\t\tlvm_command_interval=\"%s\"\n", pdlv->sinterval ?: "<undefined>") > 0)
 		buffer_append(buff, tmp);
-	if (dm_snprintf(tmp, sizeof(tmp), "\t\tLVM_SYSTEM_DIR=\"%s\"\n",
-			(*pdlv->lvm_system_dir_env ? (pdlv->lvm_system_dir_env + strlen("LVM_SYSTEM_DIR=")) : "<undefined>")) > 0)
+	if (dm_snprintf(tmp, sizeof(tmp), "\t\t%s\"%s\"\n", LVM_SYSTEM_DIR,
+			(*pdlv->lvm_system_dir_env ? (pdlv->lvm_system_dir_env + (sizeof(LVM_SYSTEM_DIR) - 1)) : "<undefined>")) > 0)
 		buffer_append(buff, tmp);
 	if (dm_snprintf(tmp, sizeof(tmp), "\t\tlvm_command_pid=%d\n", pdlv->cmd_pid) > 0)
 		buffer_append(buff, tmp);

doc/license.txt

@@ -0,0 +1,14 @@
+/*
+ * Copyright (C) 2001-2004 Sistina Software, Inc. All rights reserved.  
+ * Copyright (C) 2004-2017 Red Hat, Inc. All rights reserved.
+ *
+ * This file is part of LVM2.
+ *
+ * This copyrighted material is made available to anyone wishing to use,
+ * modify, copy, or redistribute it subject to the terms and conditions
+ * of the GNU Lesser General Public License v.2.1.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */

include/.symlinks.in

@@ -50,6 +50,7 @@
 @top_srcdir@/lib/misc/lvm-file.h
 @top_srcdir@/lib/misc/lvm-flock.h
 @top_srcdir@/lib/misc/lvm-globals.h
+@top_srcdir@/lib/misc/lvm-maths.h
 @top_srcdir@/lib/misc/lvm-percent.h
 @top_srcdir@/lib/misc/lvm-signal.h
 @top_srcdir@/lib/misc/lvm-string.h

include/configure.h.in

@@ -341,6 +341,9 @@
 /* Define to 1 if the system has the type `ptrdiff_t'. */
 #undef HAVE_PTRDIFF_T
 
+/* Define to 1 if the compiler has the `__builtin_clz` builtin. */
+#undef HAVE___BUILTIN_CLZ
+
 /* Define to 1 if you have the <readline/history.h> header file. */
 #undef HAVE_READLINE_HISTORY_H
 

lib/Makefile.in

@@ -112,6 +112,7 @@ SOURCES =\
 	misc/lvm-file.c \
 	misc/lvm-flock.c \
 	misc/lvm-globals.c \
+	misc/lvm-maths.c \
 	misc/lvm-signal.c \
 	misc/lvm-string.c \
 	misc/lvm-wrappers.c \

lib/activate/activate.c

@@ -358,6 +358,10 @@ int lv_mknodes(struct cmd_context *cmd, const struct logical_volume *lv)
 {
 	return 1;
 }
+int lv_deactivate_any_missing_subdevs(const struct logical_volume *lv)
+{
+	return 1;
+}
 int pv_uses_vg(struct physical_volume *pv,
 	       struct volume_group *vg)
 {
@@ -370,6 +374,11 @@ void activation_exit(void)
 {
 }
 
+int raid4_is_supported(struct cmd_context *cmd, const struct segment_type *segtype)
+{
+	return 1;
+}
+
 int lv_is_active(const struct logical_volume *lv)
 {
 	return 0;
@@ -650,6 +659,10 @@ int target_present(struct cmd_context *cmd, const char *target_name,
 				      &maj, &min, &patchlevel);
 }
 
+/*
+ * When '*info' is NULL, returns 1 only when LV is active.
+ * When '*info' != NULL, returns 1 when info structure is populated.
+ */
 static int _lv_info(struct cmd_context *cmd, const struct logical_volume *lv,
 		    int use_layer, struct lvinfo *info,
 		    const struct lv_segment *seg,
@@ -683,32 +696,6 @@ static int _lv_info(struct cmd_context *cmd, const struct logical_volume *lv,
 	if (seg_status) {
 		/* TODO: for now it's mess with seg_status */
 		seg_status->seg = seg;
-		if (lv_is_merging_cow(lv)) {
-			if (lv_has_target_type(cmd->mem, origin_from_cow(lv), NULL, TARGET_NAME_SNAPSHOT_MERGE)) {
-				/*
-				 * When the snapshot-merge has not yet started, query COW LVs as is.
-				 * When merge is in progress, query merging origin LV instead.
-				 * COW volume is already mapped as error target in this case.
-				 */
-				lv = origin_from_cow(lv);
-				seg_status->seg = first_seg(lv);
-				log_debug_activation("Snapshot merge is in progress, querying status of %s instead.",
-						     display_lvname(lv));
-			}
-		} else if (!use_layer && lv_is_origin(lv) && !lv_is_external_origin(lv)) {
-			/*
-			 * Query status for 'layered' (-real) device most of the time,
-			 * only when snapshot merge started, query its progress.
-			 * TODO: single LV may need couple status to be exposed at once....
-			 *       but this needs more logical background
-			 */
-			if (!lv_is_merging_origin(lv) ||
-			    !lv_has_target_type(cmd->mem, origin_from_cow(lv), NULL, TARGET_NAME_SNAPSHOT_MERGE))
-				use_layer = 1;
-		} else if (lv_is_cow(lv)) {
-			/* Hadle fictional lvm2 snapshot and query snapshotX volume */
-			seg_status->seg = find_snapshot(lv);
-		}
 	}
 
 	if (!dev_manager_info(cmd, lv,
@@ -762,44 +749,96 @@ int lv_info_by_lvid(struct cmd_context *cmd, const char *lvid_s, int use_layer,
 }
 
 /*
- * Returns 1 if lv_seg_status structure populated,
- * else 0 on failure or if device not active locally.
- */
-int lv_status(struct cmd_context *cmd, const struct lv_segment *lv_seg,
-	      int use_layer, struct lv_seg_status *lv_seg_status)
-{
-	if (!activation())
-		return 0;
-
-	return _lv_info(cmd, lv_seg->lv, use_layer, NULL, lv_seg, lv_seg_status, 0, 0);
-}
-
-/*
- * Returns 1 if lv_with_info_and_seg_status structure populated,
+ * Returns 1 if lv_with_info_and_seg_status info structure populated,
  * else 0 on failure or if device not active locally.
  *
- * This is the same as calling lv_info and lv_status,
- * but* it's done in one go with one ioctl if possible!                                                             ]
+ * When seg_status parsing had troubles it will set type to SEG_STATUS_UNKNOWN.
+ *
+ * Using usually one ioctl to obtain info and status.
+ * More complex segment do collect info from one device,
+ * but status from another device.
+ *
+ * TODO: further improve with more statuses (i.e. snapshot's origin/merge)
  */
-int lv_info_with_seg_status(struct cmd_context *cmd, const struct logical_volume *lv,
-			    const struct lv_segment *lv_seg, int use_layer,
+int lv_info_with_seg_status(struct cmd_context *cmd,
+			    const struct lv_segment *lv_seg,
 			    struct lv_with_info_and_seg_status *status,
 			    int with_open_count, int with_read_ahead)
 {
+	const struct logical_volume *olv, *lv = status->lv = lv_seg->lv;
+
 	if (!activation())
 		return 0;
 
-	if (lv == lv_seg->lv)
-		return _lv_info(cmd, lv, use_layer, &status->info, lv_seg, &status->seg_status,
-				with_open_count, with_read_ahead);
+	if (lv_is_used_cache_pool(lv)) {
+		/* INFO is not set as cache-pool cannot be active.
+		 * STATUS is collected from cache LV */
+		lv_seg = get_only_segment_using_this_lv(lv);
+		(void) _lv_info(cmd, lv_seg->lv, 1, NULL, lv_seg, &status->seg_status, 0, 0);
+		return 1;
+	}
 
-	/*
-	 * If the info is requested for an LV and segment
-	 * status for segment that belong to another LV,
-	 * we need to acquire info and status separately!
-	 */
-	return _lv_info(cmd, lv, use_layer, &status->info, NULL, NULL, with_open_count, with_read_ahead) &&
-	       _lv_info(cmd, lv_seg->lv, use_layer, NULL, lv_seg, &status->seg_status, 0, 0);
+	if (lv_is_thin_pool(lv)) {
+		/* Always collect status for '-tpool' */
+		if (_lv_info(cmd, lv, 1, &status->info, lv_seg, &status->seg_status, 0, 0) &&
+		    (status->seg_status.type == SEG_STATUS_THIN_POOL)) {
+			/* There is -tpool device, but query 'active' state of 'fake' thin-pool */
+			if (!_lv_info(cmd, lv, 0, NULL, NULL, NULL, 0, 0) &&
+			    !status->seg_status.thin_pool->needs_check)
+				status->info.exists = 0; /* So pool LV is not active */
+		}
+		return 1;
+	} else if (lv_is_external_origin(lv)) {
+		if (!_lv_info(cmd, lv, 0, &status->info, NULL, NULL,
+			      with_open_count, with_read_ahead))
+			return_0;
+
+		(void) _lv_info(cmd, lv, 1, NULL, lv_seg, &status->seg_status, 0, 0);
+		return 1;
+	} else if (lv_is_origin(lv)) {
+		/* Query segment status for 'layered' (-real) device most of the time,
+		 * only for merging snapshot, query its progress.
+		 * TODO: single LV may need couple status to be exposed at once....
+		 *       but this needs more logical background
+		 */
+		/* Show INFO for actual origin and grab status for merging origin */
+		if (!_lv_info(cmd, lv, 0, &status->info, lv_seg,
+			      lv_is_merging_origin(lv) ? &status->seg_status : NULL,
+			      with_open_count, with_read_ahead))
+			return_0;
+
+		if (status->info.exists &&
+		    (status->seg_status.type != SEG_STATUS_SNAPSHOT)) /* Not merging */
+			/* Grab STATUS from layered -real */
+			(void) _lv_info(cmd, lv, 1, NULL, lv_seg, &status->seg_status, 0, 0);
+		return 1;
+	} else if (lv_is_cow(lv)) {
+		if (lv_is_merging_cow(lv)) {
+			olv = origin_from_cow(lv);
+
+			if (!_lv_info(cmd, olv, 0, &status->info, first_seg(olv), &status->seg_status,
+				      with_open_count, with_read_ahead))
+				return_0;
+
+			if (status->seg_status.type == SEG_STATUS_SNAPSHOT) {
+				log_debug_activation("Snapshot merge is in progress, querying status of %s instead.",
+						     display_lvname(lv));
+				/*
+				 * When merge is in progress, query merging origin LV instead.
+				 * COW volume is already mapped as error target in this case.
+				 */
+				status->lv = olv;
+				return 1;
+			}
+
+			/* Merge not yet started, still a snapshot... */
+		}
+		/* Hadle fictional lvm2 snapshot and query snapshotX volume */
+		lv_seg = find_snapshot(lv);
+	}
+
+	return _lv_info(cmd, lv, 0, &status->info, lv_seg, &status->seg_status,
+			with_open_count, with_read_ahead);
 }
 
 #define OPEN_COUNT_CHECK_RETRIES 25
@@ -1136,7 +1175,7 @@ int lv_cache_status(const struct logical_volume *cache_lv,
 		return 0;
 	}
 
-	if (!lv_info(cache_lv->vg->cmd, cache_lv, 0, NULL, 0, 0)) {
+	if (!lv_info(cache_lv->vg->cmd, cache_lv, 1, NULL, 0, 0)) {
 		log_error("Cannot check status for locally inactive cache volume %s.",
 			  display_lvname(cache_lv));
 		return 0;
@@ -1489,6 +1528,26 @@ out:
 	return r || l;
 }
 
+/*
+ * Check if "raid4" @segtype is supported by kernel.
+ *
+ * if segment type is not raid4, return 1.
+ */
+int raid4_is_supported(struct cmd_context *cmd, const struct segment_type *segtype)
+{
+	unsigned attrs;
+
+	if (segtype_is_raid4(segtype) &&
+	    (!segtype->ops->target_present ||
+             !segtype->ops->target_present(cmd, NULL, &attrs) ||
+             !(attrs & RAID_FEATURE_RAID4))) {
+		log_error("RAID module does not support RAID4.");
+		return 0;
+	}
+
+	return 1;
+}
+
 int lv_is_active(const struct logical_volume *lv)
 {
 	return _lv_is_active(lv, NULL, NULL, NULL);
@@ -1548,7 +1607,7 @@ static struct dm_event_handler *_create_dm_event_handler(struct cmd_context *cmd
 	if (dm_event_handler_set_dmeventd_path(dmevh, find_config_tree_str(cmd, dmeventd_executable_CFG, NULL)))
 		goto_bad;
 
-	if (dm_event_handler_set_dso(dmevh, dso))
+	if (dso && dm_event_handler_set_dso(dmevh, dso))
 		goto_bad;
 
 	if (dm_event_handler_set_uuid(dmevh, dmuuid))
@@ -1592,6 +1651,39 @@ static char *_build_target_uuid(struct cmd_context *cmd, const struct logical_vo
 	return build_dm_uuid(cmd->mem, lv, layer);
 }
 
+static int _device_registered_with_dmeventd(struct cmd_context *cmd, const struct logical_volume *lv, int *pending, const char **dso)
+{
+	char *uuid;
+	enum dm_event_mask evmask = 0;
+	struct dm_event_handler *dmevh;
+
+	*pending = 0;
+
+	if (!(uuid = _build_target_uuid(cmd, lv)))
+		return_0;
+
+	if (!(dmevh = _create_dm_event_handler(cmd, uuid, NULL, 0, DM_EVENT_ALL_ERRORS)))
+		return_0;
+
+	if (dm_event_get_registered_device(dmevh, 0)) {
+		dm_event_handler_destroy(dmevh);
+		return 0;
+	}
+
+	evmask = dm_event_handler_get_event_mask(dmevh);
+	if (evmask & DM_EVENT_REGISTRATION_PENDING) {
+		*pending = 1;
+		evmask &= ~DM_EVENT_REGISTRATION_PENDING;
+	}
+
+	if (dso && (*dso = dm_event_handler_get_dso(dmevh)) && !(*dso = dm_pool_strdup(cmd->mem, *dso)))
+		log_error("Failed to duplicate dso name.");
+
+	dm_event_handler_destroy(dmevh);
+
+	return evmask;
+}
+
 int target_registered_with_dmeventd(struct cmd_context *cmd, const char *dso,
 				    const struct logical_volume *lv, int *pending)
 {
@@ -1650,7 +1742,7 @@ int target_register_events(struct cmd_context *cmd, const char *dso, const struc
 	if (!r)
 		return_0;
 
-	log_info("%s %s for events", set ? "Monitored" : "Unmonitored", uuid);
+	log_very_verbose("%s %s for events", set ? "Monitored" : "Unmonitored", uuid);
 
 	return 1;
 }
@@ -1674,6 +1766,8 @@ int monitor_dev_for_events(struct cmd_context *cmd, const struct logical_volume
 	uint32_t s;
 	static const struct lv_activate_opts zlaopts = { 0 };
 	struct lvinfo info;
+	const char *dso = NULL;
+	int new_unmonitor;
 
 	if (!laopts)
 		laopts = &zlaopts;
@@ -1688,6 +1782,23 @@ int monitor_dev_for_events(struct cmd_context *cmd, const struct logical_volume
 	if (monitor && !dmeventd_monitor_mode())
 		return 1;
 
+	/*
+	 * Activation of unused cache-pool activates metadata device as
+	 * a public LV  for clearing purpose.
+	 * FIXME:
+	 *  As VG lock is held across whole operation unmonitored volume
+	 *  is usually OK since dmeventd couldn't do anything.
+	 *  However in case command would have crashed, such LV is
+	 *  left unmonitored and may potentially require dmeventd.
+	 */
+	if ((lv_is_cache_pool_data(lv) || lv_is_cache_pool_metadata(lv)) &&
+	    !lv_is_used_cache_pool((find_pool_seg(first_seg(lv))->lv))) {
+		log_debug_activation("Skipping %smonitor of %s.%s",
+				     (monitor) ? "" : "un", display_lvname(lv),
+				     (monitor) ? " Cache pool activation for clearing only." : "");
+		return 1;
+	}
+
 	/*
 	 * Allow to unmonitor thin pool via explicit pool unmonitor
 	 * or unmonitor before the last thin pool user deactivation
@@ -1722,7 +1833,8 @@ int monitor_dev_for_events(struct cmd_context *cmd, const struct logical_volume
 	/*
 	 * In case this LV is a snapshot origin, we instead monitor
 	 * each of its respective snapshots.  The origin itself may
-	 * also need to be monitored if it is a mirror, for example.
+	 * also need to be monitored if it is a mirror, for example,
+	 * so fall through to process it afterwards.
 	 */
 	if (!laopts->origin_only && lv_is_origin(lv))
 		dm_list_iterate_safe(snh, snht, &lv->snapshot_segs)
@@ -1781,55 +1893,68 @@ int monitor_dev_for_events(struct cmd_context *cmd, const struct logical_volume
 		    !seg->segtype->ops->target_monitored) /* doesn't support registration */
 			continue;
 
-		monitored = seg->segtype->ops->target_monitored(seg, &pending);
+		if (!monitor)
+			/* When unmonitoring, obtain existing dso being used. */
+			monitored = _device_registered_with_dmeventd(cmd, seg_is_snapshot(seg) ? seg->cow : seg->lv, &pending, &dso);
+		else
+			monitored = seg->segtype->ops->target_monitored(seg, &pending);
 
 		/* FIXME: We should really try again if pending */
 		monitored = (pending) ? 0 : monitored;
 
 		monitor_fn = NULL;
+		new_unmonitor = 0;
 
 		if (monitor) {
 			if (monitored)
 				log_verbose("%s already monitored.", display_lvname(lv));
-			else if (seg->segtype->ops->target_monitor_events)
+			else if (seg->segtype->ops->target_monitor_events) {
+				log_verbose("Monitoring %s%s", display_lvname(lv), test_mode() ? " [Test mode: skipping this]" : "");
 				monitor_fn = seg->segtype->ops->target_monitor_events;
+			}
 		} else {
 			if (!monitored)
 				log_verbose("%s already not monitored.", display_lvname(lv));
-			else if (seg->segtype->ops->target_unmonitor_events)
-				monitor_fn = seg->segtype->ops->target_unmonitor_events;
+			else if (dso && *dso) {
+				/*
+				 * Divert unmonitor away from code that depends on the new segment
+				 * type instead of the existing one if it's changing.
+				 */
+				log_verbose("Not monitoring %s with %s%s", display_lvname(lv), dso, test_mode() ? " [Test mode: skipping this]" : "");
+				new_unmonitor = 1;
+			}
 		}
 
-		/* Do [un]monitor */
-		if (!monitor_fn)
-			continue;
-
-		log_verbose("%sonitoring %s%s", monitor ? "M" : "Not m", display_lvname(lv),
-			    test_mode() ? " [Test mode: skipping this]" : "");
-
 		/* FIXME Test mode should really continue a bit further. */
 		if (test_mode())
 			continue;
 
-		/* FIXME specify events */
-		if (!monitor_fn(seg, 0)) {
-			log_error("%s: %s segment monitoring function failed.",
-				  display_lvname(lv), seg->segtype->name);
-			return 0;
-		}
+		if (new_unmonitor) {
+			if (!target_register_events(cmd, dso, seg_is_snapshot(seg) ? seg->cow : lv, 0, 0, 10)) {
+				log_error("%s: segment unmonitoring failed.",
+					  display_lvname(lv));
+ 
+				return 0;
+			}
+		} else if (monitor_fn) {
+			/* FIXME specify events */
+			if (!monitor_fn(seg, 0)) {
+				log_error("%s: %s segment monitoring function failed.",
+					  display_lvname(lv), lvseg_name(seg));
+				return 0;
+			}
+		} else
+			continue;
 
 		/* Check [un]monitor results */
 		/* Try a couple times if pending, but not forever... */
-		for (i = 0; i < 40; i++) {
+		for (i = 0;; i++) {
 			pending = 0;
 			monitored = seg->segtype->ops->target_monitored(seg, &pending);
-			if (pending ||
-			    (!monitored && monitor) ||
-			    (monitored && !monitor))
-				log_very_verbose("%s %smonitoring still pending: waiting...",
-						 display_lvname(lv), monitor ? "" : "un");
-			else
+			if (!pending || i >= 40)
 				break;
+			log_very_verbose("%s %smonitoring still pending: waiting...",
+					 display_lvname(lv), monitor ? "" : "un");
 			usleep(10000 * i);
 		}
 
@@ -2449,6 +2574,77 @@ int lv_mknodes(struct cmd_context *cmd, const struct logical_volume *lv)
 	return r;
 }
 
+/* Remove any existing, closed mapped device by @name */
+static int _remove_dm_dev_by_name(const char *name)
+{
+	int r = 0;
+	struct dm_task *dmt;
+	struct dm_info info;
+
+	if (!(dmt = dm_task_create(DM_DEVICE_INFO)))
+		return_0;
+
+	/* Check, if the device exists. */
+	if (dm_task_set_name(dmt, name) && dm_task_run(dmt) && dm_task_get_info(dmt, &info)) {
+		dm_task_destroy(dmt);
+
+		/* Ignore non-existing or open dm devices */
+		if (!info.exists || info.open_count)
+			return 1;
+
+		if (!(dmt = dm_task_create(DM_DEVICE_REMOVE)))
+			return_0;
+
+		if (dm_task_set_name(dmt, name))
+			r = dm_task_run(dmt);
+	}
+
+	dm_task_destroy(dmt);
+
+	return r;
+}
+
+/* Work all segments of @lv removing any existing, closed "*-missing_N_0" sub devices. */
+static int _lv_remove_any_missing_subdevs(struct logical_volume *lv)
+{
+	if (lv) {
+		uint32_t seg_no = 0;
+		char name[257];
+		struct lv_segment *seg;
+
+		dm_list_iterate_items(seg, &lv->segments) {
+			if (seg->area_count != 1)
+				return_0;
+			if (dm_snprintf(name, sizeof(name), "%s-%s-missing_%u_0", seg->lv->vg->name, seg->lv->name, seg_no) < 0)
+				return 0;
+			if (!_remove_dm_dev_by_name(name))
+				return 0;
+
+			seg_no++;
+		}
+	}
+
+	return 1;
+}
+
+/* Remove any "*-missing_*" sub devices added by the activation layer for an rmate/rimage missing PV mapping */
+int lv_deactivate_any_missing_subdevs(const struct logical_volume *lv)
+{
+	uint32_t s;
+	struct lv_segment *seg = first_seg(lv);
+
+	for (s = 0; s < seg->area_count; s++) {
+		if (seg_type(seg, s) == AREA_LV &&
+		    !_lv_remove_any_missing_subdevs(seg_lv(seg, s)))
+			return 0;
+		if (seg->meta_areas && seg_metatype(seg, s) == AREA_LV &&
+		    !_lv_remove_any_missing_subdevs(seg_metalv(seg, s)))
+			return 0;
+	}
+
+	return 1;
+}
+
 /*
  * Does PV use VG somewhere in its construction?
  * Returns 1 on failure.

lib/activate/activate.h

@@ -1,6 +1,6 @@
 /*
  * Copyright (C) 2001-2004 Sistina Software, Inc. All rights reserved.  
- * Copyright (C) 2004-2012 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2004-2016 Red Hat, Inc. All rights reserved.
  *
  * This file is part of LVM2.
  *
@@ -54,11 +54,12 @@ struct lv_seg_status {
 };
 
 struct lv_with_info_and_seg_status {
-	const struct logical_volume *lv;	/* input */
 	int info_ok;
+	const struct logical_volume *lv;        /* output */
 	struct lvinfo info;			/* output */
 	int seg_part_of_lv;			/* output */
-	struct lv_seg_status seg_status;	/* input/output, see lv_seg_status */
+	struct lv_seg_status seg_status;	/* output, see lv_seg_status */
+	/* TODO: add extra status for snapshot origin */
 };
 
 struct lv_activate_opts {
@@ -99,6 +100,7 @@ int target_present(struct cmd_context *cmd, const char *target_name,
 		   int use_modprobe);
 int target_version(const char *target_name, uint32_t *maj,
 		   uint32_t *min, uint32_t *patchlevel);
+int raid4_is_supported(struct cmd_context *cmd, const struct segment_type *segtype);
 int lvm_dm_prefix_check(int major, int minor, const char *prefix);
 int list_segment_modules(struct dm_pool *mem, const struct lv_segment *seg,
 			 struct dm_list *modules);
@@ -122,6 +124,8 @@ int lv_deactivate(struct cmd_context *cmd, const char *lvid_s, const struct logi
 
 int lv_mknodes(struct cmd_context *cmd, const struct logical_volume *lv);
 
+int lv_deactivate_any_missing_subdevs(const struct logical_volume *lv);
+
 /*
  * Returns 1 if info structure has been populated, else 0 on failure.
  * When lvinfo* is NULL, it returns 1 if the device is locally active, 0 otherwise.
@@ -131,13 +135,6 @@ int lv_info(struct cmd_context *cmd, const struct logical_volume *lv, int use_la
 int lv_info_by_lvid(struct cmd_context *cmd, const char *lvid_s, int use_layer,
 		    struct lvinfo *info, int with_open_count, int with_read_ahead);
 
-/*
- * Returns 1 if lv_seg_status structure has been populated,
- * else 0 on failure or if device not active locally.
- */
-int lv_status(struct cmd_context *cmd, const struct lv_segment *lv_seg,
-	      int use_layer, struct lv_seg_status *lv_seg_status);
-
 /*
  * Returns 1 if lv_info_and_seg_status structure has been populated,
  * else 0 on failure or if device not active locally.
@@ -145,8 +142,8 @@ int lv_status(struct cmd_context *cmd, const struct lv_segment *lv_seg,
  * lv_info_with_seg_status is the same as calling lv_info and then lv_status,
  * but this fn tries to do that with one ioctl if possible.
  */
-int lv_info_with_seg_status(struct cmd_context *cmd, const struct logical_volume *lv,
-			    const struct lv_segment *lv_seg, int use_layer,
+int lv_info_with_seg_status(struct cmd_context *cmd,
+			    const struct lv_segment *lv_seg,
 			    struct lv_with_info_and_seg_status *status,
 			    int with_open_count, int with_read_ahead);
 

lib/cache/lvmcache.c

@@ -858,7 +858,7 @@ static void _choose_preferred_devs(struct cmd_context *cmd,
 				   struct dm_list *add_cache_devs)
 {
 	char uuid[64] __attribute__((aligned(8)));
-	const char *reason = "none";
+	const char *reason;
 	struct dm_list altdevs;
 	struct dm_list new_unused;
 	struct dev_types *dt = cmd->dev_types;
@@ -1275,7 +1275,7 @@ struct volume_group *lvmcache_get_vg(struct cmd_context *cmd, const char *vgname
 	/* Build config tree from vgmetadata, if not yet cached */
 	if (!vginfo->cft &&
 	    !(vginfo->cft =
-	      dm_config_from_string(vginfo->vgmetadata)))
+	      config_tree_from_string_without_dup_node_check(vginfo->vgmetadata)))
 		goto_bad;
 
 	if (!(vg = import_vg_from_config_tree(vginfo->cft, fid)))

lib/cache/lvmetad.c

@@ -1918,15 +1918,44 @@ scan_more:
 			vgmeta_ret = vgmeta;
 			save_dev = devl->dev;
 		} else {
-			if (compare_config(vgmeta_ret->root, vgmeta->root)) {
+			struct dm_config_node *meta1 = vgmeta_ret->root;
+			struct dm_config_node *meta2 = vgmeta->root;
+			struct dm_config_node *sib1 = meta1->sib;
+			struct dm_config_node *sib2 = meta2->sib;
+
+			/*
+			 * Do not compare the extraneous data that
+			 * export_vg_to_config_tree() inserts next to the
+			 * actual VG metadata.  This includes creation_time
+			 * which may not match since it is generated separately
+			 * for each call to create the config tree.
+			 *
+			 * We're saving the sibling pointer and restoring it
+			 * after the compare because we're unsure if anything
+			 * later might want it.
+			 *
+			 * FIXME: make it clearer what we're doing here, e.g.
+			 * pass a parameter to export_vg_to_config_tree()
+			 * telling it to skip the extraneous data, or something.
+			 * It's very non-obvious that setting sib=NULL does that.
+			 */
+			meta1->sib = NULL;
+			meta2->sib = NULL;
+
+			if (compare_config(meta1, meta2)) {
 				log_error("VG %s metadata comparison failed for device %s vs %s",
 					  vg->name, dev_name(devl->dev), save_dev ? dev_name(save_dev) : "none");
 				_log_debug_inequality(vg->name, vgmeta_ret->root, vgmeta->root);
+
+				meta1->sib = sib1;
+				meta2->sib = sib2;
 				dm_config_destroy(vgmeta);
 				dm_config_destroy(vgmeta_ret);
 				release_vg(baton.vg);
 				return NULL;
 			}
+			meta1->sib = sib1;
+			meta2->sib = sib2;
 			dm_config_destroy(vgmeta);
 		}
 
@@ -2032,7 +2061,8 @@ scan_more:
 		dm_config_destroy(vgmeta_ret);
 	}
 out:
-	log_debug_lvmetad("Rescan VG %s done (seqno %u).", vg_ret->name, vg_ret->seqno);
+	if (vg_ret)
+		log_debug_lvmetad("Rescan VG %s done (seqno %u).", vg_ret->name, vg_ret->seqno);
 	return vg_ret;
 }
 
@@ -2051,6 +2081,11 @@ int lvmetad_pvscan_single(struct cmd_context *cmd, struct device *dev,
 		return 0;
 	}
 
+	if (udev_dev_is_mpath_component(dev)) {
+		log_debug("Ignore multipath component for pvscan.");
+		return 1;
+	}
+
 	if (!label_read(dev, &label, 0)) {
 		log_print_unless_silent("No PV label found on %s.", dev_name(dev));
 		if (!lvmetad_pv_gone_by_dev(dev))

lib/commands/toolcontext.c

@@ -192,6 +192,9 @@ static int _get_env_vars(struct cmd_context *cmd)
 		}
 	}
 
+	if (strcmp((getenv("LVM_RUN_BY_DMEVENTD") ? : "0"), "1") == 0)
+		init_run_by_dmeventd(cmd);
+
 	return 1;
 }
 
@@ -640,8 +643,8 @@ static int _process_config(struct cmd_context *cmd)
 	if (!strcmp(cmd->stripe_filler, "/dev/ioerror") &&
 	    stat(cmd->stripe_filler, &st))
 		cmd->stripe_filler = "error";
-
-	if (strcmp(cmd->stripe_filler, "error")) {
+	else if (strcmp(cmd->stripe_filler, "error") &&
+		 strcmp(cmd->stripe_filler, "zero")) {
 		if (stat(cmd->stripe_filler, &st)) {
 			log_warn("WARNING: activation/missing_stripe_filler = \"%s\" "
 				 "is invalid,", cmd->stripe_filler);
@@ -997,7 +1000,7 @@ static int _init_dev_cache(struct cmd_context *cmd)
 
 	if (!(cn = find_config_tree_array(cmd, devices_scan_CFG, NULL))) {
 		log_error(INTERNAL_ERROR "Unable to find configuration for devices/scan.");
-		return_0;
+		return 0;
 	}
 
 	for (cv = cn->v; cv; cv = cv->next) {
@@ -1755,6 +1758,15 @@ bad:
 	return 0;
 }
 
+int init_run_by_dmeventd(struct cmd_context *cmd)
+{
+	init_dmeventd_monitor(DMEVENTD_MONITOR_IGNORE);
+	init_ignore_suspended_devices(1);
+	init_disable_dmeventd_monitoring(1); /* Lock settings */
+
+	return 0;
+}
+
 void destroy_config_context(struct cmd_context *cmd)
 {
 	_destroy_config(cmd);
@@ -1787,10 +1799,27 @@ struct cmd_context *create_config_context(void)
 		goto_out;
 
 	dm_list_init(&cmd->config_files);
+	dm_list_init(&cmd->tags);
 
 	if (!_init_lvm_conf(cmd))
 		goto_out;
 
+	if (!_init_hostname(cmd))
+		goto_out;
+
+	if (!_init_tags(cmd, cmd->cft))
+		goto_out;
+
+	/* Load lvmlocal.conf */
+	if (*cmd->system_dir && !_load_config_file(cmd, "", 1))
+		goto_out;
+
+	if (!_init_tag_configs(cmd))
+		goto_out;
+
+	if (!(cmd->cft = _merge_config_files(cmd, cmd->cft)))
+		goto_out;
+
 	return cmd;
 out:
 	if (cmd)

lib/commands/toolcontext.h

@@ -88,11 +88,20 @@ struct cmd_context {
 	 * Command line and arguments.
 	 */
 	const char *cmd_line;
+	const char *name; /* needed before cmd->command is set */
+	struct command_name *cname;
 	struct command *command;
 	char **argv;
 	struct arg_values *opt_arg_values;
 	struct dm_list arg_value_groups;
 
+	/*
+	 * Position args remaining after command name
+	 * and --options are removed from original argc/argv.
+	 */
+	int position_argc;
+	char **position_argv;
+
 	/*
 	 * Format handlers.
 	 */
@@ -233,6 +242,7 @@ int config_files_changed(struct cmd_context *cmd);
 int init_lvmcache_orphans(struct cmd_context *cmd);
 int init_filters(struct cmd_context *cmd, unsigned load_persistent_cache);
 int init_connections(struct cmd_context *cmd);
+int init_run_by_dmeventd(struct cmd_context *cmd);
 
 /*
  * A config context is a very light weight cmd struct that

lib/config/config.c

@@ -389,7 +389,7 @@ int override_config_tree_from_string(struct cmd_context *cmd,
 	    !config_force_check(cmd, CONFIG_STRING, cft_new)) {
 		log_error("Ignoring invalid configuration string.");
 		dm_config_destroy(cft_new);
-		return_0;
+		return 0;
 	}
 
 	if (!(cs = dm_pool_zalloc(cft_new->mem, sizeof(struct config_source)))) {
@@ -496,7 +496,7 @@ int override_config_tree_from_profile(struct cmd_context *cmd,
 int config_file_read_fd(struct dm_config_tree *cft, struct device *dev,
 			off_t offset, size_t size, off_t offset2, size_t size2,
 			checksum_fn_t checksum_fn, uint32_t checksum,
-			int checksum_only)
+			int checksum_only, int no_dup_node_check)
 {
 	char *fb, *fe;
 	int r = 0;
@@ -547,8 +547,13 @@ int config_file_read_fd(struct dm_config_tree *cft, struct device *dev,
 
 	if (!checksum_only) {
 		fe = fb + size + size2;
-		if (!dm_config_parse(cft, fb, fe))
-			goto_out;
+		if (no_dup_node_check) {
+			if (!dm_config_parse_without_dup_node_check(cft, fb, fe))
+				goto_out;
+		} else {
+			if (!dm_config_parse(cft, fb, fe))
+				goto_out;
+		}
 	}
 
 	r = 1;
@@ -596,7 +601,7 @@ int config_file_read(struct dm_config_tree *cft)
 	}
 
 	r = config_file_read_fd(cft, cf->dev, 0, (size_t) info.st_size, 0, 0,
-				(checksum_fn_t) NULL, 0, 0);
+				(checksum_fn_t) NULL, 0, 0, 0);
 
 	if (!cf->keep_open) {
 		if (!dev_close(cf->dev))
@@ -2461,3 +2466,25 @@ int get_default_allocation_cache_pool_chunk_size_CFG(struct cmd_context *cmd, st
 {
 	return DEFAULT_CACHE_POOL_CHUNK_SIZE * 2;
 }
+
+uint64_t get_default_allocation_cache_pool_max_chunks_CFG(struct cmd_context *cmd, struct profile *profile)
+{
+	static int _warn_max_chunks = 0;
+	/*
+	 * TODO: In future may depend on the cache target version,
+	 *       newer targets may scale better.
+	 */
+	uint64_t default_max_chunks = DEFAULT_CACHE_POOL_MAX_CHUNKS;
+	uint64_t max_chunks = find_config_tree_int(cmd, allocation_cache_pool_max_chunks_CFG, profile);
+
+	if (!max_chunks)
+		max_chunks = default_max_chunks;
+	else if (max_chunks > default_max_chunks)
+		/* Still warn the user when the value is tweaked above recommended level */
+		/* Maybe drop to log_verbose... */
+		log_warn_suppress(_warn_max_chunks++, "WARNING: Configured cache_pool_max_chunks value "
+				  FMTu64 " is higher then recommended " FMTu64 ".",
+				  max_chunks, default_max_chunks);
+
+	return max_chunks;
+}

lib/config/config.h

@@ -239,7 +239,7 @@ struct dm_config_tree *config_open(config_source_t source, const char *filename,
 int config_file_read_fd(struct dm_config_tree *cft, struct device *dev,
 			off_t offset, size_t size, off_t offset2, size_t size2,
 			checksum_fn_t checksum_fn, uint32_t checksum,
-			int skip_parse);
+			int skip_parse, int no_dup_node_check);
 int config_file_read(struct dm_config_tree *cft);
 struct dm_config_tree *config_file_open_and_read(const char *config_file, config_source_t source,
 						 struct cmd_context *cmd);
@@ -307,5 +307,6 @@ int get_default_allocation_cache_pool_chunk_size_CFG(struct cmd_context *cmd, st
 #define get_default_unconfigured_allocation_cache_pool_chunk_size_CFG NULL
 const char *get_default_allocation_cache_policy_CFG(struct cmd_context *cmd, struct profile *profile);
 #define get_default_unconfigured_allocation_cache_policy_CFG NULL
+uint64_t get_default_allocation_cache_pool_max_chunks_CFG(struct cmd_context *cmd, struct profile *profile);
 
 #endif

lib/config/config_settings.h

@@ -516,6 +516,11 @@ cfg_runtime(allocation_cache_pool_chunk_size_CFG, "cache_pool_chunk_size", alloc
 	"on the smaller end of the spectrum. Supported values range from\n"
 	"32KiB to 1GiB in multiples of 32.\n")
 
+cfg(allocation_cache_pool_max_chunks_CFG, "cache_pool_max_chunks", allocation_CFG_SECTION, CFG_PROFILABLE | CFG_DEFAULT_UNDEFINED, CFG_TYPE_INT, 0, vsn(2, 2, 165), NULL, 0, NULL,
+	"The maximum number of chunks in a cache pool.\n"
+	"For cache target v1.9 the recommended maximumm is 1000000 chunks.\n"
+	"Using cache pool with more chunks may degrade cache performance.\n")
+
 cfg(allocation_thin_pool_metadata_require_separate_pvs_CFG, "thin_pool_metadata_require_separate_pvs", allocation_CFG_SECTION, 0, CFG_TYPE_BOOL, DEFAULT_THIN_POOL_METADATA_REQUIRE_SEPARATE_PVS, vsn(2, 2, 89), NULL, 0, NULL,
 	"Thin pool metdata and data will always use different PVs.\n")
 
@@ -1106,7 +1111,8 @@ cfg(activation_retry_deactivation_CFG, "retry_deactivation", activation_CFG_SECT
 cfg(activation_missing_stripe_filler_CFG, "missing_stripe_filler", activation_CFG_SECTION, CFG_ADVANCED, CFG_TYPE_STRING, DEFAULT_STRIPE_FILLER, vsn(1, 0, 0), NULL, 0, NULL,
 	"Method to fill missing stripes when activating an incomplete LV.\n"
 	"Using 'error' will make inaccessible parts of the device return I/O\n"
-	"errors on access. You can instead use a device path, in which case,\n"
+	"errors on access. Using 'zero' will return success (and zero) on I/O\n"
+	"You can instead use a device path, in which case,\n"
 	"that device will be used in place of missing stripes. Using anything\n"
 	"other than 'error' with mirrored or snapshotted volumes is likely to\n"
 	"result in data corruption.\n")
@@ -1215,14 +1221,15 @@ cfg_array(activation_read_only_volume_list_CFG, "read_only_volume_list", activat
 	"read_only_volume_list = [ \"vg1\", \"vg2/lvol1\", \"@tag1\", \"@*\" ]\n"
 	"#\n")
 
-cfg(activation_mirror_region_size_CFG, "mirror_region_size", activation_CFG_SECTION, 0, CFG_TYPE_INT, DEFAULT_RAID_REGION_SIZE, vsn(1, 0, 0), NULL, vsn(2, 2, 99),
+ cfg(activation_mirror_region_size_CFG, "mirror_region_size", activation_CFG_SECTION, 0, CFG_TYPE_INT, DEFAULT_RAID_REGION_SIZE, vsn(1, 0, 0), NULL, vsn(2, 2, 99),
 	"This has been replaced by the activation/raid_region_size setting.\n",
-        "Size in KiB of each copy operation when mirroring.\n")
+	"Size in KiB of each raid or mirror synchronization region.\n")
 
 cfg(activation_raid_region_size_CFG, "raid_region_size", activation_CFG_SECTION, 0, CFG_TYPE_INT, DEFAULT_RAID_REGION_SIZE, vsn(2, 2, 99), NULL, 0, NULL,
 	"Size in KiB of each raid or mirror synchronization region.\n"
-	"For raid or mirror segment types, this is the amount of data that is\n"
-	"copied at once when initializing, or moved at once by pvmove.\n")
+	"The clean/dirty state of data is tracked for each region.\n"
+	"The value is rounded down to a power of two if necessary, and\n"
+	"is ignored if it is not a multiple of the machine memory page size.\n")
 
 cfg(activation_error_when_full_CFG, "error_when_full", activation_CFG_SECTION, CFG_DEFAULT_COMMENTED, CFG_TYPE_BOOL, DEFAULT_ERROR_WHEN_FULL, vsn(2, 2, 115), NULL, 0, NULL,
 	"Return errors if a thin pool runs out of space.\n"
@@ -1571,7 +1578,7 @@ cfg(report_prefixes_CFG, "prefixes", report_CFG_SECTION, CFG_PROFILABLE | CFG_DE
 cfg(report_quoted_CFG, "quoted", report_CFG_SECTION, CFG_PROFILABLE | CFG_DEFAULT_COMMENTED, CFG_TYPE_BOOL, DEFAULT_REP_QUOTED, vsn(2, 2, 39), NULL, 0, NULL,
 	"Quote field values when using field name prefixes.\n")
 
-cfg(report_colums_as_rows_CFG, "colums_as_rows", report_CFG_SECTION, CFG_PROFILABLE | CFG_DEFAULT_COMMENTED, CFG_TYPE_BOOL, DEFAULT_REP_COLUMNS_AS_ROWS, vsn(1, 0, 0), NULL, 0, NULL,
+cfg(report_columns_as_rows_CFG, "columns_as_rows", report_CFG_SECTION, CFG_PROFILABLE | CFG_DEFAULT_COMMENTED, CFG_TYPE_BOOL, DEFAULT_REP_COLUMNS_AS_ROWS, vsn(1, 0, 0), NULL, 0, NULL,
 	"Output each column as a row.\n"
 	"If set, this also implies report/prefixes=1.\n")
 
@@ -1853,6 +1860,14 @@ cfg(dmeventd_thin_library_CFG, "thin_library", dmeventd_CFG_SECTION, 0, CFG_TYPE
 	"and emits a warning through syslog when the usage exceeds 80%. The\n"
 	"warning is repeated when 85%, 90% and 95% of the pool is filled.\n")
 
+cfg(dmeventd_thin_command_CFG, "thin_command", dmeventd_CFG_SECTION, CFG_DEFAULT_COMMENTED, CFG_TYPE_STRING, DEFAULT_DMEVENTD_THIN_COMMAND, vsn(2, 2, 169), NULL, 0, NULL,
+	"The plugin runs command with each 5% increment when thin-pool data volume\n"
+	"or metadata volume gets above 50%.\n"
+	"Command which starts with 'lvm ' prefix is internal lvm command.\n"
+	"You can write your own handler to customise behaviour in more details.\n"
+	"User handler is specified with the full path starting with '/'.\n")
+	/* TODO: systemd service handler */
+
 cfg(dmeventd_executable_CFG, "executable", dmeventd_CFG_SECTION, CFG_DEFAULT_COMMENTED, CFG_TYPE_STRING, DEFAULT_DMEVENTD_PATH, vsn(2, 2, 73), "@DMEVENTD_PATH@", 0, NULL,
 	"The full path to the dmeventd binary.\n")
 

lib/config/defaults.h

@@ -81,6 +81,7 @@
 #define DEFAULT_DMEVENTD_MIRROR_LIB "libdevmapper-event-lvm2mirror.so"
 #define DEFAULT_DMEVENTD_SNAPSHOT_LIB "libdevmapper-event-lvm2snapshot.so"
 #define DEFAULT_DMEVENTD_THIN_LIB "libdevmapper-event-lvm2thin.so"
+#define DEFAULT_DMEVENTD_THIN_COMMAND "lvm lvextend --use-policies"
 #define DEFAULT_DMEVENTD_MONITOR 1
 #define DEFAULT_BACKGROUND_POLLING 1
 
@@ -127,6 +128,7 @@
 #define DEFAULT_CACHE_REPAIR_OPTIONS_CONFIG "#S" DEFAULT_CACHE_REPAIR_OPTION1
 #define DEFAULT_CACHE_POOL_METADATA_REQUIRE_SEPARATE_PVS 0
 #define DEFAULT_CACHE_POOL_CHUNK_SIZE 64 /* KB */
+#define DEFAULT_CACHE_POOL_MAX_CHUNKS 1000000
 #define DEFAULT_CACHE_POOL_MIN_METADATA_SIZE 2048  /* KB */
 #define DEFAULT_CACHE_POOL_MAX_METADATA_SIZE (16 * 1024 * 1024)  /* KB */
 #define DEFAULT_CACHE_POLICY "mq"
@@ -175,7 +177,7 @@
 #define DEFAULT_INDENT 1
 #define DEFAULT_ABORT_ON_INTERNAL_ERRORS 0
 #define DEFAULT_DETECT_INTERNAL_VG_CACHE_CORRUPTION 0
-#define DEFAULT_UNITS "h"
+#define DEFAULT_UNITS "r"
 #define DEFAULT_SUFFIX 1
 #define DEFAULT_HOSTTAGS 0
 

lib/device/dev-ext-udev-constants.h

@@ -36,6 +36,9 @@
 #define DEV_EXT_UDEV_DEVTYPE			"DEVTYPE"
 #define DEV_EXT_UDEV_DEVTYPE_DISK		"disk"
 
+/* the list of symlinks associated with device node */
+#define DEV_EXT_UDEV_DEVLINKS			"DEVLINKS"
+
 /*
  * DEV_EXT_UDEV_MPATH_DEVICE_PATH is set by multipath in udev db
  * with value either 0 or 1. The same functionality as

lib/device/dev-type.c

@@ -125,6 +125,10 @@ struct dev_types *create_dev_types(const char *proc_dir,
 		if (!strncmp("emcpower", line + i, 8) && isspace(*(line + i + 8)))
 			dt->emcpower_major = line_maj;
 
+		/* Look for Veritas Dynamic Multipathing */
+		if (!strncmp("VxDMP", line + i, 5) && isspace(*(line + i + 5)))
+			dt->vxdmp_major = line_maj;
+
 		if (!strncmp("loop", line + i, 4) && isspace(*(line + i + 4)))
 			dt->loop_major = line_maj;
 
@@ -218,6 +222,9 @@ int dev_subsystem_part_major(struct dev_types *dt, struct device *dev)
 	if (MAJOR(dev->dev) == dt->power2_major)
 		return 1;
 
+	if (MAJOR(dev->dev) == dt->vxdmp_major)
+		return 1;
+
 	if ((MAJOR(dev->dev) == dt->blkext_major) &&
 	    dev_get_primary_dev(dt, dev, &primary_dev) &&
 	    (MAJOR(primary_dev) == dt->md_major))
@@ -246,6 +253,9 @@ const char *dev_subsystem_name(struct dev_types *dt, struct device *dev)
 	if (MAJOR(dev->dev) == dt->power2_major)
 		return "POWER2";
 
+	if (MAJOR(dev->dev) == dt->vxdmp_major)
+		return "VXDMP";
+
 	if (MAJOR(dev->dev) == dt->blkext_major)
 		return "BLKEXT";
 
@@ -976,3 +986,100 @@ int dev_is_rotational(struct dev_types *dt, struct device *dev)
 	return 1;
 }
 #endif
+
+#ifdef UDEV_SYNC_SUPPORT
+
+/*
+ * Udev daemon usually has 30s timeout to process each event by default.
+ * But still, that value can be changed in udev configuration and we
+ * don't have libudev API to read the actual timeout value used.
+ */
+
+/* FIXME: Is this long enough to wait for udev db to get initialized?
+ *
+ *        Take also into consideration that this check is done for each
+ *        device that is scanned so we don't want to wait for a long time
+ *        if there's something wrong with udev, e.g. timeouts! With current
+ *        libudev API, we can't recognize whether the event processing has
+ *        not finished yet and it's still being processed or whether it has
+ *        failed already due to timeout in udev - in both cases the
+ *        udev_device_get_is_initialized returns 0.
+ */
+#define UDEV_DEV_IS_MPATH_COMPONENT_ITERATION_COUNT 100
+#define UDEV_DEV_IS_MPATH_COMPONENT_USLEEP 100000
+
+int udev_dev_is_mpath_component(struct device *dev)
+{
+	struct udev *udev_context = udev_get_library_context();
+	struct udev_device *udev_device = NULL;
+	const char *value;
+	int initialized = 0;
+	unsigned i = 0;
+	int ret = 0;
+
+	if (!udev_context) {
+		log_warn("WARNING: No udev context available to check if device %s is multipath component.", dev_name(dev));
+		return 0;
+	}
+
+	while (1) {
+		if (i >= UDEV_DEV_IS_MPATH_COMPONENT_ITERATION_COUNT)
+			break;
+
+		if (udev_device)
+			udev_device_unref(udev_device);
+
+		if (!(udev_device = udev_device_new_from_devnum(udev_context, 'b', dev->dev))) {
+			log_warn("WARNING: Failed to get udev device handler for device %s.", dev_name(dev));
+			return 0;
+		}
+
+#ifdef HAVE_LIBUDEV_UDEV_DEVICE_GET_IS_INITIALIZED
+		if ((initialized = udev_device_get_is_initialized(udev_device)))
+			break;
+#else
+		if ((initialized = (udev_device_get_property_value(udev_device, DEV_EXT_UDEV_DEVLINKS) != NULL)))
+			break;
+#endif
+
+		log_debug("Device %s not initialized in udev database (%u/%u, %u microseconds).", dev_name(dev),
+			   i + 1, UDEV_DEV_IS_MPATH_COMPONENT_ITERATION_COUNT,
+			   i * UDEV_DEV_IS_MPATH_COMPONENT_USLEEP);
+
+		usleep(UDEV_DEV_IS_MPATH_COMPONENT_USLEEP);
+		i++;
+	}
+
+	if (!initialized) {
+		log_warn("WARNING: Device %s not initialized in udev database even after waiting %u microseconds.",
+			  dev_name(dev), i * UDEV_DEV_IS_MPATH_COMPONENT_USLEEP);
+		goto out;
+	}
+
+	value = udev_device_get_property_value(udev_device, DEV_EXT_UDEV_BLKID_TYPE);
+	if (value && !strcmp(value, DEV_EXT_UDEV_BLKID_TYPE_MPATH)) {
+		log_debug("Device %s is multipath component based on blkid variable in udev db (%s=\"%s\").",
+			   dev_name(dev), DEV_EXT_UDEV_BLKID_TYPE, value);
+		ret = 1;
+		goto out;
+	}
+
+	value = udev_device_get_property_value(udev_device, DEV_EXT_UDEV_MPATH_DEVICE_PATH);
+	if (value && !strcmp(value, "1")) {
+		log_debug("Device %s is multipath component based on multipath variable in udev db (%s=\"%s\").",
+			   dev_name(dev), DEV_EXT_UDEV_MPATH_DEVICE_PATH, value);
+		ret = 1;
+		goto out;
+	}
+out:
+	udev_device_unref(udev_device);
+	return ret;
+}
+#else
+
+int udev_dev_is_mpath_component(struct device *dev)
+{
+	return 0;
+}
+
+#endif

lib/device/dev-type.h

@@ -41,6 +41,7 @@ struct dev_types {
 	int drbd_major;
 	int device_mapper_major;
 	int emcpower_major;
+	int vxdmp_major;
 	int power2_major;
 	int dasd_major;
 	int loop_major;
@@ -59,6 +60,7 @@ int dev_is_md(struct device *dev, uint64_t *sb);
 int dev_is_swap(struct device *dev, uint64_t *signature);
 int dev_is_luks(struct device *dev, uint64_t *signature);
 int dasd_is_cdl_formatted(struct device *dev);
+int udev_dev_is_mpath_component(struct device *dev);
 
 /* Signature wiping. */
 #define TYPE_LVM1_MEMBER	0x001

lib/device/device-types.h

@@ -63,5 +63,6 @@ static const dev_known_type_t _dev_known_types[] = {
 	{"bcache", 1, "bcache block device cache"},
 	{"nvme", 64, "NVM Express"},
 	{"zvol", 16, "ZFS Zvols"},
+	{"VxDMP", 16, "Veritas Dynamic Multipathing"},
 	{"", 0, ""}
 };

lib/filters/filter-regex.c

@@ -40,7 +40,7 @@ static int _extract_pattern(struct dm_pool *mem, const char *pat,
 		break;
 
 	default:
-		log_info("pattern must begin with 'a' or 'r'");
+		log_error("Pattern must begin with 'a' or 'r'.");
 		return 0;
 	}
 	pat++;
@@ -77,7 +77,7 @@ static int _extract_pattern(struct dm_pool *mem, const char *pat,
 	 */
 	ptr = r + strlen(r) - 1;
 	if (*ptr != sep) {
-		log_info("invalid separator at end of regex");
+		log_error("Invalid separator at end of regex.");
 		return 0;
 	}
 	*ptr = '\0';

lib/filters/filter-sysfs.c

@@ -17,6 +17,7 @@
 
 #ifdef __linux__
 
+#include <sys/sysmacros.h>
 #include <dirent.h>
 
 static int _locate_sysfs_blocks(const char *sysfs_dir, char *path, size_t len,
@@ -167,7 +168,7 @@ static int _parse_dev(const char *file, FILE *fp, dev_t *result)
 	}
 
 	if (sscanf(buffer, "%u:%u", &major, &minor) != 2) {
-		log_info("sysfs device file not correct format");
+		log_error("Incorrect format for sysfs device file: %s.", file);
 		return 0;
 	}
 

lib/format1/import-export.c

@@ -398,7 +398,7 @@ int export_extents(struct disk_list *dl, uint32_t lv_num,
 			if (!(seg->segtype->flags & SEG_FORMAT1_SUPPORT)) {
 				log_error("Segment type %s in LV %s: "
 					  "unsupported by format1",
-					  seg->segtype->name, lv->name);
+					  lvseg_name(seg), lv->name);
 				return 0;
 			}
 			if (seg_type(seg, s) != AREA_PV) {
@@ -510,7 +510,7 @@ int export_lvs(struct disk_list *dl, struct volume_group *vg,
 		goto_out;
 
 	dm_list_iterate_items(ll, &vg->lvs) {
-		if (ll->lv->status & SNAPSHOT)
+		if (lv_is_snapshot(ll->lv))
 			continue;
 
 		if (!(lvdl = dm_pool_alloc(dl->mem, sizeof(*lvdl))))

lib/format1/import-extents.c

@@ -56,7 +56,7 @@ static struct dm_hash_table *_create_lv_maps(struct dm_pool *mem,
 	}
 
 	dm_list_iterate_items(ll, &vg->lvs) {
-		if (ll->lv->status & SNAPSHOT)
+		if (lv_is_snapshot(ll->lv))
 			continue;
 
 		if (!(lvm = dm_pool_alloc(mem, sizeof(*lvm))))

lib/format_pool/import_export.c

@@ -267,7 +267,7 @@ int import_pool_segments(struct dm_list *lvs, struct dm_pool *mem,
 	dm_list_iterate_items(lvl, lvs) {
 		lv = lvl->lv;
 
-		if (lv->status & SNAPSHOT)
+		if (lv_is_snapshot(lv))
 			continue;
 
 		for (i = 0; i < subpools; i++) {

lib/format_text/archiver.c

@@ -35,6 +35,7 @@ struct archive_params {
 struct backup_params {
 	int enabled;
 	char *dir;
+	int suppress;
 };
 
 int archive_init(struct cmd_context *cmd, const char *dir,
@@ -235,7 +236,8 @@ static int _backup(struct volume_group *vg)
 int backup_locally(struct volume_group *vg)
 {
 	if (!vg->cmd->backup_params->enabled || !vg->cmd->backup_params->dir) {
-		log_warn("WARNING: This metadata update is NOT backed up");
+		log_warn_suppress(vg->cmd->backup_params->suppress++,
+				  "WARNING: This metadata update is NOT backed up.");
 		return 1;
 	}
 

lib/format_text/export.c

@@ -22,6 +22,7 @@
 #include "text_export.h"
 #include "lvm-version.h"
 #include "toolcontext.h"
+#include "config-util.h"
 
 #include <stdarg.h>
 #include <time.h>
@@ -622,7 +623,7 @@ int out_areas(struct formatter *f, const struct lv_segment *seg,
 			break;
 		case AREA_LV:
 			/* FIXME This helper code should be target-independent! Check for metadata LV property. */
-			if (!(seg->status & RAID)) {
+			if (!seg_is_raid(seg)) {
 				outf(f, "\"%s\", %u%s",
 				     seg_lv(seg, s)->name,
 				     seg_le(seg, s),
@@ -1077,7 +1078,7 @@ struct dm_config_tree *export_vg_to_config_tree(struct volume_group *vg)
 		return_NULL;
 	}
 
-	if (!(vg_cft = dm_config_from_string(buf))) {
+	if (!(vg_cft = config_tree_from_string_without_dup_node_check(buf))) {
 		log_error("Error parsing metadata for VG %s.", vg->name);
 		dm_free(buf);
 		return_NULL;

lib/format_text/format-text.h

@@ -59,7 +59,7 @@ int add_da(struct dm_pool *mem, struct dm_list *das,
 void del_das(struct dm_list *das);
 int add_ba(struct dm_pool *mem, struct dm_list *eas,
 	   uint64_t start, uint64_t size);
-void del_bas(struct dm_list *eas);
+void del_bas(struct dm_list *bas);
 int add_mda(const struct format_type *fmt, struct dm_pool *mem, struct dm_list *mdas,
 	    struct device *dev, uint64_t start, uint64_t size, unsigned ignored);
 void del_mdas(struct dm_list *mdas);

lib/format_text/import.c

@@ -56,7 +56,7 @@ int text_vgsummary_import(const struct format_type *fmt,
 	    (dev && !config_file_read_fd(cft, dev, offset, size,
 					 offset2, size2, checksum_fn,
 					 vgsummary->mda_checksum,
-					 checksum_only))) {
+					 checksum_only, 1))) {
 		log_error("Couldn't read volume group metadata.");
 		goto out;
 	}
@@ -130,7 +130,7 @@ struct volume_group *text_vg_import_fd(struct format_instance *fid,
 	if ((!dev && !config_file_read(cft)) ||
 	    (dev && !config_file_read_fd(cft, dev, offset, size,
 					 offset2, size2, checksum_fn, checksum,
-					 skip_parse)))
+					 skip_parse, 1)))
 		goto_out;
 
 	if (skip_parse) {

lib/format_text/import_vsn1.c

@@ -512,7 +512,7 @@ static int _read_segments(struct logical_volume *lv, const struct dm_config_node
 			count++;
 		}
 		/* FIXME Remove this restriction */
-		if ((lv->status & SNAPSHOT) && count > 1) {
+		if (lv_is_snapshot(lv) && count > 1) {
 			log_error("Only one segment permitted for snapshot");
 			return 0;
 		}
@@ -732,7 +732,7 @@ static int _read_historical_lvnames(struct format_instance *fid __attribute__((u
 
 	if (!(hlvn = hlvn->child)) {
 		log_error("Empty removed logical volume section.");
-		goto_bad;
+		goto bad;
 	}
 
 	if (!_read_id(&glv->historical->lvid.id[1], hlvn, "id")) {

lib/label/label.c

@@ -140,18 +140,18 @@ static struct labeller *_find_labeller(struct device *dev, char *buf,
 					  sector + scan_sector);
 			}
 			if (xlate64(lh->sector_xl) != sector + scan_sector) {
-				log_info("%s: Label for sector %" PRIu64
-					 " found at sector %" PRIu64
-					 " - ignoring", dev_name(dev),
-					 (uint64_t)xlate64(lh->sector_xl),
-					 sector + scan_sector);
+				log_very_verbose("%s: Label for sector %" PRIu64
+						 " found at sector %" PRIu64
+						 " - ignoring", dev_name(dev),
+						 (uint64_t)xlate64(lh->sector_xl),
+						 sector + scan_sector);
 				continue;
 			}
 			if (calc_crc(INITIAL_CRC, (uint8_t *)&lh->offset_xl, LABEL_SIZE -
 				     ((uint8_t *) &lh->offset_xl - (uint8_t *) lh)) !=
 			    xlate32(lh->crc_xl)) {
-				log_info("Label checksum incorrect on %s - "
-					 "ignoring", dev_name(dev));
+				log_very_verbose("Label checksum incorrect on %s - "
+						 "ignoring", dev_name(dev));
 				continue;
 			}
 			if (found)
@@ -243,8 +243,8 @@ int label_remove(struct device *dev)
 		}
 
 		if (wipe) {
-			log_info("%s: Wiping label at sector %" PRIu64,
-				 dev_name(dev), sector);
+			log_very_verbose("%s: Wiping label at sector %" PRIu64,
+					 dev_name(dev), sector);
 			if (!dev_write(dev, sector << SECTOR_SHIFT, LABEL_SIZE,
 				       buf)) {
 				log_error("Failed to remove label from %s at "
@@ -333,9 +333,9 @@ int label_write(struct device *dev, struct label *label)
 	if (!dev_open(dev))
 		return_0;
 
-	log_info("%s: Writing label to sector %" PRIu64 " with stored offset %"
-		 PRIu32 ".", dev_name(dev), label->sector,
-		 xlate32(lh->offset_xl));
+	log_very_verbose("%s: Writing label to sector %" PRIu64 " with stored offset %"
+			 PRIu32 ".", dev_name(dev), label->sector,
+			 xlate32(lh->offset_xl));
 	if (!dev_write(dev, label->sector << SECTOR_SHIFT, LABEL_SIZE, buf)) {
 		log_debug_devs("Failed to write label to %s", dev_name(dev));
 		r = 0;

lib/locking/lvmlockd.c

@@ -2651,7 +2651,7 @@ int lockd_lv_uses_lock(struct logical_volume *lv)
 	if (lv_is_cow(lv))
 		return 0;
 
-	if (lv->status & SNAPSHOT)
+	if (lv_is_snapshot(lv))
 		return 0;
 
 	/* FIXME: lv_is_virtual_origin ? */

lib/log/log.c

@@ -67,7 +67,7 @@ struct log_stream_item {
 	char *buffer;
 };
 
-struct log_stream {
+static struct log_stream {
 	struct log_stream_item out;
 	struct log_stream_item err;
 	struct log_stream_item report;
@@ -476,9 +476,9 @@ static void _vprint_log(int level, const char *file, int line, int dm_errno_or_c
 	int bufused, n;
 	const char *trformat;		/* Translated format string */
 	char *newbuf;
-	int use_stderr = level & _LOG_STDERR;
-	int log_once = level & _LOG_ONCE;
-	int log_bypass_report = level & _LOG_BYPASS_REPORT;
+	int use_stderr = log_stderr(level);
+	int log_once = log_once(level);
+	int log_bypass_report = log_bypass_report(level);
 	int fatal_internal_error = 0;
 	size_t msglen;
 	const char *indent_spaces = "";
@@ -489,7 +489,7 @@ static void _vprint_log(int level, const char *file, int line, int dm_errno_or_c
 	struct dm_report *orig_report;
 	int logged_via_report = 0;
 
-	level &= ~(_LOG_STDERR|_LOG_ONCE|_LOG_BYPASS_REPORT);
+	level = log_level(level);
 
 	if (_abort_on_internal_errors_env_present < 0) {
 		if ((env_str = getenv("DM_ABORT_ON_INTERNAL_ERRORS"))) {
@@ -715,8 +715,8 @@ void print_log_libdm(int level, const char *file, int line, int dm_errno_or_clas
 	 * LOG_WARN level and it's not going to stderr (so we're
 	 * printing common message that is not an error/warning).
 	*/
-	if (!(level & _LOG_STDERR) &&
-	    ((level & ~(_LOG_STDERR|_LOG_ONCE|_LOG_BYPASS_REPORT)) == _LOG_WARN))
+	if (!log_stderr(level) &&
+	    (log_level(level) == _LOG_WARN))
 		level |= _LOG_BYPASS_REPORT;
 
 	_log_stream.out.stream = report_stream;

lib/log/log.h

@@ -50,6 +50,10 @@
 #define _LOG_STDERR        0x0080 /* force things to go to stderr, even if loglevel would make them go to stdout */
 #define _LOG_ONCE          0x0100 /* downgrade to NOTICE if this has been already logged */
 #define _LOG_BYPASS_REPORT 0x0200 /* do not log through report even if report available */
+#define log_level(x)  ((x) & 0x0f)			/* obtain message level */
+#define log_stderr(x)  ((x) & _LOG_STDERR)		/* obtain stderr bit */
+#define log_once(x)  ((x) & _LOG_ONCE)			/* obtain once bit */
+#define log_bypass_report(x)  ((x) & _LOG_BYPASS_REPORT)/* obtain bypass bit */
 
 #define INTERNAL_ERROR "Internal error: "
 

lib/metadata/cache_manip.c

@@ -161,9 +161,40 @@ int update_cache_pool_params(const struct segment_type *segtype,
 	uint64_t min_meta_size;
 	uint32_t extent_size = vg->extent_size;
 	uint64_t pool_metadata_size = (uint64_t) *pool_metadata_extents * extent_size;
+	uint64_t pool_data_size = (uint64_t) pool_data_extents * extent_size;
+	uint64_t max_chunks =
+		get_default_allocation_cache_pool_max_chunks_CFG(vg->cmd, vg->profile);
+	/* min chunk size in a multiple of DM_CACHE_MIN_DATA_BLOCK_SIZE */
+	uint64_t min_chunk_size = (((pool_data_size + max_chunks - 1) / max_chunks +
+				    DM_CACHE_MIN_DATA_BLOCK_SIZE - 1) /
+				   DM_CACHE_MIN_DATA_BLOCK_SIZE) * DM_CACHE_MIN_DATA_BLOCK_SIZE;
 
-	if (!(passed_args & PASS_ARG_CHUNK_SIZE))
+	if (!(passed_args & PASS_ARG_CHUNK_SIZE)) {
 		*chunk_size = DEFAULT_CACHE_POOL_CHUNK_SIZE * 2;
+		if (*chunk_size < min_chunk_size) {
+			/*
+			 * When using more then 'standard' default,
+			 * keep user informed he might be using things in untintended direction
+			 */
+			log_print_unless_silent("Using %s chunk size instead of default %s, "
+						"so cache pool has less then " FMTu64 " chunks.",
+						display_size(vg->cmd, min_chunk_size),
+						display_size(vg->cmd, *chunk_size),
+						max_chunks);
+			*chunk_size = min_chunk_size;
+		} else
+			log_verbose("Setting chunk size to %s.",
+				    display_size(vg->cmd, *chunk_size));
+	} else if (*chunk_size < min_chunk_size) {
+		log_error("Chunk size %s is less then required minimal chunk size %s "
+			  "for a cache pool of %s size and limit " FMTu64 " chunks.",
+			  display_size(vg->cmd, *chunk_size),
+			  display_size(vg->cmd, min_chunk_size),
+			  display_size(vg->cmd, pool_data_size),
+			  max_chunks);
+		log_error("To allow use of more chunks, see setting allocation/cache_pool_max_chunks.");
+		return 0;
+	}
 
 	if (!validate_pool_chunk_size(vg->cmd, segtype, *chunk_size))
 		return_0;
@@ -204,18 +235,39 @@ int update_cache_pool_params(const struct segment_type *segtype,
  */
 int validate_lv_cache_chunk_size(struct logical_volume *pool_lv, uint32_t chunk_size)
 {
+	struct volume_group *vg = pool_lv->vg;
+	uint64_t max_chunks = get_default_allocation_cache_pool_max_chunks_CFG(vg->cmd, vg->profile);
 	uint64_t min_size = _cache_min_metadata_size(pool_lv->size, chunk_size);
+	uint64_t chunks = pool_lv->size / chunk_size;
+	int r = 1;
 
 	if (min_size > first_seg(pool_lv)->metadata_lv->size) {
-		log_error("Cannot use chunk size %s with cache pool %s. "
-			  "Minimal required size for metadata is %s.",
-			  display_size(pool_lv->vg->cmd, chunk_size),
+		log_error("Cannot use chunk size %s with cache pool %s metadata size %s.",
+			  display_size(vg->cmd, chunk_size),
 			  display_lvname(pool_lv),
-			  display_size(pool_lv->vg->cmd, min_size));
-		return 0;
+			  display_size(vg->cmd, first_seg(pool_lv)->metadata_lv->size));
+		log_error("Minimal size for cache pool %s metadata with chunk size %s would be %s.",
+			  display_lvname(pool_lv),
+			  display_size(vg->cmd, chunk_size),
+			  display_size(vg->cmd, min_size));
+		r = 0;
 	}
 
-	return 1;
+	if (chunks > max_chunks) {
+		log_error("Cannot use too small chunk size %s with cache pool %s data volume size %s.",
+			  display_size(vg->cmd, chunk_size),
+			  display_lvname(pool_lv),
+			  display_size(pool_lv->vg->cmd, pool_lv->size));
+		log_error("Maximum configured chunks for a cache pool is " FMTu64 ".",
+			  max_chunks);
+		log_error("Use smaller cache pool (<%s) or bigger cache chunk size (>=%s) or enable higher "
+			  "values in 'allocation/cache_pool_max_chunks'.",
+			  display_size(vg->cmd, chunk_size * max_chunks),
+			  display_size(vg->cmd, pool_lv->size / max_chunks));
+		r = 0;
+	}
+
+	return r;
 }
 /*
  * Validate arguments for converting origin into cached volume with given cache pool.
@@ -270,12 +322,11 @@ int validate_lv_cache_create_origin(const struct logical_volume *origin_lv)
 	if (lv_is_cache_type(origin_lv) ||
 	    lv_is_mirror_type(origin_lv) ||
 	    lv_is_thin_volume(origin_lv) || lv_is_thin_pool_metadata(origin_lv) ||
-	    lv_is_origin(origin_lv) || lv_is_merging_origin(origin_lv) ||
+	    lv_is_merging_origin(origin_lv) ||
 	    lv_is_cow(origin_lv) || lv_is_merging_cow(origin_lv) ||
-	    lv_is_external_origin(origin_lv) ||
 	    lv_is_virtual(origin_lv)) {
 		log_error("Cache is not supported with %s segment type of the original logical volume %s.",
-			  first_seg(origin_lv)->segtype->name, display_lvname(origin_lv));
+			  lvseg_name(first_seg(origin_lv)), display_lvname(origin_lv));
 		return 0;
 	}
 
@@ -331,7 +382,7 @@ int lv_cache_wait_for_clean(struct logical_volume *cache_lv, int *is_clean)
 	const struct logical_volume *lock_lv = lv_lock_holder(cache_lv);
 	struct lv_segment *cache_seg = first_seg(cache_lv);
 	struct lv_status_cache *status;
-	int cleaner_policy;
+	int cleaner_policy, writeback;
 	uint64_t dirty_blocks;
 
 	*is_clean = 0;
@@ -350,14 +401,11 @@ int lv_cache_wait_for_clean(struct logical_volume *cache_lv, int *is_clean)
 
 		cleaner_policy = !strcmp(status->cache->policy_name, "cleaner");
 		dirty_blocks = status->cache->dirty_blocks;
-
-		/* No clear policy and writeback mode means dirty */
-		if (!cleaner_policy &&
-		    (status->cache->feature_flags & DM_CACHE_FEATURE_WRITEBACK))
-			dirty_blocks++;
+		writeback = (status->cache->feature_flags & DM_CACHE_FEATURE_WRITEBACK);
 		dm_pool_destroy(status->mem);
 
-		if (!dirty_blocks)
+		/* Only clear when policy is Clear or mode != writeback */
+		if (!dirty_blocks && (cleaner_policy || !writeback))
 			break;
 
 		log_print_unless_silent("Flushing " FMTu64 " blocks for cache %s.",
@@ -368,11 +416,23 @@ int lv_cache_wait_for_clean(struct logical_volume *cache_lv, int *is_clean)
 			continue;
 		}
 
+		if (!(cache_lv->status & LVM_WRITE)) {
+			log_warn("WARNING: Dirty blocks found on read-only cache volume %s.",
+				 display_lvname(cache_lv));
+			/* TODO: can we actually clean something? */
+		}
+
 		/* Switch to cleaner policy to flush the cache */
 		cache_seg->cleaner_policy = 1;
-		/* Reaload kernel with "cleaner" policy */
+		/* Reload cache volume with "cleaner" policy */
 		if (!lv_update_and_reload_origin(cache_lv))
 			return_0;
+
+		if (!sync_local_dev_names(cache_lv->vg->cmd)) {
+			log_error("Failed to sync local devices when clearing cache volume %s.",
+				  display_lvname(cache_lv));
+			return 0;
+		}
 	}
 
 	/*
@@ -382,6 +442,12 @@ int lv_cache_wait_for_clean(struct logical_volume *cache_lv, int *is_clean)
 	if (1) {
 		if (!lv_refresh_suspend_resume(lock_lv))
 			return_0;
+
+		if (!sync_local_dev_names(cache_lv->vg->cmd)) {
+			log_error("Failed to sync local devices after final clearing of cache %s.",
+				  display_lvname(cache_lv));
+			return 0;
+		}
 	}
 
 	cache_seg->cleaner_policy = 0;
@@ -420,7 +486,7 @@ int lv_cache_remove(struct logical_volume *cache_lv)
 	}
 
 	/* Localy active volume is needed for writeback */
-	if (!lv_is_active_locally(cache_lv)) {
+	if (!lv_info(cache_lv->vg->cmd, cache_lv, 1, NULL, 0, 0)) {
 		/* Give up any remote locks */
 		if (!deactivate_lv(cache_lv->vg->cmd, cache_lv)) {
 			log_error("Cannot deactivate remotely active cache volume %s.",

lib/metadata/lv.c

@@ -1,6 +1,6 @@
 /*
  * Copyright (C) 2001-2004 Sistina Software, Inc. All rights reserved.
- * Copyright (C) 2004-2013 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2004-2016 Red Hat, Inc. All rights reserved.
  *
  * This file is part of LVM2.
  *
@@ -243,10 +243,7 @@ char *lvseg_kernel_discards_dup(struct dm_pool *mem, const struct lv_segment *se
 {
 	char *ret = NULL;
 	struct lv_with_info_and_seg_status status = {
-		.seg_status = {
-			.type = SEG_STATUS_NONE,
-			.seg = seg
-		},
+		.seg_status.type = SEG_STATUS_NONE
 	};
 
 	if (!lv_is_thin_pool(seg->lv))
@@ -255,12 +252,14 @@ char *lvseg_kernel_discards_dup(struct dm_pool *mem, const struct lv_segment *se
 	if (!(status.seg_status.mem = dm_pool_create("reporter_pool", 1024)))
 		return_NULL;
 
-	if (!(status.info_ok = lv_info_with_seg_status(seg->lv->vg->cmd, seg->lv, seg, 1, &status, 0, 0)))
+	if (!(status.info_ok = lv_info_with_seg_status(seg->lv->vg->cmd, seg, &status, 0, 0)))
 		goto_bad;
 
-	ret = lvseg_kernel_discards_dup_with_info_and_seg_status(mem, &status);
+	if (!(ret = lvseg_kernel_discards_dup_with_info_and_seg_status(mem, &status)))
+		stack;
 bad:
 	dm_pool_destroy(status.seg_status.mem);
+
 	return ret;
 }
 
@@ -366,8 +365,9 @@ dm_percent_t lvseg_percent_with_info_and_seg_status(const struct lv_with_info_an
 		else {
 			switch (type) {
 			case PERCENT_GET_DIRTY:
-				p = dm_make_percent(s->cache->dirty_blocks,
-						    s->cache->used_blocks);
+				p = (s->cache->used_blocks) ?
+					dm_make_percent(s->cache->dirty_blocks,
+							s->cache->used_blocks) : DM_PERCENT_0;
 				break;
 			case PERCENT_GET_METADATA:
 				p = dm_make_percent(s->cache->metadata_used_blocks,
@@ -406,7 +406,7 @@ dm_percent_t lvseg_percent_with_info_and_seg_status(const struct lv_with_info_an
 			/* TODO: expose highest mapped sector */
 			p = DM_PERCENT_INVALID;
 		else {
-			seg = first_seg(lvdm->lv);
+			seg = lvdm->seg_status.seg;
 			/* Pool allocates whole chunk so round-up to nearest one */
 			csize = first_seg(seg->pool_lv)->chunk_size;
 			csize = ((seg->lv->size + csize - 1) / csize) * csize;
@@ -415,8 +415,8 @@ dm_percent_t lvseg_percent_with_info_and_seg_status(const struct lv_with_info_an
 			else {
 				log_warn("WARNING: Thin volume %s maps %s while the size is only %s.",
 					 display_lvname(seg->lv),
-					 display_size(lvdm->lv->vg->cmd, s->thin->mapped_sectors),
-					 display_size(lvdm->lv->vg->cmd, csize));
+					 display_size(seg->lv->vg->cmd, s->thin->mapped_sectors),
+					 display_size(seg->lv->vg->cmd, csize));
 				/* Don't show nonsense numbers like i.e. 1000% full */
 				p = DM_PERCENT_100;
 			}
@@ -536,8 +536,8 @@ struct logical_volume *lv_origin_lv(const struct logical_volume *lv)
 
 	if (lv_is_cow(lv))
 		origin = origin_from_cow(lv);
-	else if (lv_is_cache(lv) && first_seg(lv)->origin)
-		origin = first_seg(lv)->origin;
+	else if (lv_is_cache(lv) && !lv_is_pending_delete(lv))
+		origin = seg_lv(first_seg(lv), 0);
 	else if (lv_is_thin_volume(lv) && first_seg(lv)->origin)
 		origin = first_seg(lv)->origin;
 	else if (lv_is_thin_volume(lv) && first_seg(lv)->external_lv)
@@ -971,7 +971,7 @@ int lv_mirror_image_in_sync(const struct logical_volume *lv)
 	struct lv_segment *seg = first_seg(lv);
 	struct lv_segment *mirror_seg;
 
-	if (!(lv->status & MIRROR_IMAGE) || !seg ||
+	if (!lv_is_mirror_image(lv) || !seg ||
 	    !(mirror_seg = find_mirror_seg(seg))) {
 		log_error(INTERNAL_ERROR "Cannot find mirror segment.");
 		return 0;
@@ -987,7 +987,6 @@ int lv_mirror_image_in_sync(const struct logical_volume *lv)
 int lv_raid_image_in_sync(const struct logical_volume *lv)
 {
 	unsigned s;
-	dm_percent_t percent;
 	char *raid_health;
 	struct lv_segment *seg, *raid_seg = NULL;
 
@@ -1017,12 +1016,6 @@ int lv_raid_image_in_sync(const struct logical_volume *lv)
 		return 0;
 	}
 
-	if (!lv_raid_percent(raid_seg->lv, &percent))
-		return_0;
-
-	if (percent == DM_PERCENT_100)
-		return 1;
-
 	/* Find out which sub-LV this is. */
 	for (s = 0; s < raid_seg->area_count; s++)
 		if (seg_lv(raid_seg, s) == lv)
@@ -1190,7 +1183,8 @@ char *lv_attr_dup_with_info_and_seg_status(struct dm_pool *mem, const struct lv_
 	if (lv_is_historical(lv)) {
 		repstr[4] = 'h';
 		repstr[5] = '-';
-	} else if (!activation() || !lvdm->info_ok) {
+	} else if (!activation() || !lvdm->info_ok ||
+		   (lvdm->seg_status.type == SEG_STATUS_UNKNOWN)) {
 		repstr[4] = 'X';		/* Unknown */
 		repstr[5] = 'X';		/* Unknown */
 	} else if (lvdm->info.exists) {
@@ -1222,8 +1216,10 @@ char *lv_attr_dup_with_info_and_seg_status(struct dm_pool *mem, const struct lv_
 		/* 'c' when cache/thin-pool is active with needs_check flag
 		 * 'C' for suspend */
 		if ((lv_is_thin_pool(lv) &&
+		     (lvdm->seg_status.type == SEG_STATUS_THIN_POOL) &&
 		     lvdm->seg_status.thin_pool->needs_check) ||
 		    (lv_is_cache(lv) &&
+		     (lvdm->seg_status.type == SEG_STATUS_CACHE) &&
 		     lvdm->seg_status.cache->needs_check))
 			repstr[4] = lvdm->info.suspended ? 'C' : 'c';
 
@@ -1265,6 +1261,10 @@ char *lv_attr_dup_with_info_and_seg_status(struct dm_pool *mem, const struct lv_
 		repstr[7] = '-';
 
 	repstr[8] = '-';
+	/* TODO: also convert raid health
+	 * lv_is_raid_type() is to wide
+	 * NOTE: snapshot origin is 'mostly' showing it's layered status
+	 */
 	if (lv_is_partial(lv))
 		repstr[8] = 'p';
 	else if (lv_is_raid_type(lv)) {
@@ -1278,31 +1278,23 @@ char *lv_attr_dup_with_info_and_seg_status(struct dm_pool *mem, const struct lv_
 				repstr[8] = 'm';  /* RAID has 'm'ismatches */
 		} else if (lv->status & LV_WRITEMOSTLY)
 			repstr[8] = 'w';  /* sub-LV has 'w'ritemostly */
-	} else if (lv_is_cache(lv) &&
-		   (lvdm->seg_status.type != SEG_STATUS_NONE)) {
-		if (lvdm->seg_status.type == SEG_STATUS_UNKNOWN)
-			repstr[8] = 'X'; /* Unknown */
-		else if (lvdm->seg_status.cache->fail)
+	} else if (lvdm->seg_status.type == SEG_STATUS_CACHE) {
+		if (lvdm->seg_status.cache->fail)
 			repstr[8] = 'F';
 		else if (lvdm->seg_status.cache->read_only)
 			repstr[8] = 'M';
-	} else if (lv_is_thin_pool(lv) &&
-		   (lvdm->seg_status.type != SEG_STATUS_NONE)) {
-		if (lvdm->seg_status.type == SEG_STATUS_UNKNOWN)
-			repstr[8] = 'X'; /* Unknown */
-		else if (lvdm->seg_status.thin_pool->fail)
+	} else if (lvdm->seg_status.type == SEG_STATUS_THIN_POOL) {
+		if (lvdm->seg_status.thin_pool->fail)
 			repstr[8] = 'F';
 		else if (lvdm->seg_status.thin_pool->out_of_data_space)
 			repstr[8] = 'D';
 		else if (lvdm->seg_status.thin_pool->read_only)
 			repstr[8] = 'M';
-	} else if (lv_is_thin_volume(lv) &&
-		   (lvdm->seg_status.type != SEG_STATUS_NONE)) {
-		if (lvdm->seg_status.type == SEG_STATUS_UNKNOWN)
-			repstr[8] = 'X'; /* Unknown */
-		else if (lvdm->seg_status.thin->fail)
+	} else if (lvdm->seg_status.type == SEG_STATUS_THIN) {
+		if (lvdm->seg_status.thin->fail)
 			repstr[8] = 'F';
-	}
+	} else if (lvdm->seg_status.type == SEG_STATUS_UNKNOWN)
+		repstr[8] = 'X'; /* Unknown */
 
 	if (lv->status & LV_ACTIVATION_SKIP)
 		repstr[9] = 'k';
@@ -1319,13 +1311,12 @@ char *lv_attr_dup(struct dm_pool *mem, const struct logical_volume *lv)
 	char *ret = NULL;
 	struct lv_with_info_and_seg_status status = {
 		.seg_status.type = SEG_STATUS_NONE,
-		.lv = lv
 	};
 
 	if (!(status.seg_status.mem = dm_pool_create("reporter_pool", 1024)))
 		return_0;
 
-	if (!(status.info_ok = lv_info_with_seg_status(lv->vg->cmd, lv, first_seg(lv), 1, &status, 1, 1)))
+	if (!(status.info_ok = lv_info_with_seg_status(lv->vg->cmd, first_seg(lv), &status, 1, 1)))
 		goto_bad;
 
 	ret = lv_attr_dup_with_info_and_seg_status(mem, &status);
@@ -1424,6 +1415,7 @@ int lv_active_change(struct cmd_context *cmd, struct logical_volume *lv,
 		     enum activation_change activate, int needs_exclusive)
 {
 	const char *ay_with_mode = NULL;
+	struct lv_segment *seg = first_seg(lv);
 
 	if (activate == CHANGE_ASY)
 		ay_with_mode = "sh";
@@ -1460,6 +1452,9 @@ deactivate:
 		break;
 	case CHANGE_ALY:
 	case CHANGE_AAY:
+		if (!raid4_is_supported(cmd, seg->segtype))
+			goto no_raid4;
+
 		if (needs_exclusive || _lv_is_exclusive(lv)) {
 			log_verbose("Activating logical volume %s exclusively locally.",
 				    display_lvname(lv));
@@ -1474,6 +1469,9 @@ deactivate:
 		break;
 	case CHANGE_AEY:
 exclusive:
+		if (!raid4_is_supported(cmd, seg->segtype))
+			goto no_raid4;
+
 		log_verbose("Activating logical volume %s exclusively.",
 			    display_lvname(lv));
 		if (!activate_lv_excl(cmd, lv))
@@ -1482,6 +1480,9 @@ exclusive:
 	case CHANGE_ASY:
 	case CHANGE_AY:
 	default:
+		if (!raid4_is_supported(cmd, seg->segtype))
+			goto no_raid4;
+
 		if (needs_exclusive || _lv_is_exclusive(lv))
 			goto exclusive;
 		log_verbose("Activating logical volume %s.", display_lvname(lv));
@@ -1494,6 +1495,10 @@ exclusive:
 		log_error("Failed to unlock logical volume %s.", display_lvname(lv));
 
 	return 1;
+
+no_raid4:
+	log_error("Failed to activate %s LV %s", lvseg_name(seg), display_lvname(lv));
+	return 0;
 }
 
 char *lv_active_dup(struct dm_pool *mem, const struct logical_volume *lv)
@@ -1546,14 +1551,21 @@ const struct logical_volume *lv_lock_holder(const struct logical_volume *lv)
 	if (lv_is_cow(lv))
 		return lv_lock_holder(origin_from_cow(lv));
 
-	if (lv_is_thin_pool(lv))
-		/* Find any active LV from the pool */
-		dm_list_iterate_items(sl, &lv->segs_using_this_lv)
-			if (lv_is_active(sl->seg->lv)) {
-				log_debug_activation("Thin volume %s is active.",
-						     display_lvname(lv));
-				return sl->seg->lv;
-			}
+	if (lv_is_thin_pool(lv) ||
+	    lv_is_external_origin(lv)) {
+		/* FIXME: Ensure cluster keeps thin-pool active exlusively.
+		 * External origin can be activated on more nodes (depends on type).
+		 */
+		if (!lv_is_active(lv))
+			/* Find any active LV from the pool or external origin */
+			dm_list_iterate_items(sl, &lv->segs_using_this_lv)
+				if (lv_is_active(sl->seg->lv)) {
+					log_debug_activation("Thin volume %s is active.",
+							     display_lvname(lv));
+					return sl->seg->lv;
+				}
+		return lv;
+	}
 
 	/* RAID changes visibility of splitted LVs but references them still as leg/meta */
 	if ((lv_is_raid_image(lv) || lv_is_raid_metadata(lv)) && lv_is_visible(lv))
@@ -1566,9 +1578,6 @@ const struct logical_volume *lv_lock_holder(const struct logical_volume *lv)
 		    lv_is_thin_volume(sl->seg->lv) &&
 		    first_seg(lv)->pool_lv == sl->seg->pool_lv)
 			continue; /* Skip thin snaphost */
-		if (lv_is_external_origin(lv) &&
-		    lv_is_thin_volume(sl->seg->lv))
-			continue; /* Skip external origin */
 		if (lv_is_pending_delete(sl->seg->lv))
 			continue; /* Skip deleted LVs */
 		return lv_lock_holder(sl->seg->lv);

lib/metadata/merge.c

@@ -1,6 +1,6 @@
 /*
  * Copyright (C) 2001-2004 Sistina Software, Inc. All rights reserved.
- * Copyright (C) 2004-2016 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2004-2017 Red Hat, Inc. All rights reserved.
  *
  * This file is part of LVM2.
  *
@@ -71,6 +71,13 @@ int lv_merge_segments(struct logical_volume *lv)
 	if (error_count++ > ERROR_MAX)	\
 		goto out
 
+#define seg_error(msg) { \
+		log_error("LV %s, segment %u invalid: %s for %s segment.", \
+			  seg->lv->name, seg_count, (msg), lvseg_name(seg)); \
+		if ((*error_count)++ > ERROR_MAX) \
+			return; \
+	}
+
 /*
  * RAID segment property checks.
  *
@@ -141,7 +148,15 @@ static void _check_raid1_seg(struct lv_segment *seg, int *error_count)
 static void _check_raid45610_seg(struct lv_segment *seg, int *error_count)
 {
 	/* Checks applying to any raid4/5/6/10 */
-	if (!seg->meta_areas)
+	/*
+	 * Allow raid4 + raid5_n to get activated w/o metadata.
+	 *
+	 * This is mandatory during conversion between them,
+	 * because switching the dedicated parity SubLVs
+	 * beginning <-> end changes the roles of all SubLVs
+	 * which the kernel would reject.
+	 */
+	if (!(seg_is_raid4(seg) || seg_is_raid5_n(seg)) && !seg->meta_areas)
 		raid_seg_error("no meta areas");
 	if (!seg->stripe_size)
 		raid_seg_error("zero stripe size");
@@ -188,44 +203,10 @@ static void _check_non_raid_seg_members(struct lv_segment *seg, int *error_count
 {
 	if (seg->origin) /* snap and thin */
 		raid_seg_error("non-zero origin LV");
-	if (seg->indirect_origin) /* thin */
-		raid_seg_error("non-zero indirect_origin LV");
-	if (seg->merge_lv) /* thin */
-		raid_seg_error("non-zero merge LV");
 	if (seg->cow) /* snap */
 		raid_seg_error("non-zero cow LV");
 	if (!dm_list_empty(&seg->origin_list)) /* snap */
 		raid_seg_error("non-zero origin_list");
-	if (seg->log_lv)
-		raid_seg_error("non-zero log LV");
-	if (seg->segtype_private)
-		raid_seg_error("non-zero segtype_private");
-	/* thin members */
-	if (seg->metadata_lv)
-		raid_seg_error("non-zero metadata LV");
-	if (seg->transaction_id)
-		raid_seg_error("non-zero transaction_id");
-	if (seg->zero_new_blocks)
-		raid_seg_error("non-zero zero_new_blocks");
-	if (seg->discards)
-		raid_seg_error("non-zero discards");
-	if (!dm_list_empty(&seg->thin_messages))
-		raid_seg_error("non-zero thin_messages list");
-	if (seg->external_lv)
-		raid_seg_error("non-zero external LV");
-	if (seg->pool_lv)
-		raid_seg_error("non-zero pool LV");
-	if (seg->device_id)
-		raid_seg_error("non-zero device_id");
-	/* cache members */
-	if (seg->cache_mode)
-		raid_seg_error("non-zero cache_mode");
-	if (seg->policy_name)
-		raid_seg_error("non-zero policy_name");
-	if (seg->policy_settings)
-		raid_seg_error("non-zero policy_settings");
-	if (seg->cleaner_policy)
-		raid_seg_error("non-zero cleaner_policy");
 	/* replicator members (deprecated) */
 	if (seg->replicator)
 		raid_seg_error("non-zero replicator");
@@ -249,9 +230,6 @@ static void _check_raid_seg(struct lv_segment *seg, int *error_count)
 	uint32_t area_len, s;
 
 	/* General checks applying to all RAIDs */
-	if (!seg_is_raid(seg))
-		raid_seg_error("erroneous RAID check");
-
 	if (!seg->area_count)
 		raid_seg_error("zero area count");
 
@@ -276,9 +254,6 @@ static void _check_raid_seg(struct lv_segment *seg, int *error_count)
 				return;
 	}
 
-	if (seg->chunk_size)
-		raid_seg_error_val("non-zero chunk_size", seg->chunk_size);
-
 	/* FIXME: should we check any non-RAID segment struct members at all? */
 	_check_non_raid_seg_members(seg, error_count);
 
@@ -329,6 +304,169 @@ static void _check_raid_seg(struct lv_segment *seg, int *error_count)
 }
 /* END: RAID segment property checks. */
 
+static void _check_lv_segment(struct logical_volume *lv, struct lv_segment *seg,
+			      unsigned seg_count, int *error_count)
+{
+	struct lv_segment *seg2;
+
+	if (lv_is_mirror_image(lv) &&
+	    (!(seg2 = find_mirror_seg(seg)) || !seg_is_mirrored(seg2)))
+		seg_error("mirror image is not mirrored");
+
+	if (seg_is_cache(seg)) {
+		if (!lv_is_cache(lv))
+			seg_error("is not flagged as cache LV");
+
+		if (!seg->pool_lv) {
+			seg_error("is missing cache pool LV");
+		} else if (!lv_is_cache_pool(seg->pool_lv))
+			seg_error("is not referencing cache pool LV");
+	} else { /* !cache */
+		if (seg->cleaner_policy)
+			seg_error("sets cleaner_policy");
+	}
+
+	if (seg_is_cache_pool(seg)) {
+		if (!dm_list_empty(&seg->lv->segs_using_this_lv)) {
+			switch (seg->cache_mode) {
+			case CACHE_MODE_WRITETHROUGH:
+			case CACHE_MODE_WRITEBACK:
+			case CACHE_MODE_PASSTHROUGH:
+				break;
+			default:
+				seg_error("has invalid cache's feature flag")
+			}
+			if (!seg->policy_name)
+				seg_error("is missing cache policy name");
+		}
+	} else { /* !cache_pool */
+		if (seg->cache_mode)
+			seg_error("sets cache mode");
+		if (seg->policy_name)
+			seg_error("sets policy name");
+		if (seg->policy_settings)
+			seg_error("sets policy settings");
+	}
+
+	if (!seg_can_error_when_full(seg) && lv_is_error_when_full(lv))
+		seg_error("does not support flag ERROR_WHEN_FULL.");
+
+	if (seg_is_mirrored(seg)) {
+		/* Check mirror log - which is attached to the mirrored seg */
+		if (seg->log_lv) {
+			if (!lv_is_mirror_log(seg->log_lv))
+				seg_error("log LV is not a mirror log");
+
+			if (!(seg2 = first_seg(seg->log_lv)) || (find_mirror_seg(seg2) != seg))
+				seg_error("log LV does not point back to mirror segment");
+		}
+	} else { /* !mirrored */
+		if (seg->log_lv) {
+			if (lv_is_raid_image(lv))
+				seg_error("log LV is not a mirror log or a RAID image");
+		}
+	}
+
+	if (seg_is_raid(seg))
+		_check_raid_seg(seg, error_count);
+
+	if (seg_is_pool(seg)) {
+		if ((seg->area_count != 1) || (seg_type(seg, 0) != AREA_LV)) {
+			seg_error("is missing a pool data LV");
+		} else if (!(seg2 = first_seg(seg_lv(seg, 0))) || (find_pool_seg(seg2) != seg))
+			seg_error("data LV does not refer back to pool LV");
+
+		if (!seg->metadata_lv) {
+			seg_error("is missing a pool metadata LV");
+		} else if (!(seg2 = first_seg(seg->metadata_lv)) || (find_pool_seg(seg2) != seg))
+			seg_error("metadata LV does not refer back to pool LV");
+
+		if (!validate_pool_chunk_size(lv->vg->cmd, seg->segtype, seg->chunk_size))
+			seg_error("has invalid chunk size.");
+	} else { /* !thin_pool && !cache_pool */
+		if (seg->metadata_lv)
+			seg_error("must not have pool metadata LV set");
+	}
+
+	if (seg_is_thin_pool(seg)) {
+		if (!lv_is_thin_pool(lv))
+			seg_error("is not flagged as thin pool LV");
+
+		if (lv_is_thin_volume(lv))
+			seg_error("is a thin volume that must not contain thin pool segment");
+	} else { /* !thin_pool */
+		if (seg->zero_new_blocks)
+			seg_error("sets zero_new_blocks");
+		if (seg->discards)
+			seg_error("sets discards");
+		if (!dm_list_empty(&seg->thin_messages))
+			seg_error("sets thin_messages list");
+	}
+
+	if (seg_is_thin_volume(seg)) {
+		if (!lv_is_thin_volume(lv))
+			seg_error("is not flagged as thin volume LV");
+
+		if (lv_is_thin_pool(lv))
+			seg_error("is a thin pool that must not contain thin volume segment");
+
+		if (!seg->pool_lv) {
+			seg_error("is missing thin pool LV");
+		} else if (!lv_is_thin_pool(seg->pool_lv))
+			seg_error("is not referencing thin pool LV");
+
+		if (seg->device_id > DM_THIN_MAX_DEVICE_ID)
+			seg_error("has too large device id");
+
+		if (seg->external_lv &&
+		    !lv_is_external_origin(seg->external_lv))
+			seg_error("external LV is not flagged as a external origin LV");
+
+		if (seg->merge_lv) {
+			if (!lv_is_thin_volume(seg->merge_lv))
+				seg_error("merge LV is not flagged as a thin LV");
+
+			if (!lv_is_merging_origin(seg->merge_lv))
+				seg_error("merge LV is not flagged as merging");
+		}
+	} else { /* !thin */
+		if (seg->device_id)
+			seg_error("sets device_id");
+		if (seg->external_lv)
+			seg_error("sets external LV");
+		if (seg->merge_lv)
+			seg_error("sets merge LV");
+		if (seg->indirect_origin)
+			seg_error("sets indirect_origin LV");
+	}
+
+	/* Some multi-seg vars excluded here */
+	if (!seg_is_cache(seg) &&
+	    !seg_is_thin_volume(seg)) {
+		if (seg->pool_lv)
+			seg_error("sets pool LV");
+	}
+
+	if (!seg_is_pool(seg) &&
+	    /* FIXME: format_pool/import_export.c  _add_linear_seg() sets chunk_size */
+	    !seg_is_linear(seg) &&
+	    !seg_is_snapshot(seg)) {
+		if (seg->chunk_size)
+			seg_error("sets chunk_size");
+	}
+
+	if (!seg_is_thin_pool(seg) &&
+	    !seg_is_thin_volume(seg)) {
+		if (seg->transaction_id)
+			seg_error("sets transaction_id");
+	}
+
+	if (!seg_unknown(seg)) {
+		if (seg->segtype_private)
+			seg_error("set segtype_private");
+	}
+}
+
 /*
  * Verify that an LV's segments are consecutive, complete and don't overlap.
  */
@@ -336,7 +474,7 @@ int check_lv_segments(struct logical_volume *lv, int complete_vg)
 {
 	struct lv_segment *seg, *seg2;
 	uint32_t le = 0;
-	unsigned seg_count = 0, seg_found;
+	unsigned seg_count = 0, seg_found, external_lv_found = 0;
 	uint32_t area_multiplier, s;
 	struct seg_list *sl;
 	struct glv_list *glvl;
@@ -344,68 +482,15 @@ int check_lv_segments(struct logical_volume *lv, int complete_vg)
 	struct replicator_site *rsite;
 	struct replicator_device *rdev;
 
-	/* Check LV flags match first segment type */
-	if (complete_vg) {
-		if (lv_is_thin_volume(lv)) {
-			if (dm_list_size(&lv->segments) != 1) {
-				log_error("LV %s is thin volume without exactly one segment.",
-					  lv->name);
-				inc_error_count;
-			} else if (!seg_is_thin_volume(first_seg(lv))) {
-				log_error("LV %s is thin volume without first thin volume segment.",
-					  lv->name);
-				inc_error_count;
-			}
-		}
-
-		if (lv_is_thin_pool(lv)) {
-			if (dm_list_size(&lv->segments) != 1) {
-				log_error("LV %s is thin pool volume without exactly one segment.",
-					  lv->name);
-				inc_error_count;
-			} else if (!seg_is_thin_pool(first_seg(lv))) {
-				log_error("LV %s is thin pool without first thin pool segment.",
-					  lv->name);
-				inc_error_count;
-			}
-		}
-
-		if (lv_is_pool_data(lv) &&
-		    (!(seg2 = first_seg(lv)) || !(seg2 = find_pool_seg(seg2)) ||
-		     seg2->area_count != 1 || seg_type(seg2, 0) != AREA_LV ||
-		     seg_lv(seg2, 0) != lv)) {
-			log_error("LV %s: segment 1 pool data LV does not point back to same LV",
-				  lv->name);
-			inc_error_count;
-		}
-
-		if (lv_is_pool_metadata(lv)) {
-			if (!(seg2 = first_seg(lv)) || !(seg2 = find_pool_seg(seg2)) ||
-			    seg2->metadata_lv != lv) {
-				log_error("LV %s: segment 1 pool metadata LV does not point back to same LV",
-					  lv->name);
-				inc_error_count;
-			}
-			if (lv_is_thin_pool_metadata(lv) &&
-			    !strstr(lv->name, "_tmeta")) {
-				log_error("LV %s: thin pool metadata LV does not use _tmeta",
-					  lv->name);
-				inc_error_count;
-			} else if (lv_is_cache_pool_metadata(lv) &&
-				   !strstr(lv->name, "_cmeta")) {
-				log_error("LV %s: cache pool metadata LV does not use _cmeta",
-					  lv->name);
-				inc_error_count;
-			}
-		}
-	}
-
 	dm_list_iterate_items(seg, &lv->segments) {
 		seg_count++;
 
-		if (complete_vg && seg_is_raid(seg))
-			 _check_raid_seg(seg, &error_count);
-		
+		if (seg->lv != lv) {
+			log_error("LV %s invalid: segment %u is referencing different LV.",
+				  lv->name, seg_count);
+			inc_error_count;
+		}
+
 		if (seg->le != le) {
 			log_error("LV %s invalid: segment %u should begin at "
 				  "LE %" PRIu32 " (found %" PRIu32 ").",
@@ -423,186 +508,6 @@ int check_lv_segments(struct logical_volume *lv, int complete_vg)
 			inc_error_count;
 		}
 
-		if (lv_is_error_when_full(lv) &&
-		    !seg_can_error_when_full(seg)) {
-			log_error("LV %s: segment %u (%s) does not support flag "
-				  "ERROR_WHEN_FULL.", lv->name, seg_count, seg->segtype->name);
-			inc_error_count;
-		}
-
-		if (complete_vg && seg->log_lv &&
-		    !seg_is_mirrored(seg) && !(seg->status & RAID_IMAGE)) {
-			log_error("LV %s: segment %u log LV %s is not a "
-				  "mirror log or a RAID image",
-				  lv->name, seg_count, seg->log_lv->name);
-			inc_error_count;
-		}
-
-		/*
-		 * Check mirror log - which is attached to the mirrored seg
-		 */
-		if (complete_vg && seg->log_lv && seg_is_mirrored(seg)) {
-			if (!lv_is_mirror_log(seg->log_lv)) {
-				log_error("LV %s: segment %u log LV %s is not "
-					  "a mirror log",
-					  lv->name, seg_count, seg->log_lv->name);
-				inc_error_count;
-			}
-
-			if (!(seg2 = first_seg(seg->log_lv)) ||
-			    find_mirror_seg(seg2) != seg) {
-				log_error("LV %s: segment %u log LV does not "
-					  "point back to mirror segment",
-					  lv->name, seg_count);
-				inc_error_count;
-			}
-		}
-
-		if (complete_vg && seg->status & MIRROR_IMAGE) {
-			if (!find_mirror_seg(seg) ||
-			    !seg_is_mirrored(find_mirror_seg(seg))) {
-				log_error("LV %s: segment %u mirror image "
-					  "is not mirrored",
-					  lv->name, seg_count);
-				inc_error_count;
-			}
-		}
-
-		/* Check the various thin segment types */
-		if (complete_vg) {
-			if (seg_is_thin_pool(seg)) {
-				if (!lv_is_thin_pool(lv)) {
-					log_error("LV %s is missing thin pool flag for segment %u",
-						  lv->name, seg_count);
-					inc_error_count;
-				}
-
-				if (lv_is_thin_volume(lv)) {
-					log_error("LV %s is a thin volume that must not contain thin pool segment %u",
-						  lv->name, seg_count);
-					inc_error_count;
-				}
-
-			}
-			if (seg_is_cache_pool(seg) &&
-			    !dm_list_empty(&seg->lv->segs_using_this_lv)) {
-				switch (seg->cache_mode) {
-				case CACHE_MODE_WRITETHROUGH:
-				case CACHE_MODE_WRITEBACK:
-				case CACHE_MODE_PASSTHROUGH:
-					break;
-				default:
-					log_error("LV %s has invalid cache's feature flag.",
-						  lv->name);
-					inc_error_count;
-				}
-				if (!seg->policy_name) {
-					log_error("LV %s is missing cache policy name.", lv->name);
-					inc_error_count;
-				}
-			}
-			if (seg_is_pool(seg)) {
-				if (seg->area_count != 1 ||
-				    seg_type(seg, 0) != AREA_LV) {
-					log_error("LV %s: %s segment %u is missing a pool data LV",
-						  lv->name, seg->segtype->name, seg_count);
-					inc_error_count;
-				} else if (!(seg2 = first_seg(seg_lv(seg, 0))) || find_pool_seg(seg2) != seg) {
-					log_error("LV %s: %s segment %u data LV does not refer back to pool LV",
-						  lv->name, seg->segtype->name, seg_count);
-					inc_error_count;
-				}
-
-				if (!seg->metadata_lv) {
-					log_error("LV %s: %s segment %u is missing a pool metadata LV",
-						  lv->name, seg->segtype->name, seg_count);
-					inc_error_count;
-				} else if (!(seg2 = first_seg(seg->metadata_lv)) ||
-					   find_pool_seg(seg2) != seg) {
-					log_error("LV %s: %s segment %u metadata LV does not refer back to pool LV",
-						  lv->name, seg->segtype->name, seg_count);
-					inc_error_count;
-				}
-
-				if (!validate_pool_chunk_size(lv->vg->cmd, seg->segtype, seg->chunk_size)) {
-					log_error("LV %s: %s segment %u has invalid chunk size %u.",
-						  lv->name, seg->segtype->name, seg_count, seg->chunk_size);
-					inc_error_count;
-				}
-			} else {
-				if (seg->metadata_lv) {
-					log_error("LV %s: segment %u must not have pool metadata LV set",
-						  lv->name, seg_count);
-					inc_error_count;
-				}
-			}
-
-			if (seg_is_thin_volume(seg)) {
-				if (!lv_is_thin_volume(lv)) {
-					log_error("LV %s is missing thin volume flag for segment %u",
-						  lv->name, seg_count);
-					inc_error_count;
-				}
-
-				if (lv_is_thin_pool(lv)) {
-					log_error("LV %s is a thin pool that must not contain thin volume segment %u",
-						  lv->name, seg_count);
-					inc_error_count;
-				}
-
-				if (!seg->pool_lv) {
-					log_error("LV %s: segment %u is missing thin pool LV",
-						  lv->name, seg_count);
-					inc_error_count;
-				} else if (!lv_is_thin_pool(seg->pool_lv)) {
-					log_error("LV %s: thin volume segment %u pool LV is not flagged as a pool LV",
-						  lv->name, seg_count);
-					inc_error_count;
-				}
-
-				if (seg->device_id > DM_THIN_MAX_DEVICE_ID) {
-					log_error("LV %s: thin volume segment %u has too large device id %u",
-						  lv->name, seg_count, seg->device_id);
-					inc_error_count;
-				}
-				if (seg->external_lv && (seg->external_lv->status & LVM_WRITE)) {
-					log_error("LV %s: external origin %s is writable.",
-						  lv->name, seg->external_lv->name);
-					inc_error_count;
-				}
-
-				if (seg->merge_lv) {
-					if (!lv_is_thin_volume(seg->merge_lv)) {
-						log_error("LV %s: thin volume segment %u merging LV %s is not flagged as a thin LV",
-							  lv->name, seg_count, seg->merge_lv->name);
-						inc_error_count;
-					}
-					if (!lv_is_merging_origin(seg->merge_lv)) {
-						log_error("LV %s: merging LV %s is not flagged as merging.",
-							  lv->name, seg->merge_lv->name);
-						inc_error_count;
-					}
-				}
-			} else if (seg_is_cache(seg)) {
-				if (!lv_is_cache(lv)) {
-					log_error("LV %s is missing cache flag for segment %u",
-						  lv->name, seg_count);
-					inc_error_count;
-				}
-				if (!seg->pool_lv) {
-					log_error("LV %s: segment %u is missing cache_pool LV",
-						  lv->name, seg_count);
-					inc_error_count;
-				}
-			} else {
-				if (seg->pool_lv) {
-					log_error("LV %s: segment %u must not have pool LV set",
-						  lv->name, seg_count);
-					inc_error_count;
-				}
-			}
-		}
-
 		if (seg_is_snapshot(seg)) {
 			if (seg->cow && seg->cow == seg->origin) {
 				log_error("LV %s: segment %u has same LV %s for "
@@ -615,6 +520,9 @@ int check_lv_segments(struct logical_volume *lv, int complete_vg)
 		if (seg_is_replicator(seg) && !check_replicator_segment(seg))
 			inc_error_count;
 
+		if (complete_vg)
+			_check_lv_segment(lv, seg, seg_count, &error_count);
+
 		for (s = 0; s < seg->area_count; s++) {
 			if (seg_type(seg, s) == AREA_UNASSIGNED) {
 				log_error("LV %s: segment %u has unassigned "
@@ -696,6 +604,12 @@ int check_lv_segments(struct logical_volume *lv, int complete_vg)
 		le += seg->len;
 	}
 
+	if (le != lv->le_count) {
+		log_error("LV %s: inconsistent LE count %u != %u",
+			  lv->name, le, lv->le_count);
+		inc_error_count;
+	}
+
 	dm_list_iterate_items(sl, &lv->segs_using_this_lv) {
 		seg = sl->seg;
 		seg_found = 0;
@@ -704,7 +618,7 @@ int check_lv_segments(struct logical_volume *lv, int complete_vg)
 				continue;
 			if (lv == seg_lv(seg, s))
 				seg_found++;
-			if (seg_is_raid_with_meta(seg) && (lv == seg_metalv(seg, s)))
+			if (seg->meta_areas && seg_is_raid_with_meta(seg) && (lv == seg_metalv(seg, s)))
 				seg_found++;
 		}
 		if (seg_is_replicator_dev(seg)) {
@@ -756,6 +670,10 @@ int check_lv_segments(struct logical_volume *lv, int complete_vg)
 				  lv->name);
 			inc_error_count;
 		}
+
+		/* Validation of external origin counter */
+		if (seg->external_lv == lv)
+			external_lv_found++;
 	}
 
 	dm_list_iterate_items(glvl, &lv->indirect_glvs) {
@@ -777,10 +695,51 @@ int check_lv_segments(struct logical_volume *lv, int complete_vg)
 		}
 	}
 
-	if (le != lv->le_count) {
-		log_error("LV %s: inconsistent LE count %u != %u",
-			  lv->name, le, lv->le_count);
-		inc_error_count;
+	/* Check LV flags match first segment type */
+	if (complete_vg) {
+		if ((seg_count != 1) &&
+		    (lv_is_cache(lv) ||
+		     lv_is_cache_pool(lv) ||
+		     lv_is_raid(lv) ||
+		     lv_is_snapshot(lv) ||
+		     lv_is_thin_pool(lv) ||
+		     lv_is_thin_volume(lv))) {
+			log_error("LV %s must have exactly one segment.",
+				  lv->name);
+			inc_error_count;
+		}
+
+		if (lv_is_pool_data(lv) &&
+		    (!(seg2 = first_seg(lv)) || !(seg2 = find_pool_seg(seg2)) ||
+		     seg2->area_count != 1 || seg_type(seg2, 0) != AREA_LV ||
+		     seg_lv(seg2, 0) != lv)) {
+			log_error("LV %s: segment 1 pool data LV does not point back to same LV",
+				  lv->name);
+			inc_error_count;
+		}
+
+		if (lv_is_thin_pool_metadata(lv) && !strstr(lv->name, "_tmeta")) {
+			log_error("LV %s: thin pool metadata LV does not use _tmeta.",
+				  lv->name);
+			inc_error_count;
+		} else if (lv_is_cache_pool_metadata(lv) && !strstr(lv->name, "_cmeta")) {
+			log_error("LV %s: cache pool metadata LV does not use _cmeta.",
+				  lv->name);
+			inc_error_count;
+		}
+
+		if (lv_is_external_origin(lv)) {
+			if (lv->external_count != external_lv_found) {
+				log_error("LV %s: external origin count does not match.",
+					  lv->name);
+				inc_error_count;
+			}
+			if (lv->status & LVM_WRITE) {
+				log_error("LV %s: external origin cant't be writable.",
+					  lv->name);
+				inc_error_count;
+			}
+		}
 	}
 
 out:

lib/metadata/metadata-exported.h

@@ -198,7 +198,7 @@
 #define lv_is_partial(lv)	(((lv)->status & PARTIAL_LV) ? 1 : 0)
 #define lv_is_virtual(lv)	(((lv)->status & VIRTUAL) ? 1 : 0)
 #define lv_is_merging(lv)	(((lv)->status & MERGING) ? 1 : 0)
-#define lv_is_merging_origin(lv) (lv_is_merging(lv))
+#define lv_is_merging_origin(lv) (lv_is_merging(lv) && (lv)->snapshot)
 #define lv_is_snapshot(lv)	(((lv)->status & SNAPSHOT) ? 1 : 0)
 #define lv_is_converting(lv)	(((lv)->status & CONVERTING) ? 1 : 0)
 #define lv_is_external_origin(lv)	(((lv)->external_count > 0) ? 1 : 0)
@@ -226,6 +226,7 @@
 
 #define lv_is_raid(lv)		(((lv)->status & RAID) ? 1 : 0)
 #define lv_is_raid_image(lv)	(((lv)->status & RAID_IMAGE) ? 1 : 0)
+#define lv_is_raid_image_with_tracking(lv)	((lv_is_raid_image(lv) && !((lv)->status & LVM_WRITE)) ? 1 : 0)
 #define lv_is_raid_metadata(lv)	(((lv)->status & RAID_META) ? 1 : 0)
 #define lv_is_raid_type(lv)	(((lv)->status & (RAID | RAID_IMAGE | RAID_META)) ? 1 : 0)
 
@@ -945,6 +946,9 @@ struct lvcreate_params {
 	uint32_t chunk_size; /* snapshot */
 	uint32_t region_size; /* mirror */
 
+	unsigned stripes_supplied; /* striped */
+	unsigned stripe_size_supplied; /* striped */
+
 	uint32_t mirrors; /* mirror */
 
 	uint32_t min_recovery_rate; /* RAID */
@@ -1063,9 +1067,16 @@ struct lv_segment *get_only_segment_using_this_lv(const struct logical_volume *l
 * Useful functions for managing snapshots.
 */
 int lv_is_origin(const struct logical_volume *lv);
+#define lv_is_thick_origin lv_is_origin
+
 int lv_is_thin_origin(const struct logical_volume *lv, unsigned *snap_count);
-int lv_is_cache_origin(const struct logical_volume *lv);
+int lv_is_thin_snapshot(const struct logical_volume *lv);
+
 int lv_is_cow(const struct logical_volume *lv);
+#define lv_is_thick_snapshot lv_is_cow
+
+int lv_is_cache_origin(const struct logical_volume *lv);
+
 int lv_is_merging_cow(const struct logical_volume *cow);
 uint32_t cow_max_extents(const struct logical_volume *origin, uint32_t chunk_size);
 int cow_has_min_chunks(const struct volume_group *vg, uint32_t cow_extents, uint32_t chunk_size);
@@ -1154,6 +1165,10 @@ struct logical_volume *detach_mirror_log(struct lv_segment *seg);
 int attach_mirror_log(struct lv_segment *seg, struct logical_volume *lv);
 int remove_mirror_log(struct cmd_context *cmd, struct logical_volume *lv,
 		      struct dm_list *removable_pvs, int force);
+struct logical_volume *prepare_mirror_log(struct logical_volume *lv,
+					  int in_sync, uint32_t region_size,
+					  struct dm_list *allocatable_pvs,
+					  alloc_policy_t alloc);
 int add_mirror_log(struct cmd_context *cmd, struct logical_volume *lv,
 		   uint32_t log_count, uint32_t region_size,
 		   struct dm_list *allocatable_pvs, alloc_policy_t alloc);
@@ -1205,10 +1220,19 @@ int lv_raid_convert(struct logical_volume *lv,
 		    const uint32_t new_region_size,
 		    struct dm_list *allocate_pvs);
 int lv_raid_rebuild(struct logical_volume *lv, struct dm_list *rebuild_pvs);
-int lv_raid_replace(struct logical_volume *lv, struct dm_list *remove_pvs,
-		    struct dm_list *allocate_pvs);
+int lv_raid_replace(struct logical_volume *lv, int force,
+		    struct dm_list *remove_pvs, struct dm_list *allocate_pvs);
 int lv_raid_remove_missing(struct logical_volume *lv);
 int partial_raid_lv_supports_degraded_activation(const struct logical_volume *lv);
+uint32_t raid_rmeta_extents_delta(struct cmd_context *cmd,
+				  uint32_t rimage_extents_cur, uint32_t rimage_extents_new,
+				  uint32_t region_size, uint32_t extent_size);
+uint32_t raid_rimage_extents(const struct segment_type *segtype,
+			     uint32_t extents, uint32_t stripes, uint32_t data_copies);
+uint32_t raid_ensure_min_region_size(const struct logical_volume *lv, uint64_t raid_size, uint32_t region_size);
+int lv_raid_change_region_size(struct logical_volume *lv,
+                               int yes, int force, uint32_t new_region_size);
+int lv_raid_in_sync(const struct logical_volume *lv);
 /* --  metadata/raid_manip.c */
 
 /* ++  metadata/cache_manip.c */

lib/metadata/metadata.c

@@ -828,25 +828,25 @@ int vg_extend_each_pv(struct volume_group *vg, struct pvcreate_params *pp)
 	struct pv_list *pvl;
 	unsigned int max_phys_block_size = 0;
 
-	log_debug_metadata("Adding PVs to VG %s", vg->name);
+	log_debug_metadata("Adding PVs to VG %s.", vg->name);
 
 	if (_vg_bad_status_bits(vg, RESIZEABLE_VG))
 		return_0;
 
 	dm_list_iterate_items(pvl, &pp->pvs) {
-		log_debug_metadata("Adding PV %s to VG %s", pv_dev_name(pvl->pv), vg->name);
+		log_debug_metadata("Adding PV %s to VG %s.", pv_dev_name(pvl->pv), vg->name);
 
 		if (!(check_dev_block_size_for_vg(pvl->pv->dev,
 						  (const struct volume_group *) vg,
 						  &max_phys_block_size))) {
-			log_error("PV %s has wrong block size", pv_dev_name(pvl->pv));
-			return_0;
+			log_error("PV %s has wrong block size.", pv_dev_name(pvl->pv));
+			return 0;
 		}
 
 		if (!add_pv_to_vg(vg, pv_dev_name(pvl->pv), pvl->pv, 0)) {
 			log_error("PV %s cannot be added to VG %s.",
 				  pv_dev_name(pvl->pv), vg->name);
-			return_0;
+			return 0;
 		}
 	}
 
@@ -1256,7 +1256,7 @@ uint32_t extents_from_percent_size(struct volume_group *vg, const struct dm_list
 			}
 			break;
 		}
-		/* Fall back to use all PVs in VG like %FREE */
+		/* fall through to use all PVs in VG like %FREE */
 	case PERCENT_FREE:
 		if (!(extents = vg->free_count)) {
 			log_error("No free extents in Volume group %s.", vg->name);
@@ -2544,7 +2544,7 @@ static int _lv_mark_if_partial_collect(struct logical_volume *lv, void *data)
 static int _lv_mark_if_partial_single(struct logical_volume *lv, void *data)
 {
 	unsigned s;
-	struct _lv_mark_if_partial_baton baton;
+	struct _lv_mark_if_partial_baton baton = { .partial = 0 };
 	struct lv_segment *lvseg;
 
 	dm_list_iterate_items(lvseg, &lv->segments) {
@@ -2556,7 +2556,6 @@ static int _lv_mark_if_partial_single(struct logical_volume *lv, void *data)
 		}
 	}
 
-	baton.partial = 0;
 	if (!_lv_each_dependency(lv, _lv_mark_if_partial_collect, &baton))
 		return_0;
 
@@ -5448,6 +5447,19 @@ int vg_flag_write_locked(struct volume_group *vg)
 	return 0;
 }
 
+static int _access_vg_clustered(struct cmd_context *cmd, const struct volume_group *vg)
+{
+	if (vg_is_clustered(vg) && !locking_is_clustered()) {
+		if (!cmd->ignore_clustered_vgs)
+			log_error("Skipping clustered volume group %s", vg->name);
+		else
+			log_verbose("Skipping clustered volume group %s", vg->name);
+		return 0;
+	}
+
+	return 1;
+}
+
 /*
  * Performs a set of checks against a VG according to bits set in status
  * and returns FAILED_* bits for those that aren't acceptable.
@@ -5459,15 +5471,9 @@ static uint32_t _vg_bad_status_bits(const struct volume_group *vg,
 {
 	uint32_t failure = 0;
 
-	if ((status & CLUSTERED) &&
-	    (vg_is_clustered(vg)) && !locking_is_clustered()) {
-		if (!vg->cmd->ignore_clustered_vgs)
-			log_error("Skipping clustered volume group %s", vg->name);
-		else
-			log_verbose("Skipping clustered volume group %s", vg->name);
+	if ((status & CLUSTERED) && !_access_vg_clustered(vg->cmd, vg))
 		/* Return because other flags are considered undefined. */
 		return FAILED_CLUSTERED;
-	}
 
 	if ((status & EXPORTED_VG) &&
 	    vg_is_exported(vg)) {
@@ -5556,19 +5562,6 @@ static int _allow_extra_system_id(struct cmd_context *cmd, const char *system_id
 	return 0;
 }
 
-static int _access_vg_clustered(struct cmd_context *cmd, struct volume_group *vg)
-{
-	if (vg_is_clustered(vg) && !locking_is_clustered()) {
-		if (!cmd->ignore_clustered_vgs)
-			log_error("Skipping clustered volume group %s", vg->name);
-		else
-			log_verbose("Skipping clustered volume group %s", vg->name);
-		return 0;
-	}
-
-	return 1;
-}
-
 static int _access_vg_lock_type(struct cmd_context *cmd, struct volume_group *vg,
 				uint32_t lockd_state, uint32_t *failure)
 {
@@ -5636,6 +5629,11 @@ static int _access_vg_lock_type(struct cmd_context *cmd, struct volume_group *vg
 		}
 	}
 
+	if (test_mode()) {
+		log_error("Test mode is not yet supported with lock type %s.", vg->lock_type);
+		return 0;
+	}
+
 	return 1;
 }
 
@@ -6390,7 +6388,7 @@ int vg_strip_outdated_historical_lvs(struct volume_group *vg) {
 		 * Removal time in the future? Not likely,
 		 * but skip this item in any case.
 		*/
-		if ((current_time) < glvl->glv->historical->timestamp_removed)
+		if (current_time < (time_t) glvl->glv->historical->timestamp_removed)
 			continue;
 
 		if ((current_time - glvl->glv->historical->timestamp_removed) > threshold) {

lib/metadata/metadata.h

@@ -501,6 +501,7 @@ int attach_pool_message(struct lv_segment *pool_seg, dm_thin_message_t type,
 int lv_is_merging_thin_snapshot(const struct logical_volume *lv);
 int pool_has_message(const struct lv_segment *seg,
 		     const struct logical_volume *lv, uint32_t device_id);
+int pool_metadata_min_threshold(const struct lv_segment *pool_seg);
 int pool_below_threshold(const struct lv_segment *pool_seg);
 int pool_check_overprovisioning(const struct logical_volume *lv);
 int create_pool(struct logical_volume *lv, const struct segment_type *segtype,

lib/metadata/mirror.c

@@ -136,16 +136,15 @@ struct lv_segment *find_mirror_seg(struct lv_segment *seg)
 {
 	struct lv_segment *mirror_seg;
 
-	mirror_seg = get_only_segment_using_this_lv(seg->lv);
-
-	if (!mirror_seg) {
-		log_error("Failed to find mirror_seg for %s", seg->lv->name);
+	if (!(mirror_seg = get_only_segment_using_this_lv(seg->lv))) {
+		log_error("Failed to find mirror_seg for %s", display_lvname(seg->lv));
 		return NULL;
 	}
 
 	if (!seg_is_mirrored(mirror_seg)) {
-		log_error("%s on %s is not a mirror segments",
-			  mirror_seg->lv->name, seg->lv->name);
+		log_error("LV %s on %s is not a mirror segments.",
+			  display_lvname(mirror_seg->lv),
+			  display_lvname(seg->lv));
 		return NULL;
 	}
 
@@ -246,7 +245,7 @@ int shift_mirror_images(struct lv_segment *mirrored_seg, unsigned mimage)
 
 	if (mimage >= mirrored_seg->area_count) {
 		log_error("Invalid index (%u) of mirror image supplied "
-			  "to shift_mirror_images()", mimage);
+			  "to shift_mirror_images().", mimage);
 		return 0;
 	}
 
@@ -282,21 +281,22 @@ static int _write_log_header(struct cmd_context *cmd, struct logical_volume *lv)
 	log_header.nr_regions = xlate64((uint64_t)-1);
 
 	if (!(name = dm_pool_alloc(cmd->mem, PATH_MAX))) {
-		log_error("Name allocation failed - log header not written (%s)",
-			lv->name);
+		log_error("Name allocation failed - log header not written (%s).",
+			  display_lvname(lv));
 		return 0;
 	}
 
 	if (dm_snprintf(name, PATH_MAX, "%s%s/%s", cmd->dev_dir,
 			 lv->vg->name, lv->name) < 0) {
-		log_error("Name too long - log header not written (%s)", lv->name);
+		log_error("Name too long - log header not written (%s).",
+			  display_lvname(lv));
 		return 0;
 	}
 
-	log_verbose("Writing log header to device, %s", lv->name);
+	log_verbose("Writing log header to device %s.", display_lvname(lv));
 
 	if (!(dev = dev_cache_get(name, NULL))) {
-		log_error("%s: not found: log header not written", name);
+		log_error("%s: not found: log header not written.", name);
 		return 0;
 	}
 
@@ -304,7 +304,7 @@ static int _write_log_header(struct cmd_context *cmd, struct logical_volume *lv)
 		return 0;
 
 	if (!dev_write(dev, UINT64_C(0), sizeof(log_header), &log_header)) {
-		log_error("Failed to write log header to %s", name);
+		log_error("Failed to write log header to %s.", name);
 		dev_close_immediate(dev);
 		return 0;
 	}
@@ -661,10 +661,11 @@ static int _split_mirror_images(struct logical_volume *lv,
 	struct dm_list split_images;
 	struct lv_list *lvl;
 	struct cmd_context *cmd = lv->vg->cmd;
+	char layer_name[NAME_LEN], format[NAME_LEN];
 
 	if (!lv_is_mirrored(lv)) {
-		log_error("Unable to split non-mirrored LV, %s",
-			  lv->name);
+		log_error("Unable to split non-mirrored LV %s.",
+			  display_lvname(lv));
 		return 0;
 	}
 
@@ -673,8 +674,8 @@ static int _split_mirror_images(struct logical_volume *lv,
 		return 0;
 	}
 
-	log_verbose("Detaching %d images from mirror, %s",
-		    split_count, lv->name);
+	log_verbose("Detaching %d images from mirror %s.",
+		    split_count, display_lvname(lv));
 
 	if (!_move_removable_mimages_to_end(lv, split_count, removable_pvs)) {
 		/*
@@ -684,8 +685,7 @@ static int _split_mirror_images(struct logical_volume *lv,
 		 * removable PVs or all of them.  Should we allow
 		 * them to just specify some - making us pick the rest?
 		 */
-		log_error("Insufficient removable PVs given"
-			  " to satisfy request");
+		log_error("Insufficient removable PVs given to satisfy request.");
 		return 0;
 	}
 
@@ -704,7 +704,7 @@ static int _split_mirror_images(struct logical_volume *lv,
 		if (!release_lv_segment_area(mirrored_seg, mirrored_seg->area_count, mirrored_seg->area_len))
 			return_0;
 
-		log_very_verbose("%s assigned to be split", sub_lv->name);
+		log_very_verbose("LV %s assigned to be split.", display_lvname(sub_lv));
 
 		if (!new_lv) {
 			lv_set_visible(sub_lv);
@@ -715,7 +715,7 @@ static int _split_mirror_images(struct logical_volume *lv,
 		/* If there is more than one image being split, add to list */
 		lvl = dm_pool_alloc(lv->vg->vgmem, sizeof(*lvl));
 		if (!lvl) {
-			log_error("lv_list alloc failed");
+			log_error("lv_list alloc failed.");
 			return 0;
 		}
 		lvl->lv = sub_lv;
@@ -729,17 +729,14 @@ static int _split_mirror_images(struct logical_volume *lv,
 	}
 
 	if (!dm_list_empty(&split_images)) {
-		size_t len = strlen(new_lv->name) + 32;
-		char *layer_name, format[len];
-
 		/*
 		 * A number of images have been split and
 		 * a new mirror layer must be formed
 		 */
 
 		if (!insert_layer_for_lv(cmd, new_lv, 0, "_mimage_%d")) {
-			log_error("Failed to build new mirror, %s",
-				  new_lv->name);
+			log_error("Failed to build new mirror, %s.",
+				  display_lvname(new_lv));
 			return 0;
 		}
 
@@ -748,27 +745,25 @@ static int _split_mirror_images(struct logical_volume *lv,
 		dm_list_iterate_items(lvl, &split_images) {
 			sub_lv = lvl->lv;
 
-			if (dm_snprintf(format, len, "%s_mimage_%%d",
+			if (dm_snprintf(format, sizeof(format), "%s_mimage_%%d",
 					new_lv->name) < 0) {
-				log_error("Failed to build new image name.");
+				log_error("Failed to build new image name for %s.",
+					  display_lvname(new_lv));
 				return 0;
 			}
-			layer_name = dm_pool_alloc(lv->vg->vgmem, len);
-			if (!layer_name) {
-				log_error("Unable to allocate memory");
+			if (!generate_lv_name(lv->vg, format, layer_name, sizeof(layer_name))) {
+				log_error("Failed to generate new image names for %s.",
+					  display_lvname(new_lv));
 				return 0;
 			}
-			if (!generate_lv_name(lv->vg, format, layer_name, len)||
-			    sscanf(layer_name, format, &i) != 1) {
-				log_error("Failed to generate new image names");
+			if (!(sub_lv->name = dm_pool_strdup(lv->vg->vgmem, layer_name))) {
+				log_error("Unable to allocate memory.");
 				return 0;
 			}
-			sub_lv->name = layer_name;
 		}
 
 		if (!_merge_mirror_images(new_lv, &split_images)) {
-			log_error("Failed to group split "
-				  "images into new mirror");
+			log_error("Failed to group split images into new mirror.");
 			return 0;
 		}
 
@@ -794,41 +789,15 @@ static int _split_mirror_images(struct logical_volume *lv,
 		detached_log_lv = detach_mirror_log(mirrored_seg);
 		if (!remove_layer_from_lv(lv, sub_lv))
 			return_0;
-		lv->status &= ~MIRROR;
-		lv->status &= ~MIRRORED;
-		lv->status &= ~LV_NOTSYNCED;
-	}
-
-	if (!vg_write(mirrored_seg->lv->vg)) {
-		log_error("Intermediate VG metadata write failed.");
-		return 0;
+		lv->status &= ~(MIRROR | MIRRORED | LV_NOTSYNCED);
 	}
 
 	/*
-	 * Suspend the mirror - this includes all the sub-LVs and
-	 *                      soon-to-be-split sub-LVs
+	 * Suspend and resume the mirror - this includes all
+	 * the sub-LVs and soon-to-be-split sub-LVs
 	 */
-	if (!suspend_lv(cmd, mirrored_seg->lv)) {
-		log_error("Failed to lock %s", mirrored_seg->lv->name);
-		vg_revert(mirrored_seg->lv->vg);
-		return 0;
-	}
-
-	if (!vg_commit(mirrored_seg->lv->vg)) {
-		resume_lv(cmd, mirrored_seg->lv);
-		return 0;
-	}
-
-	log_very_verbose("Updating \"%s\" in kernel", mirrored_seg->lv->name);
-
-	/*
-	 * Resume the mirror - this also activates the visible, independent
-	 *                     soon-to-be-split sub-LVs
-	 */
-	if (!resume_lv(cmd, mirrored_seg->lv)) {
-		log_error("Problem resuming %s", mirrored_seg->lv->name);
-		return 0;
-	}
+	if (!lv_update_and_reload(mirrored_seg->lv))
+		return_0;
 
 	/*
 	 * Recycle newly split LV so it is properly renamed.
@@ -893,7 +862,7 @@ static int _remove_mirror_images(struct logical_volume *lv,
 	struct logical_volume *sub_lv;
 	struct logical_volume *detached_log_lv = NULL;
 	struct logical_volume *temp_layer_lv = NULL;
-	struct lv_segment *pvmove_seg, *mirrored_seg = first_seg(lv);
+	struct lv_segment *seg, *pvmove_seg, *mirrored_seg = first_seg(lv);
 	uint32_t old_area_count = mirrored_seg->area_count;
 	uint32_t new_area_count = mirrored_seg->area_count;
 	struct lv_list *lvl;
@@ -904,15 +873,14 @@ static int _remove_mirror_images(struct logical_volume *lv,
 	if (removed)
 		*removed = 0;
 
-	log_very_verbose("Reducing mirror set %s from %" PRIu32 " to %"
-			 PRIu32 " image(s)%s.", lv->name,
+	log_very_verbose("Reducing mirror set %s from " FMTu32 " to " FMTu32
+			 " image(s)%s.", display_lvname(lv),
 			 old_area_count, old_area_count - num_removed,
 			 remove_log ? " and no log volume" : "");
 
 	if (collapse && (old_area_count - num_removed != 1)) {
-		log_error("Incompatible parameters to _remove_mirror_images");
+		log_error("Incompatible parameters to _remove_mirror_images.");
 		return 0;
-
 	}
 
 	num_removed = 0;
@@ -936,7 +904,8 @@ static int _remove_mirror_images(struct logical_volume *lv,
 			 */
 			if ((s == 0) && !_mirrored_lv_in_sync(lv) &&
 			    !(lv_is_partial(lv))) {
-				log_error("Unable to remove primary mirror image while mirror is not in-sync");
+				log_error("Unable to remove primary mirror image while mirror volume "
+					  "%s is not in-sync.", display_lvname(lv));
 				return 0;
 			}
 			if (!shift_mirror_images(mirrored_seg, s))
@@ -965,7 +934,7 @@ static int _remove_mirror_images(struct logical_volume *lv,
 		seg_lv(mirrored_seg, m)->status &= ~MIRROR_IMAGE;
 		lv_set_visible(seg_lv(mirrored_seg, m));
 		if (!(lvl = dm_pool_alloc(lv->vg->cmd->mem, sizeof(*lvl)))) {
-			log_error("lv_list alloc failed");
+			log_error("lv_list alloc failed.");
 			return 0;
 		}
 		lvl->lv = seg_lv(mirrored_seg, m);
@@ -987,19 +956,17 @@ static int _remove_mirror_images(struct logical_volume *lv,
 		if (!remove_layer_from_lv(lv, temp_layer_lv))
 			return_0;
 		if (collapse && !_merge_mirror_images(lv, &tmp_orphan_lvs)) {
-			log_error("Failed to add mirror images");
+			log_error("Failed to add mirror images.");
 			return 0;
 		}
-                /*
-                 * No longer a mirror? Even though new_area_count was 1,
-                 * _merge_mirror_images may have resulted into lv being still a
-                 * mirror. Fix up the flags if we only have one image left.
-                 */
-                if (lv_mirror_count(lv) == 1) {
-                    lv->status &= ~MIRROR;
-                    lv->status &= ~MIRRORED;
-                    lv->status &= ~LV_NOTSYNCED;
-                }
+		/*
+		 * No longer a mirror? Even though new_area_count was 1,
+		 * _merge_mirror_images may have resulted into lv being still a
+		 * mirror. Fix up the flags if we only have one image left.
+		 */
+		if (lv_mirror_count(lv) == 1)
+			lv->status &= ~(MIRROR | MIRRORED | LV_NOTSYNCED);
+
 		mirrored_seg = first_seg(lv);
 		if (remove_log && !detached_log_lv)
 			detached_log_lv = detach_mirror_log(mirrored_seg);
@@ -1008,14 +975,12 @@ static int _remove_mirror_images(struct logical_volume *lv,
 			dm_list_iterate_items(pvmove_seg, &lv->segments)
 				pvmove_seg->status |= PVMOVE;
 	} else if (new_area_count == 0) {
-		log_very_verbose("All mimages of %s are gone", lv->name);
+		log_very_verbose("All mimages of %s are gone.", display_lvname(lv));
 
 		/* All mirror images are gone.
 		 * It can happen for vgreduce --removemissing. */
 		detached_log_lv = detach_mirror_log(mirrored_seg);
-		lv->status &= ~MIRROR;
-		lv->status &= ~MIRRORED;
-		lv->status &= ~LV_NOTSYNCED;
+		lv->status &= ~(MIRROR | MIRRORED | LV_NOTSYNCED);
 		if (!replace_lv_with_error_segment(lv))
 			return_0;
 	} else if (remove_log)
@@ -1035,10 +1000,10 @@ static int _remove_mirror_images(struct logical_volume *lv,
 	 */
 	if (detached_log_lv && lv_is_mirrored(detached_log_lv) &&
 	    lv_is_partial(detached_log_lv)) {
-		struct lv_segment *seg = first_seg(detached_log_lv);
+		seg = first_seg(detached_log_lv);
 
-		log_very_verbose("%s being removed due to failures",
-				 detached_log_lv->name);
+		log_very_verbose("%s being removed due to failures.",
+				 display_lvname(detached_log_lv));
 
 		/*
 		 * We are going to replace the mirror with an
@@ -1047,47 +1012,24 @@ static int _remove_mirror_images(struct logical_volume *lv,
 		 * the sub-lv's)
 		 */
 		for (m = 0; m < seg->area_count; m++) {
+			if (!(lvl = dm_pool_alloc(lv->vg->cmd->mem,
+						  sizeof(*lvl))))
+				return_0;
+
 			seg_lv(seg, m)->status &= ~MIRROR_IMAGE;
 			lv_set_visible(seg_lv(seg, m));
-			if (!(lvl = dm_pool_alloc(lv->vg->cmd->mem,
-						  sizeof(*lvl)))) {
-				log_error("dm_pool_alloc failed");
-				return 0;
-			}
 			lvl->lv = seg_lv(seg, m);
 			dm_list_add(&tmp_orphan_lvs, &lvl->list);
 		}
 
 		if (!replace_lv_with_error_segment(detached_log_lv)) {
-			log_error("Failed error target substitution for %s",
-				  detached_log_lv->name);
+			log_error("Failed error target substitution for %s.",
+				  display_lvname(detached_log_lv));
 			return 0;
 		}
 
-		if (!vg_write(detached_log_lv->vg)) {
-			log_error("intermediate VG write failed.");
-			return 0;
-		}
-
-		if (!suspend_lv(detached_log_lv->vg->cmd,
-				detached_log_lv)) {
-			log_error("Failed to suspend %s",
-				  detached_log_lv->name);
-			return 0;
- 		}
-
-		if (!vg_commit(detached_log_lv->vg)) {
-			if (!resume_lv(detached_log_lv->vg->cmd,
-				       detached_log_lv))
-				stack;
+		if (!lv_update_and_reload(detached_log_lv))
 			return_0;
-		}
-
-		if (!resume_lv(detached_log_lv->vg->cmd, detached_log_lv)) {
-			log_error("Failed to resume %s",
-				  detached_log_lv->name);
-			return 0;
-		}
 	}
 
 	/*
@@ -1095,14 +1037,13 @@ static int _remove_mirror_images(struct logical_volume *lv,
 	 * remove the LVs from the mirror set, commit that metadata
 	 * then deactivate and remove them fully.
 	 */
-
 	if (!vg_write(mirrored_seg->lv->vg)) {
 		log_error("intermediate VG write failed.");
 		return 0;
 	}
 
 	if (!suspend_lv_origin(mirrored_seg->lv->vg->cmd, mirrored_seg->lv)) {
-		log_error("Failed to lock %s", mirrored_seg->lv->name);
+		log_error("Failed to lock %s.", display_lvname(mirrored_seg->lv));
 		vg_revert(mirrored_seg->lv->vg);
 		return 0;
 	}
@@ -1115,7 +1056,7 @@ static int _remove_mirror_images(struct logical_volume *lv,
 	 * FIXME: check propagation of suspend with visible flag
 	 */
 	if (temp_layer_lv && !suspend_lv(temp_layer_lv->vg->cmd, temp_layer_lv))
-		log_error("Problem suspending temporary LV %s", temp_layer_lv->name);
+		log_error("Problem suspending temporary LV %s.", display_lvname(temp_layer_lv));
 
 	if (!vg_commit(mirrored_seg->lv->vg)) {
 		if (!resume_lv(mirrored_seg->lv->vg->cmd, mirrored_seg->lv))
@@ -1123,7 +1064,7 @@ static int _remove_mirror_images(struct logical_volume *lv,
 		return_0;
 	}
 
-	log_very_verbose("Updating \"%s\" in kernel", mirrored_seg->lv->name);
+	log_very_verbose("Updating %s in kernel.", display_lvname(mirrored_seg->lv));
 
 	/*
 	 * Avoid having same mirror target loaded twice simultaneously by first
@@ -1132,12 +1073,12 @@ static int _remove_mirror_images(struct logical_volume *lv,
 	 * explicitly.
 	 */
 	if (temp_layer_lv && !resume_lv(temp_layer_lv->vg->cmd, temp_layer_lv)) {
-		log_error("Problem resuming temporary LV, %s", temp_layer_lv->name);
+		log_error("Problem resuming temporary LV %s.", display_lvname(temp_layer_lv));
 		return 0;
 	}
 
 	if (!resume_lv_origin(mirrored_seg->lv->vg->cmd, mirrored_seg->lv)) {
-		log_error("Problem reactivating %s", mirrored_seg->lv->name);
+		log_error("Problem reactivating %s.", display_lvname(mirrored_seg->lv));
 		return 0;
 	}
 
@@ -1162,7 +1103,8 @@ static int _remove_mirror_images(struct logical_volume *lv,
 				      1, &lv->tags, 0)) {
 			/* As a result, unnecessary sync may run after
 			 * collapsing. But safe.*/
-			log_error("Failed to initialize log device");
+			log_error("Failed to initialize log device %s.",
+				  display_lvname(first_seg(lv)->log_lv));
 			return 0;
 		}
 	}
@@ -1170,8 +1112,9 @@ static int _remove_mirror_images(struct logical_volume *lv,
 	if (removed)
 		*removed = old_area_count - new_area_count;
 
-	log_very_verbose(FMTu32 " image(s) removed from %s",
-			 old_area_count - new_area_count, lv->name);
+	log_very_verbose(FMTu32 " image(s) removed from %s.",
+			 old_area_count - new_area_count,
+			 display_lvname(lv));
 
 	return 1;
 }
@@ -1353,15 +1296,15 @@ static int _replace_mirror_images(struct lv_segment *mirrored_seg,
 	/* FIXME: Use lvconvert rather than duplicating its code */
 
 	if (mirrored_seg->area_count < num_mirrors) {
-		log_warn("WARNING: Failed to replace mirror device in %s/%s",
-			 mirrored_seg->lv->vg->name, mirrored_seg->lv->name);
+		log_warn("WARNING: Failed to replace mirror device in %s.",
+			 display_lvname(mirrored_seg->lv);
 
 		if ((mirrored_seg->area_count > 1) && !mirrored_seg->log_lv)
-			log_warn("WARNING: Use 'lvconvert -m %d %s/%s --corelog' to replace failed devices",
-				 num_mirrors - 1, lv->vg->name, lv->name);
+			log_warn("WARNING: Use 'lvconvert -m %d %s --corelog' to replace failed devices.",
+				 num_mirrors - 1, display_lvname(lv));
 		else
-			log_warn("WARNING: Use 'lvconvert -m %d %s/%s' to replace failed devices",
-				 num_mirrors - 1, lv->vg->name, lv->name);
+			log_warn("WARNING: Use 'lvconvert -m %d %s' to replace failed devices.",
+				 num_mirrors - 1, display_lvname(lv));
 		r = 0;
 
 		/* REMEMBER/FIXME: set in_sync to 0 if a new mirror device was added */
@@ -1374,11 +1317,11 @@ static int _replace_mirror_images(struct lv_segment *mirrored_seg,
 	 */
 	if ((mirrored_seg->area_count > 1) && !mirrored_seg->log_lv &&
 	    (log_policy != MIRROR_REMOVE)) {
-		log_warn("WARNING: Failed to replace mirror log device in %s/%s",
-			 lv->vg->name, lv->name);
+		log_warn("WARNING: Failed to replace mirror log device in %s.",
+			 display_lvname(lv));
 
-		log_warn("WARNING: Use 'lvconvert -m %d %s/%s' to replace failed devices",
-			 mirrored_seg->area_count - 1 , lv->vg->name, lv->name);
+		log_warn("WARNING: Use 'lvconvert -m %d %s' to replace failed devices.",
+			 mirrored_seg->area_count - 1 , display_lvname(lv));
 		r = 0;
 	}
 
@@ -1411,8 +1354,8 @@ int reconfigure_mirror_images(struct lv_segment *mirrored_seg, uint32_t num_mirr
 		/* Unable to remove bad devices */
 		return 0;
 
-	log_warn("WARNING: Bad device removed from mirror volume, %s/%s",
-		  mirrored_seg->lv->vg->name, mirrored_seg->lv->name);
+	log_warn("WARNING: Bad device removed from mirror volume %s.",
+		  display_lvname(mirrored_seg->lv));
 
 	log_policy = _get_mirror_log_fault_policy(mirrored_seg->lv->vg->cmd);
 	dev_policy = _get_mirror_device_fault_policy(mirrored_seg->lv->vg->cmd);
@@ -1424,20 +1367,20 @@ int reconfigure_mirror_images(struct lv_segment *mirrored_seg, uint32_t num_mirr
 
 	if (!r)
 		/* Failed to replace device(s) */
-		log_warn("WARNING: Unable to find substitute device for mirror volume, %s/%s",
-			 mirrored_seg->lv->vg->name, mirrored_seg->lv->name);
+		log_warn("WARNING: Unable to find substitute device for mirror volume %s.",
+			 display_lvname(mirrored_seg->lv));
 	else if (r > 0)
 		/* Success in replacing device(s) */
-		log_warn("WARNING: Mirror volume, %s/%s restored - substitute for failed device found.",
-			  mirrored_seg->lv->vg->name, mirrored_seg->lv->name);
+		log_warn("WARNING: Mirror volume %s restored - substitute for failed device found.",
+			 display_lvname(mirrored_seg->lv));
 	else
 		/* Bad device removed, but not replaced because of policy */
 		if (mirrored_seg->area_count == 1) {
-			log_warn("WARNING: Mirror volume, %s/%s converted to linear due to device failure.",
-				  mirrored_seg->lv->vg->name, mirrored_seg->lv->name);
+			log_warn("WARNING: Mirror volume %s converted to linear due to device failure.",
+				  display_lvname(mirrored_seg->lv);
 		} else if (had_log && !mirrored_seg->log_lv) {
-			log_warn("WARNING: Mirror volume, %s/%s disk log removed due to device failure.",
-				  mirrored_seg->lv->vg->name, mirrored_seg->lv->name);
+			log_warn("WARNING: Mirror volume %s disk log removed due to device failure.",
+				  display_lvname(mirrored_seg->lv));
 		}
 	/*
 	 * If we made it here, we at least removed the bad device.
@@ -1456,15 +1399,11 @@ static int _create_mimage_lvs(struct alloc_handle *ah,
 			      int log)
 {
 	uint32_t m, first_area;
-	char *img_name;
-	size_t len;
-	
-	len = strlen(lv->name) + 32;
-	img_name = alloca(len);
+	char img_name[NAME_LEN];
 
-	if (dm_snprintf(img_name, len, "%s_mimage_%%d", lv->name) < 0) {
-		log_error("img_name allocation failed. "
-			  "Remove new LV and retry.");
+	if (dm_snprintf(img_name, sizeof(img_name), "%s_mimage_%%d", lv->name) < 0) {
+		log_error("Failed to build new mirror image name for %s.",
+			  display_lvname(lv));
 		return 0;
 	}
 
@@ -1734,23 +1673,26 @@ static int _add_mirrors_that_preserve_segments(struct logical_volume *lv,
 	if (!(ah = allocate_extents(lv->vg, NULL, segtype, 1, mirrors, 0, 0,
 				    lv->le_count, allocatable_pvs, alloc, 0,
 				    parallel_areas))) {
-		log_error("Unable to allocate mirror extents for %s.", lv->name);
+		log_error("Unable to allocate mirror extents for %s.",
+			  display_lvname(lv));
 		return 0;
 	}
 
 	if (flags & MIRROR_BY_SEG) {
 		if (!lv_add_mirror_areas(ah, lv, 0, adjusted_region_size)) {
-			log_error("Failed to add mirror areas to %s", lv->name);
+			log_error("Failed to add mirror areas to %s.",
+				  display_lvname(lv));
 			r = 0;
 		}
 	} else if (flags & MIRROR_BY_SEGMENTED_LV) {
 		if (!lv_add_segmented_mirror_image(ah, lv, 0,
 						   adjusted_region_size)) {
-			log_error("Failed to add mirror areas to %s", lv->name);
+			log_error("Failed to add mirror areas to %s.",
+				  display_lvname(lv));
 			r = 0;
 		}
 	} else {
-		log_error(INTERNAL_ERROR "Unknown mirror flag");
+		log_error(INTERNAL_ERROR "Unknown mirror flag.");
 		r = 0;
 	}
 	alloc_destroy(ah);
@@ -1784,7 +1726,7 @@ int remove_mirror_log(struct cmd_context *cmd,
 
 	/* Unimplemented features */
 	if (dm_list_size(&lv->segments) != 1) {
-		log_error("Multiple-segment mirror is not supported");
+		log_error("Multiple-segment mirror is not supported.");
 		return 0;
 	}
 
@@ -1796,19 +1738,19 @@ int remove_mirror_log(struct cmd_context *cmd,
 			return 0;
 		}
 	} else if (lv_is_active(lv)) {
-		log_error("Unable to determine sync status of"
-			  " remotely active mirror, %s", lv->name);
+		log_error("Unable to determine sync status of "
+			  "remotely active mirror volume %s.", display_lvname(lv));
 		return 0;
 	} else if (vg_is_clustered(vg)) {
 		log_error("Unable to convert the log of an inactive "
-			  "cluster mirror, %s", lv->name);
+			  "cluster mirror volume %s.", display_lvname(lv));
 		return 0;
-	} else if (force || yes_no_prompt("Full resync required to convert "
-				 "inactive mirror %s to core log. "
-				 "Proceed? [y/n]: ", lv->name) == 'y')
+	} else if (force || yes_no_prompt("Full resync required to convert inactive "
+					  "mirror volume %s to core log. "
+					  "Proceed? [y/n]: ", display_lvname(lv)) == 'y')
 		sync_percent = 0;
 	else {
-		log_error("Logical volume %s NOT converted.", lv->name);
+		log_error("Logical volume %s NOT converted.", display_lvname(lv));
 		return 0;
 	}
 
@@ -1834,14 +1776,10 @@ static struct logical_volume *_create_mirror_log(struct logical_volume *lv,
 						 const char *suffix)
 {
 	struct logical_volume *log_lv;
-	char *log_name;
-	size_t len;
+	char log_name[NAME_LEN];
 
-	len = strlen(lv_name) + 32;
-	log_name = alloca(len); /* alloca never fails */
-
-	if (dm_snprintf(log_name, len, "%s%s", lv_name, suffix) < 0) {
-		log_error("log_name allocation failed.");
+	if (dm_snprintf(log_name, sizeof(log_name), "%s%s", lv_name, suffix) < 0) {
+		log_error("Failed to build new mirror log name for %s.", lv_name);
 		return NULL;
 	}
 
@@ -1872,7 +1810,7 @@ static int _form_mirror(struct cmd_context *cmd, struct alloc_handle *ah,
 	if (dm_list_size(&lv->segments) != 1 ||
 	    seg_type(first_seg(lv), 0) != AREA_LV)
 		if (!insert_layer_for_lv(cmd, lv, 0, "_mimage_%d"))
-			return 0;
+			return_0;
 
 	/*
 	 * create mirror image LVs
@@ -1883,7 +1821,9 @@ static int _form_mirror(struct cmd_context *cmd, struct alloc_handle *ah,
 		return_0;
 
 	if (!lv_add_mirror_lvs(lv, img_lvs, mirrors,
-			       MIRROR_IMAGE | (lv->status & LOCKED),
+			       /* Pass through MIRRORED & LOCKED status flag
+				* TODO: Any other would be needed ?? */
+			       MIRROR_IMAGE | (lv->status & (MIRRORED | LOCKED)),
 			       region_size)) {
 		log_error("Aborting. Failed to add mirror segment. "
 			  "Remove new LV and retry.");
@@ -1960,6 +1900,49 @@ int attach_mirror_log(struct lv_segment *seg, struct logical_volume *log_lv)
 	return add_seg_to_segs_using_this_lv(log_lv, seg);
 }
 
+/* Prepare disk mirror log for raid1->mirror conversion */
+struct logical_volume *prepare_mirror_log(struct logical_volume *lv,
+					  int in_sync, uint32_t region_size,
+					  struct dm_list *allocatable_pvs,
+					  alloc_policy_t alloc)
+{
+	struct cmd_context *cmd = lv->vg->cmd;
+	const struct segment_type *segtype;
+	struct dm_list *parallel_areas;
+	struct alloc_handle *ah;
+	struct logical_volume *log_lv;
+
+	if (!(parallel_areas = build_parallel_areas_from_lv(lv, 0, 0)))
+		return_NULL;
+
+	if (!(segtype = get_segtype_from_string(cmd, SEG_TYPE_NAME_MIRROR)))
+		return_NULL;
+
+	/* Allocate destination extents */
+	if (!(ah = allocate_extents(lv->vg, NULL, segtype,
+				    0, 0, 1, region_size,
+				    lv->le_count, allocatable_pvs,
+				    alloc, 0, parallel_areas))) {
+		log_error("Unable to allocate extents for mirror log.");
+		return NULL;
+	}
+
+	if (!(log_lv = _create_mirror_log(lv, ah, alloc, lv->name, "_mlog"))) {
+		log_error("Failed to create mirror log.");
+		goto out;
+	}
+
+	if (!_init_mirror_log(cmd, log_lv, in_sync, &lv->tags, 1)) {
+		log_error("Failed to initialise mirror log.");
+		log_lv = NULL;
+		goto out;
+	}
+out:
+	alloc_destroy(ah);
+
+	return log_lv;
+}
+
 int add_mirror_log(struct cmd_context *cmd, struct logical_volume *lv,
 		   uint32_t log_count, uint32_t region_size,
 		   struct dm_list *allocatable_pvs, alloc_policy_t alloc)
@@ -1974,25 +1957,25 @@ int add_mirror_log(struct cmd_context *cmd, struct logical_volume *lv,
 	int r = 0;
 
 	if (vg_is_clustered(lv->vg) && (log_count > 1)) {
-		log_error("Log type, \"mirrored\", is unavailable to cluster mirrors");
+		log_error("Log type, \"mirrored\", is unavailable to cluster mirrors.");
 		return 0;
 	}
 
 	if (dm_list_size(&lv->segments) != 1) {
-		log_error("Multiple-segment mirror is not supported");
+		log_error("Multiple-segment mirror is not supported.");
 		return 0;
 	}
 
 	if (lv_is_active_but_not_locally(lv)) {
 		log_error("Unable to convert the log of a mirror, %s, that is "
-			  "active remotely but not locally", lv->name);
+			  "active remotely but not locally.", display_lvname(lv));
 		return 0;
 	}
 
 	log_lv = first_seg(lv)->log_lv;
 	old_log_count = (log_lv) ? lv_mirror_count(log_lv) : 0;
 	if (old_log_count == log_count) {
-		log_verbose("Mirror already has a %s log",
+		log_verbose("Mirror %s already has a %s log.", display_lvname(lv),
 			    !log_count ? "core" :
 			    (log_count == 1) ? "disk" : "mirrored");
 		return 1;
@@ -2007,16 +1990,15 @@ int add_mirror_log(struct cmd_context *cmd, struct logical_volume *lv,
 	if (activation() && segtype->ops->target_present &&
 	    !segtype->ops->target_present(cmd, NULL, NULL)) {
 		log_error("%s: Required device-mapper target(s) not "
-			  "detected in your kernel", segtype->name);
+			  "detected in your kernel.", segtype->name);
 		return 0;
 	}
 
 	/* allocate destination extents */
-	ah = allocate_extents(lv->vg, NULL, segtype,
-			      0, 0, log_count - old_log_count, region_size,
-			      lv->le_count, allocatable_pvs,
-			      alloc, 0, parallel_areas);
-	if (!ah) {
+	if (!(ah = allocate_extents(lv->vg, NULL, segtype,
+				    0, 0, log_count - old_log_count, region_size,
+				    lv->le_count, allocatable_pvs,
+				    alloc, 0, parallel_areas))) {
 		log_error("Unable to allocate extents for mirror log.");
 		return 0;
 	}
@@ -2077,10 +2059,9 @@ int add_mirror_images(struct cmd_context *cmd, struct logical_volume *lv,
 	if (!(segtype = get_segtype_from_string(cmd, SEG_TYPE_NAME_MIRROR)))
 		return_0;
 
-	ah = allocate_extents(lv->vg, NULL, segtype,
-			      stripes, mirrors, log_count, region_size, lv->le_count,
-			      allocatable_pvs, alloc, 0, parallel_areas);
-	if (!ah) {
+	if (!(ah = allocate_extents(lv->vg, NULL, segtype,
+				    stripes, mirrors, log_count, region_size, lv->le_count,
+				    allocatable_pvs, alloc, 0, parallel_areas))) {
 		log_error("Unable to allocate extents for mirror(s).");
 		return 0;
 	}
@@ -2136,7 +2117,7 @@ int lv_add_mirrors(struct cmd_context *cmd, struct logical_volume *lv,
 		   struct dm_list *pvs, alloc_policy_t alloc, uint32_t flags)
 {
 	if (!mirrors && !log_count) {
-		log_error("No conversion is requested");
+		log_error("No conversion is requested.");
 		return 0;
 	}
 
@@ -2156,7 +2137,7 @@ int lv_add_mirrors(struct cmd_context *cmd, struct logical_volume *lv,
 		 * log daemon is multi-threaded.
 		 */
 		if (log_count > 1) {
-			log_error("Log type, \"mirrored\", is unavailable to cluster mirrors");
+			log_error("Log type, \"mirrored\", is unavailable to cluster mirrors.");
 			return 0;
 		}
 	}
@@ -2174,12 +2155,12 @@ int lv_add_mirrors(struct cmd_context *cmd, struct logical_volume *lv,
 	if (flags & MIRROR_BY_SEG) {
 		if (log_count) {
 			log_error("Persistent log is not supported on "
-				  "segment-by-segment mirroring");
+				  "segment-by-segment mirroring.");
 			return 0;
 		}
 		if (stripes > 1) {
 			log_error("Striped-mirroring is not supported on "
-				  "segment-by-segment mirroring");
+				  "segment-by-segment mirroring.");
 			return 0;
 		}
 
@@ -2189,7 +2170,7 @@ int lv_add_mirrors(struct cmd_context *cmd, struct logical_volume *lv,
 	} else if (flags & MIRROR_BY_SEGMENTED_LV) {
 		if (stripes > 1) {
 			log_error("Striped-mirroring is not supported on "
-				  "segment-by-segment mirroring");
+				  "segment-by-segment mirroring.");
 			return 0;
 		}
 
@@ -2205,26 +2186,27 @@ int lv_add_mirrors(struct cmd_context *cmd, struct logical_volume *lv,
 					 pvs, alloc, log_count);
 	}
 
-	log_error("Unsupported mirror conversion type");
+	log_error("Unsupported mirror conversion type.");
+
 	return 0;
 }
 
 int lv_split_mirror_images(struct logical_volume *lv, const char *split_name,
 			   uint32_t split_count, struct dm_list *removable_pvs)
 {
-	int r;
 	int historical;
 
 	if (lv_name_is_used_in_vg(lv->vg, split_name, &historical)) {
 		log_error("%sLogical Volume \"%s\" already exists in "
-			  "volume group \"%s\"", historical ? "historical " : "",
+			  "volume group \"%s\".", historical ? "historical " : "",
 			  split_name, lv->vg->name);
 		return 0;
 	}
 
 	/* Can't split a mirror that is not in-sync... unless force? */
 	if (!_mirrored_lv_in_sync(lv)) {
-		log_error("Unable to split mirror that is not in-sync.");
+		log_error("Unable to split mirror %s that is not in-sync.",
+			  display_lvname(lv));
 		return 0;
 	}
 
@@ -2238,8 +2220,7 @@ int lv_split_mirror_images(struct logical_volume *lv, const char *split_name,
 	 * is being implemented.  For now, we force the user to
 	 * come up with a name for their LV.
 	 */
-	r = _split_mirror_images(lv, split_name, split_count, removable_pvs);
-	if (!r)
+	if (!_split_mirror_images(lv, split_name, split_count, removable_pvs))
 		return_0;
 
 	return 1;
@@ -2261,18 +2242,18 @@ int lv_remove_mirrors(struct cmd_context *cmd __attribute__((unused)),
 	struct lv_segment *seg;
 
 	if (!mirrors && !log_count) {
-		log_error("No conversion is requested");
+		log_error("No conversion is requested.");
 		return 0;
 	}
 
 	seg = first_seg(lv);
 	if (!seg_is_mirrored(seg)) {
-		log_error("Not a mirror segment");
+		log_error("Not a mirror segment.");
 		return 0;
 	}
 
 	if (lv_mirror_count(lv) <= mirrors) {
-		log_error("Removing more than existing: %d <= %d",
+		log_error("Removing more than existing: %d <= %d.",
 			  seg->area_count, mirrors);
 		return 0;
 	}
@@ -2288,7 +2269,7 @@ int lv_remove_mirrors(struct cmd_context *cmd __attribute__((unused)),
 	/* MIRROR_BY_SEG */
 	if (log_count) {
 		log_error("Persistent log is not supported on "
-			  "segment-by-segment mirroring");
+			  "segment-by-segment mirroring.");
 		return 0;
 	}
 	return remove_mirrors_from_segments(lv, new_mirrors, status_mask);

lib/metadata/pool_manip.c

@@ -430,28 +430,6 @@ int validate_pool_chunk_size(struct cmd_context *cmd,
 	return r;
 }
 
-/* Greatest common divisor */
-static unsigned long _gcd(unsigned long n1, unsigned long n2)
-{
-	unsigned long remainder;
-
-	do {
-		remainder = n1 % n2;
-		n1 = n2;
-		n2 = remainder;
-	} while (n2);
-
-	return n1;
-}
-
-/* Least common multiple */
-static unsigned long _lcm(unsigned long n1, unsigned long n2)
-{
-	if (!n1 || !n2)
-		return 0;
-	return (n1 * n2) / _gcd(n1, n2);
-}
-
 int recalculate_pool_chunk_size_with_dev_hints(struct logical_volume *pool_lv,
 					       int passed_args,
 					       int chunk_size_calc_policy)
@@ -497,7 +475,7 @@ int recalculate_pool_chunk_size_with_dev_hints(struct logical_volume *pool_lv,
 				continue;
 
 			if (previous_hint)
-				hint = _lcm(previous_hint, hint);
+				hint = lcm(previous_hint, hint);
 			previous_hint = hint;
 			break;
 		case AREA_LV:
@@ -547,7 +525,7 @@ int update_pool_params(const struct segment_type *segtype,
 		return 0;
 	}
 
-	log_verbose("Using pool metadata size %s.",
+	log_verbose("Preferred pool metadata size %s.",
 		    display_size(vg->cmd, (uint64_t)*pool_metadata_extents * vg->extent_size));
 
 	return 1;

lib/metadata/segtype.h

@@ -1,6 +1,6 @@
 /*
  * Copyright (C) 2001-2004 Sistina Software, Inc. All rights reserved.  
- * Copyright (C) 2004-2015 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2004-2016 Red Hat, Inc. All rights reserved.
  *
  * This file is part of LVM2.
  *
@@ -50,7 +50,8 @@ struct dev_manager;
 #define SEG_RAID0		0x0000000000040000ULL
 #define SEG_RAID0_META		0x0000000000080000ULL
 #define SEG_RAID1		0x0000000000100000ULL
-#define SEG_RAID10		0x0000000000200000ULL
+#define SEG_RAID10_NEAR		0x0000000000200000ULL
+#define SEG_RAID10		SEG_RAID10_NEAR
 #define SEG_RAID4		0x0000000000400000ULL
 #define SEG_RAID5_N		0x0000000000800000ULL
 #define SEG_RAID5_LA		0x0000000001000000ULL
@@ -132,6 +133,10 @@ struct dev_manager;
 #define segtype_is_raid6_nr(segtype)	((segtype)->flags & SEG_RAID6_NR ? 1 : 0)
 #define segtype_is_raid6_n_6(segtype)	((segtype)->flags & SEG_RAID6_N_6 ? 1 : 0)
 #define segtype_is_raid6_zr(segtype)	((segtype)->flags & SEG_RAID6_ZR ? 1 : 0)
+#define segtype_is_raid6_ls_6(segtype)	((segtype)->flags & SEG_RAID6_LS_6 ? 1 : 0)
+#define segtype_is_raid6_rs_6(segtype)	((segtype)->flags & SEG_RAID6_RS_6 ? 1 : 0)
+#define segtype_is_raid6_la_6(segtype)	((segtype)->flags & SEG_RAID6_LA_6 ? 1 : 0)
+#define segtype_is_raid6_ra_6(segtype)	((segtype)->flags & SEG_RAID6_RA_6 ? 1 : 0)
 #define segtype_is_any_raid10(segtype)	((segtype)->flags & SEG_RAID10 ? 1 : 0)
 #define segtype_is_raid10(segtype)	((segtype)->flags & SEG_RAID10 ? 1 : 0)
 #define segtype_is_raid10_near(segtype)	segtype_is_raid10(segtype)
@@ -144,6 +149,12 @@ struct dev_manager;
 #define segtype_is_virtual(segtype)	((segtype)->flags & SEG_VIRTUAL ? 1 : 0)
 #define segtype_is_unknown(segtype)	((segtype)->flags & SEG_UNKNOWN ? 1 : 0)
 
+#define segtype_can_split(segtype)	((segtype)->flags & SEG_CAN_SPLIT ? 1 : 0)
+#define segtype_cannot_be_zeroed(segtype) ((segtype)->flags & SEG_CANNOT_BE_ZEROED ? 1 : 0)
+#define segtype_monitored(segtype)	((segtype)->flags & SEG_MONITORED ? 1 : 0)
+#define segtype_only_exclusive(segtype)	((segtype)->flags & SEG_ONLY_EXCLUSIVE ? 1 : 0)
+#define segtype_can_error_when_full(segtype)	((segtype)->flags & SEG_CAN_ERROR_WHEN_FULL ? 1 : 0)
+
 #define segtype_supports_stripe_size(segtype)	\
 	((segtype_is_striped(segtype) || segtype_is_mirror(segtype) || \
 	  segtype_is_cache(segtype) || segtype_is_cache_pool(segtype) || \
@@ -153,6 +164,7 @@ struct dev_manager;
 #define seg_is_striped_target(seg)	segtype_is_striped_target((seg)->segtype)
 #define seg_is_cache(seg)	segtype_is_cache((seg)->segtype)
 #define seg_is_cache_pool(seg)	segtype_is_cache_pool((seg)->segtype)
+#define seg_is_used_cache_pool(seg) (seg_is_cache_pool(seg) && (!dm_list_empty(&(seg->lv)->segs_using_this_lv)))
 #define seg_is_linear(seg)	(seg_is_striped(seg) && ((seg)->area_count == 1))
 #define seg_is_mirror(seg)	segtype_is_mirror((seg)->segtype)
 #define seg_is_mirrored(seg)	segtype_is_mirrored((seg)->segtype)
@@ -187,11 +199,11 @@ struct dev_manager;
 #define seg_is_thin_volume(seg)	segtype_is_thin_volume((seg)->segtype)
 #define seg_is_virtual(seg)	segtype_is_virtual((seg)->segtype)
 #define seg_unknown(seg)	segtype_is_unknown((seg)->segtype)
-#define seg_can_split(seg)	((seg)->segtype->flags & SEG_CAN_SPLIT ? 1 : 0)
-#define seg_cannot_be_zeroed(seg) ((seg)->segtype->flags & SEG_CANNOT_BE_ZEROED ? 1 : 0)
-#define seg_monitored(seg)	((seg)->segtype->flags & SEG_MONITORED ? 1 : 0)
-#define seg_only_exclusive(seg)	((seg)->segtype->flags & SEG_ONLY_EXCLUSIVE ? 1 : 0)
-#define seg_can_error_when_full(seg) ((seg)->segtype->flags & SEG_CAN_ERROR_WHEN_FULL ? 1 : 0)
+#define seg_can_split(seg)	segtype_can_split((seg)->segtype)
+#define seg_cannot_be_zeroed(seg)	segtype_cannot_be_zeroed((seg)->segtype)
+#define seg_monitored(seg)	segtype_monitored((seg)->segtype)
+#define seg_only_exclusive(seg)	segtype_only_exclusive((seg)->segtype)
+#define seg_can_error_when_full(seg)	segtype_can_error_when_full((seg)->segtype)
 
 struct segment_type {
 	struct dm_list list;		/* Internal */
@@ -267,6 +279,7 @@ struct segment_type *init_unknown_segtype(struct cmd_context *cmd,
 #define RAID_FEATURE_RAID10			(1U << 0) /* version 1.3 */
 #define RAID_FEATURE_RAID0			(1U << 1) /* version 1.7 */
 #define RAID_FEATURE_RESHAPING			(1U << 2) /* version 1.8 */
+#define RAID_FEATURE_RAID4			(1U << 3) /* ! version 1.8 or 1.9.0 */
 
 #ifdef RAID_INTERNAL
 int init_raid_segtypes(struct cmd_context *cmd, struct segtype_library *seglib);

lib/metadata/snapshot_manip.c

@@ -121,7 +121,7 @@ int lv_is_visible(const struct logical_volume *lv)
 	if (lv_is_historical(lv))
 		return 1;
 
-	if (lv->status & SNAPSHOT)
+	if (lv_is_snapshot(lv))
 		return 0;
 
 	if (lv_is_cow(lv)) {

lib/metadata/takeover_matrix.h

@@ -112,7 +112,7 @@ static takeover_fn_t _takeover_fns[][11] = {
         /* raid1      */ { r1__lin,  r1__str,  r1__mir,  r1__r0,  r1__r0m,  r1__r1,  r1__r45,  X     ,  r1__r10,  X      , X },
         /* raid4/5    */ { r45_lin,  r45_str,  r45_mir,  r45_r0,  r45_r0m,  r45_r1,  r45_r54,  r45_r6,  X      ,  X      , X },
         /* raid6      */ { X      ,  r6__str,  X      ,  r6__r0,  r6__r0m,  X     ,  r6__r45,  X     ,  X      ,  X      , X },
-        /* raid10     */ // { r10_lin,  r10_str,  r10_mir,  r10_r0,  r10_r0m,  r10_r1,  X      ,  X     ,  r10_r10,  r10_r01, X },
+        /* raid10     */ { r10_lin,  r10_str,  r10_mir,  r10_r0,  r10_r0m,  r10_r1,  X      ,  X     ,  X      ,  X	 , X },
         /* raid01     */ // { X      ,  r01_str,  X      ,  X     ,  X      ,  X     ,  X      ,  X     ,  r01_r10,  r01_r01, X },
         /* other      */ { X      ,  X      ,  X      ,  X     ,  X      ,  X     ,  X      ,  X     ,  X      ,  X      , X },
 };

lib/metadata/thin_manip.c

@@ -99,6 +99,10 @@ int attach_thin_external_origin(struct lv_segment *seg,
 				    external_lv->name);
 			external_lv->status &= ~LVM_WRITE;
 		}
+
+		// TODO: should we mark even origin read-only ?
+		//if (lv_is_cache(external_lv)) /* read-only corigin of cache LV */
+		//	seg_lv(first_seg(external_lv), 0)->status &= ~LVM_WRITE;
 	}
 
 	return 1;
@@ -215,9 +219,29 @@ int thin_pool_feature_supported(const struct logical_volume *lv, int feature)
 	return (attr & feature) ? 1 : 0;
 }
 
+int pool_metadata_min_threshold(const struct lv_segment *pool_seg)
+{
+	/*
+	 * Hardcoded minimal requirment for thin pool target.
+	 *
+	 * In the metadata LV there should be minimum from either 4MiB of free space
+	 * or at least 25% of free space, which applies when the size of thin pool's
+	 * metadata is less then 16MiB.
+	 */
+	const dm_percent_t meta_min = DM_PERCENT_1 * 25;
+	dm_percent_t meta_free = dm_make_percent(((4096 * 1024) >> SECTOR_SHIFT),
+						pool_seg->metadata_lv->size);
+
+	if (meta_min < meta_free)
+		meta_free = meta_min;
+
+	return DM_PERCENT_100 - meta_free;
+}
+
 int pool_below_threshold(const struct lv_segment *pool_seg)
 {
 	dm_percent_t percent;
+	dm_percent_t min_threshold = pool_metadata_min_threshold(pool_seg);
 	dm_percent_t threshold = DM_PERCENT_1 *
 		find_config_tree_int(pool_seg->lv->vg->cmd, activation_thin_pool_autoextend_threshold_CFG,
 				     lv_config_profile(pool_seg->lv));
@@ -226,7 +250,7 @@ int pool_below_threshold(const struct lv_segment *pool_seg)
 	if (!lv_thin_pool_percent(pool_seg->lv, 0, &percent))
 		return_0;
 
-	if (percent > threshold) {
+	if (percent > threshold || percent >= DM_PERCENT_100) {
 		log_debug("Threshold configured for free data space in "
 			  "thin pool %s has been reached (%.2f%% >= %.2f%%).",
 			  display_lvname(pool_seg->lv),
@@ -239,6 +263,18 @@ int pool_below_threshold(const struct lv_segment *pool_seg)
 	if (!lv_thin_pool_percent(pool_seg->lv, 1, &percent))
 		return_0;
 
+
+	if (percent >= min_threshold) {
+		log_warn("WARNING: Remaining free space in metadata of thin pool %s "
+			 "is too low (%.2f%% >= %.2f%%). "
+			 "Resize is recommended.",
+			 display_lvname(pool_seg->lv),
+			 dm_percent_to_float(percent),
+			 dm_percent_to_float(min_threshold));
+		return 0;
+	}
+
+
 	if (percent > threshold) {
 		log_debug("Threshold configured for free metadata space in "
 			  "thin pool %s has been reached (%.2f%% > %.2f%%).",
@@ -440,7 +476,7 @@ static int _check_pool_create(const struct logical_volume *lv)
 
 int update_pool_lv(struct logical_volume *lv, int activate)
 {
-	int monitored = DMEVENTD_MONITOR_IGNORE;
+	int monitored;
 	int ret = 1;
 
 	if (!lv_is_thin_pool(lv)) {
@@ -716,6 +752,19 @@ int lv_is_thin_origin(const struct logical_volume *lv, unsigned int *snap_count)
 	return r;
 }
 
+int lv_is_thin_snapshot(const struct logical_volume *lv)
+{
+	struct lv_segment *seg;
+
+	if (!lv_is_thin_volume(lv))
+		return 0;
+
+	if ((seg = first_seg(lv)) && (seg->origin || seg->external_lv))
+		return 1;
+
+	return 0;
+}
+
 /*
  * Explict check of new thin pool for usability
  *

lib/misc/lib.h

@@ -89,6 +89,7 @@
 #  include "lvm-logging.h"
 #  include "lvm-globals.h"
 #  include "lvm-wrappers.h"
+#  include "lvm-maths.h"
 #endif
 
 #include <unistd.h>

lib/misc/lvm-flock.c

@@ -15,7 +15,6 @@
 
 #include "lib.h"
 #include "config.h"
-#include "lvm-file.h"
 #include "lvm-flock.h"
 #include "lvm-signal.h"
 #include "locking.h"

lib/misc/lvm-maths.c

@@ -0,0 +1,38 @@
+/*
+ * Copyright (C) 2016 Red Hat, Inc. All rights reserved.
+ *
+ * This file is part of LVM2.
+ *
+ * This copyrighted material is made available to anyone wishing to use,
+ * modify, copy, or redistribute it subject to the terms and conditions
+ * of the GNU Lesser General Public License v.2.1.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "lib.h"
+
+/* Greatest common divisor */
+unsigned long gcd(unsigned long n1, unsigned long n2)
+{
+	unsigned long remainder;
+
+	do {
+		remainder = n1 % n2;
+		n1 = n2;
+		n2 = remainder;
+	} while (n2);
+
+	return n1;
+}
+
+/* Least common multiple */
+unsigned long lcm(unsigned long n1, unsigned long n2)
+{
+	if (!n1 || !n2)
+		return 0;
+
+	return (n1 * n2) / gcd(n1, n2);
+}

lib/misc/lvm-maths.h

@@ -0,0 +1,24 @@
+/*
+ * Copyright (C) 2016 Red Hat, Inc. All rights reserved.
+ *
+ * This file is part of LVM2.
+ *
+ * This copyrighted material is made available to anyone wishing to use,
+ * modify, copy, or redistribute it subject to the terms and conditions
+ * of the GNU Lesser General Public License v.2.1.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef _LVM_MATH_H
+#define _LVM_MATH_H
+
+/* Greatest common divisor */
+unsigned long gcd(unsigned long n1, unsigned long n2);
+
+/* Least common multiple */
+unsigned long lcm(unsigned long n1, unsigned long n2);
+
+#endif

lib/misc/util.h

@@ -35,6 +35,47 @@
 #define uninitialized_var(x) x = x
 #endif
 
+/*
+ * GCC 3.4 adds a __builtin_clz, which uses the count leading zeros (clz)
+ * instruction on arches that have one. Provide a fallback using shifts
+ * and comparisons for older compilers.
+ */
+#ifdef HAVE___BUILTIN_CLZ
+#define clz(x) __builtin_clz((x))
+#else /* ifdef HAVE___BUILTIN_CLZ */
+unsigned _dm_clz(unsigned x)
+{
+	int n;
+
+	if ((int)x <= 0) return (~x >> 26) & 32;
+
+	n = 1;
+
+	if ((x >> 16) == 0) {
+		n = n + 16;
+		x = x << 16;
+	}
+
+	if ((x >> 24) == 0) {
+		n = n + 8;
+		x = x << 8;
+	}
+
+	if ((x >> 28) == 0) {
+		n = n + 4;
+		x = x << 4;
+	}
+
+	if ((x >> 30) == 0) {
+		n = n + 2;
+		x = x << 2;
+	}
+	n = n - (x >> 31);
+	return n;
+}
+#define clz(x) _dm_clz((x))
+#endif /* ifdef HAVE___BUILTIN_CLZ */
+
 #define KERNEL_VERSION(major, minor, release) (((major) << 16) + ((minor) << 8) + (release))
 
 /* Define some portable printing types */

lib/notify/lvmnotify.c

@@ -15,6 +15,8 @@
 #define LVM_DBUS_DESTINATION "com.redhat.lvmdbus1"
 #define LVM_DBUS_PATH        "/com/redhat/lvmdbus1/Manager"
 #define LVM_DBUS_INTERFACE   "com.redhat.lvmdbus1.Manager"
+#define SD_BUS_SYSTEMD_NO_SUCH_UNIT_ERROR "org.freedesktop.systemd1.NoSuchUnit"
+#define SD_BUS_DBUS_SERVICE_UNKNOWN_ERROR "org.freedesktop.DBus.Error.ServiceUnknown"
 
 #ifdef NOTIFYDBUS_SUPPORT
 #include <systemd/sd-bus.h>
@@ -26,6 +28,7 @@ int lvmnotify_is_supported(void)
 
 void lvmnotify_send(struct cmd_context *cmd)
 {
+	static const char _dbus_notification_failed_msg[] = "D-Bus notification failed";
 	sd_bus *bus = NULL;
 	sd_bus_message *m = NULL;
 	sd_bus_error error = SD_BUS_ERROR_NULL;
@@ -61,7 +64,11 @@ void lvmnotify_send(struct cmd_context *cmd)
 				 cmd_name);
 
 	if (ret < 0) {
-		log_warn("WARNING: D-Bus notification failed: %s", error.message);
+		if (sd_bus_error_has_name(&error, SD_BUS_SYSTEMD_NO_SUCH_UNIT_ERROR) ||
+		    sd_bus_error_has_name(&error, SD_BUS_DBUS_SERVICE_UNKNOWN_ERROR))
+			log_debug_dbus("%s: %s", _dbus_notification_failed_msg, error.message);
+		else
+			log_warn("WARNING: %s: %s", _dbus_notification_failed_msg, error.message);
 		goto out;
 	}