1
0
mirror of git://sourceware.org/git/lvm2.git synced 2024-12-22 17:35:59 +03:00
Commit Graph

1765 Commits

Author SHA1 Message Date
Zdenek Kabelac
9c7ee1e1c4 build: link dmeventd plugins with internal libs 2018-06-11 22:25:42 +02:00
Joe Thornber
0ac89fb860 various: some missing #include zalloc.h 2018-06-08 14:18:09 +01:00
Joe Thornber
d5da55ed85 device_mapper: remove dbg_malloc.
I wrote dbg_malloc before we had valgrind.  These days there's just
no need.
2018-06-08 13:40:53 +01:00
Joe Thornber
286c1ba336 device_mapper: rename libdevmapper.h -> all.h
I'm paranoid a file will include the global one in /usr/include
by accident.
2018-06-08 12:31:45 +01:00
David Teigland
3e781ea446 Remove clvmd and associated code
More code reduction and simplification can follow.
2018-06-05 11:09:13 -05:00
Joe Thornber
232918fb86 build: libbase.a 2018-06-04 13:53:07 +01:00
Zdenek Kabelac
1140d70893 build: fixes 2018-06-04 12:28:13 +02:00
Joe Thornber
02c4901d89 build: get clvmd building again 2018-06-01 19:18:36 +01:00
Joe Thornber
dbba1e9b93 Merge branch 'master' into 2018-05-11-fork-libdm 2018-06-01 13:04:12 +01:00
David Teigland
6cd0523337 lvmlockd: enable repairing shared VG while reading it
When the lvmlockd lock is shared, upgrade it to ex
when repair (writing) is needed during vg_read.

Pass the lockd state through additional read-related
functions so the instances of repair scattered through
vg_read can be handled.

(Temporary solution until the ad hoc repairs can be
pulled out of vg_read into a top level, centralized
repair function.)
2018-05-30 12:56:46 -05:00
David Teigland
a9f2c1e1f5 lvmlockd: suppress error messages related to lvmetad
Log lvmetad related messages as debug, not as errors,
when using lvmlockd without lvmetad.
2018-05-18 16:00:54 -05:00
Zdenek Kabelac
b2574c2f3a python: use // for integer division 2018-05-18 16:25:44 +02:00
Zdenek Kabelac
5b86b0e3dc build: set clean vars earlier
For better cleaning of test dirs.
2018-05-18 16:25:44 +02:00
Joe Thornber
89fdc0b588 Merge branch 'master' into 2018-05-11-fork-libdm 2018-05-16 13:43:02 +01:00
Joe Thornber
ccc35e2647 device-mapper: Fork libdm internally.
The device-mapper directory now holds a copy of libdm source.  At
the moment this code is identical to libdm.  Over time code will
migrate out to appropriate places (see doc/refactoring.txt).

The libdm directory still exists, and contains the source for the
libdevmapper shared library, which we will continue to ship (though
not neccessarily update).

All code using libdm should now use the version in device-mapper.
2018-05-16 13:00:50 +01:00
Joe Thornber
7f97c7ea9a build: Don't generate symlinks in include/ dir
As we start refactoring the code to break dependencies (see doc/refactoring.txt),
I want us to use full paths in the includes (eg, #include "base/data-struct/list.h").
This makes it more obvious when we're breaking abstraction boundaries, eg, including a file in
metadata/ from base/
2018-05-14 10:30:20 +01:00
Zdenek Kabelac
7616a7f46e build: properly track source file for lmvlockctl
Ensure the source file is tracked by various cleanup functions.
2018-05-12 18:05:50 +02:00
David Teigland
c016b573ee clvmd: separate saved_vg from vginfo
The clvmd saved_vg data is independent from the normal lvm
lvmcache vginfo data, so separate saved_vg from vginfo.
Normal lvm doesn't need to use save_vg at all, and in clvmd,
lvmcache changes on vginfo can be made without worrying
about unwanted effects on saved_vg.
2018-05-03 14:54:48 -05:00
Joe Thornber
972b535220 build: add -D_FILE_OFFSET_BITS=64
I don't like having this in a common header because it means you end
up including too much and causing unneccessary dependencies.  eg,
lib/misc/lib.h includes libdevmapper.h, internationalisation, and
logging stuff.
2018-05-02 18:40:38 +01:00
David Teigland
c1cd18f21e Remove lvm1 and pool disk formats
There are likely more bits of code that can be removed,
e.g. lvm1/pool-specific bits of code that were identified
using FMT flags.

The vgconvert command can likely be reduced further.

The lvm1-specific config settings should probably have
some other fields set for proper deprecation.
2018-04-30 16:55:02 -05:00
Joe Thornber
1ddbbb67e0 build: fix typo in dmeventd/plugins/Makefile.in 2018-04-30 15:31:57 +01:00
Joe Thornber
2bc896f2a3 build: remove --with-{snapshots,mirrors,raid,thin,cache} options from ./configure
It now behaves as if the were all set as 'internal'
2018-04-30 10:11:23 +01:00
David Teigland
4670e9f698 skip some clvmd-specific code in common cases
This, or something like it, can probably be done
in many other places.
2018-04-25 16:40:08 -05:00
David Teigland
9b6a62f944 lvmcache: simplify
Recent changes allow some major simplification of the way
lvmcache works and is used.  lvmcache_label_scan is now
called in a controlled fashion at the start of commands,
and not via various unpredictable side effects.  Remove
various calls to it from other places.  lvmcache_label_scan
should not be called from anywhere during a command, because
it produces an incorrect representation of PVs with no MDAs,
and misclassifies them as orphans.  This has been a long
standing problem.  The invalid flag and rescanning based on
that is no longer used and removed.  The 'force' variation is
no longer needed and removed.
2018-04-20 11:22:48 -05:00
David Teigland
c0973e70a5 dev_cache: clean up scan
Pull out all of the twisted logic and simply call dev_cache_scan
at the start of the command prior to label scan.
2018-04-20 11:22:48 -05:00
David Teigland
ddb5de7a98 clvm: fix bcache scan handling
We can't let clvmd keep all scanned devs open,
which prevents them from being removed.  So
drop the bcache data (and close fds) affter
doing a label scan.

Also set up bcache before the clvm-specific
vg_read (which needs to rescan the vg's devs
using bcache) and destroy the bcache after.
2018-04-20 11:22:48 -05:00
David Teigland
697fa7aa1d [makefile] add -laio to makefiles 2018-04-20 11:13:17 -05:00
Zdenek Kabelac
fa5ba7e42d coverity: ensure 0 end string
Use dm_strncpy() to enusure string ends with '\0'.
In case uuid does not fit, report error.
2018-04-20 12:17:01 +02:00
Zdenek Kabelac
037c234eaa cleanup: avoid compiler warn
When variable is unused...
2018-04-20 12:17:01 +02:00
Zdenek Kabelac
c82ab92d04 cleanup: use zalloc
Replace malloc() + memset()   with zalloc().
2018-03-17 23:33:58 +01:00
Zdenek Kabelac
5c40e81a7e cleanup: use direct initializer 2018-03-17 23:33:58 +01:00
Zdenek Kabelac
b4c69320fc coverity: move declaration out of the loop
Move declaration of count counter outside the while loop.
2018-03-17 23:33:58 +01:00
Zdenek Kabelac
26c58027fb coverity: validate descriptor
Since this function is called with 'fd == -1', but Coverity can't see
this path can't be visited with this argument, add explicit check for
valid descriptor.
2018-03-17 23:33:58 +01:00
Zdenek Kabelac
b6e7a0b490 cleanup: more usage of dm_strncpy
Use existing wrapper function arournd  strncpy + buf[] = 0;
2018-03-06 15:40:34 +01:00
Zdenek Kabelac
9be086fbee thin: pass environment to scripts
When dmeventd thin plugin forks a configurable script, switch to use
execvp to pass whole environment present to dmeventd - so all configured
paths present at dmeventd startup are visible to script.

This was likely not a problem for common user enviroment,
however in test suite case variable like LVM_SYSTEM_DIR were
not actually used from test itself but rather from
a system present lvm.conf and this may have cause strange
behavior of a testing script.
2018-03-06 15:35:04 +01:00
Zdenek Kabelac
9bfc8881cb coverity: missing free on error path 2018-02-28 21:05:18 +01:00
Zdenek Kabelac
e87fa7c9ce sanlock: set proper return value
In last patch one error path missed to assign correct return value.
Assing it directly to 'ret' as log_error was already reported.
2018-02-19 16:44:10 +01:00
Zdenek Kabelac
d90a647802 activation: separate reporting of error and monitoring status
Avoid using same return code for reporting 2 different things
and stricly report error code by return value and add new
parameter for reporting monitoring status.

This makes easier to recognize which error we got from dm_event
and continue only with  ENOENT.
2018-02-12 22:14:59 +01:00
Zdenek Kabelac
4f278324c7 lvmlockd: improve dm path creation for sanlock LV
Use devmapper function to create matching dm name with mangling.
Drop extra '-1' from buffer passed to snprintf.
2018-02-12 22:14:25 +01:00
Zdenek Kabelac
7239a45b79 clean: drop unneeded -1 for snprintf
man gives:
snprintf() and vsnprintf() write at most size bytes
(including the terminating null byte ('\0')) to str.
2018-02-12 22:14:25 +01:00
Tim Foerster
34fb5202bd lvmdbusd: Remove duplicated DataPercent definition 2018-01-25 08:37:24 -06:00
Zdenek Kabelac
f41935909f dmeventd: add check for result code
Check result from pthread_kill.
2018-01-17 14:44:33 +01:00
Marian Csontos
c957d46f1d lvmdbusd: Make lvmdbusd executable
- Add files built from *.in to builddir files.
- Add all files built from *.in to DISTCLEAN_TARGETS.
2017-12-14 16:45:53 +01:00
Marian Csontos
7272fd2210 lvmdbusd: All tools use detected python3
- lvmdb.py and lvm_shell_proxy.py can be used as standalone tools, so
  should use detected value.
- clean executable bit on *.in files.
2017-12-12 13:17:07 +01:00
Marian Csontos
2aedc98242 lvmdbusd: Fix path to python3
lvmdbusd executable script must use python3 interpreter detected by
configure script, as site-packages directory used for library is only
used by that interpreter.
2017-12-11 12:36:54 +01:00
David Teigland
0f0f6978e7 lvmlock: return an error if lvb cannot be written
It doesn't make much difference, because if this happens,
it means there are other more significant problems.
2017-12-06 10:49:31 -06:00
David Teigland
46d6f7a639 lvmlockd: recognize normal errors from sanlock_convert
Don't log an error message for error values returned
by sanlock_convert for expected conditions like lock
contention or io timeouts.
2017-12-05 11:53:03 -06:00
David Teigland
bd893348b4 lvmlockd: avoid blocking in sanlock_convert
Use the same flag as sanlock_acquire() for the
same purpose.  sanlock_convert can block like
acquire when doing a sh to ex conversion.
2017-12-05 11:53:03 -06:00
Zdenek Kabelac
76954884c7 cleanup: drop unused define 2017-12-04 15:38:50 +01:00
David Teigland
bbaaf4f1d3 lvmlockd: override unknown lock manager error numbers
When sanlock or dlm lock managers return an error number
that we don't recognize, replace it with a generic -ELMERR
which is defined in the set of special lvmlockd error
numbers.  Otherwise, an unknown lock manager error number
could be misinterpreted for something else if it happened
to overlap another set of error numbers (which they have
not thus far.)
2017-11-17 10:59:12 -06:00
David Teigland
e52d2e3bd8 lvmlockd: retry on other sanlock errors
These less common errors returned from sanlock should
also cause sanlock to retry the lock acquire:

- i/o timeout occurs during sanlock_acquire().
  other i/o on the same disk as the leases can cause
  sanlock i/o timeouts.

- low level disk paxos contention between hosts naturally
  causes one host to not acquire the lease.  There are a
  couple special error numbers associated with these cases
  that should just be recognized as a normal failure to
  acquire the lease.
2017-11-17 10:59:12 -06:00
Zdenek Kabelac
0c9e3e8df2 coverity: add some initilizers
Coverity cannot do a deeper analyzis so let's make just reports
go away and initialize them to 0.
2017-11-07 21:26:11 +01:00
Eric Ren
14d0b0bbdd clvmd: supress ENOENT error on testing connection
In HA cluster, we have "clvm" resource agent to manage clvmd daemon.
The agent invokes clvmd like: "clvmd -T90 -d0", which  always prints
a scaring error message:

"""
local socket: connect failed: No such file or directory
"""

When specifed with "-d" option, clvmd tries to check if an instance
of the clvmd daemon is already running through a testing connection.
The connect() will fail with this ENOENT error in such case, so supress
the error message in such case.

TODO: add missing error reaction code - since ofter log_error, program
is not supposed to continue running (log_error() is for reporting
stopping problems).

Signed-off-by: Eric Ren <zren@suse.com>
2017-11-07 21:24:39 +01:00
David Teigland
1b319f39d6 lvmlockd: check error for sanlock access to lvmlock LV
When the sanlock daemon does not have permission to access
the lvmlock LV, make the error messages more helpful.
2017-10-17 13:45:53 -05:00
Zdenek Kabelac
9940c2f754 dmeventd: schedule exit on break
When dmeventd receives SIGTERM/INT/HUP/QUIT it validates if exit is possible.
If there was any device still monitored, such exit request used to
be ignored/refused. This 'usually' worked reasonably well, however if there
is very short time period between last device is unmonitored and signal
reception - there was possibility such EXIT was ignored, as dmeventd has
not yet got into idle state even commands like 'vgchange -an' has already
finished.

This patch changes logic towards scheduling EXIT to the nearest
point when there is no monitored device.

EXIT is never forgotten.

NOTE: if there is only a single monitored device and someone sends
SIGTERM and later someone uses i.e. 'lvchange --refresh' after
unmonitoring dmeventd will exit and new instance needs to be
started.
2017-10-05 10:19:21 +02:00
Tony Asleson
32c87d56b1 lvmdbusd: thread stacks dump support
If you send a SIGUSR1 (10) to the daemon it will dump all the
threads current stacks to stdout.  This will be useful when the
daemon is apparently hung and not processing requests.

eg.
$ sudo kill -10 <daemon pid>
2017-09-27 07:45:00 -05:00
Tony Asleson
60e3dbd6d5 lvmdbusd: Give threads names
This will allow easier debug.
2017-09-27 07:45:00 -05:00
Tony Asleson
2074094e77 lvmdbusd: Main thread exception logging
Make sure that any and all code that executes in the main thread is
wrapped with a try/except block to ensure that at the very least
we log when things are going wrong.
2017-09-27 07:45:00 -05:00
Tony Asleson
bdccab07f9 lvmdbusd: Improve args comparison 2017-09-21 14:35:36 -05:00
Tony Asleson
7a6e438df8 lvmdbusd: Ensure vg_uuid is present
In some cases we are seeing where there are no VGs, but the data returned from
lvm shows that the PVs have the following for the VG:

"vg_name":"[unknown]", "vg_uuid":""

The code was only checking for the exitence of the VG name and we called into
the function get_object_path_by_uuid_lvm_id which requires both the VG name and
the LV name to exist (asserts this) which results in the following stack trace:

Traceback (most recent call last):
  File "/home/tasleson/lvm2/daemons/lvmdbusd/utils.py", line 563, in runner
    obj._run()
  File "/home/tasleson/lvm2/daemons/lvmdbusd/utils.py", line 584, in _run
    self.rc = self.f(*self.args)
  File "/home/tasleson/lvm2/daemons/lvmdbusd/fetch.py", line 26, in
		_main_thread_load
    cache_refresh=False)[1]
  File "/home/tasleson/lvm2/daemons/lvmdbusd/pv.py", line 48, in load_pvs
    emit_signal, cache_refresh)
  File "/home/tasleson/lvm2/daemons/lvmdbusd/loader.py", line 37, in common
    objects = retrieve(search_keys, cache_refresh=False)
  File "/home/tasleson/lvm2/daemons/lvmdbusd/pv.py", line 40, in
		pvs_state_retrieve
    p["pv_attr"], p["pv_tags"], p["vg_name"], p["vg_uuid"]))
  File "/home/tasleson/lvm2/daemons/lvmdbusd/pv.py", line 84, in __init__
    vg_uuid, vg_name, vg_obj_path_generate)
  File "/home/tasleson/lvm2/daemons/lvmdbusd/objectmanager.py", line 318,
		in get_object_path_by_uuid_lvm_id
    assert uuid
AssertionError
2017-09-21 14:35:36 -05:00
Tony Asleson
e3965d392c lvmdbusd: Fix hang in MThreadRunner
When executing in the main thread, if we encounter an exception we
will bypass the notify_all call on the condition and the calling thread
never wakes up.

@staticmethod
    def runner(obj):
        # noinspection PyProtectedMember
Exception thrown here
 ----> obj._run()
So the following code doesn't run, which causes calling thread to hang
	with obj.cond:
            obj.function_complete = True
            obj.cond.notify_all()

Additionally for some unknown reason the stderr is lost.
Best guess is it's something to do with scheduling a python function
into the GLib.idle_add.  That made finding issue quite difficult.
2017-09-21 14:35:36 -05:00
Zdenek Kabelac
03efec2712 deamonize: restore detection of errors
Keep forked environment for daemon more strick and check even
for nearly impossible to happen errors.
2017-09-06 11:47:53 +02:00
David Teigland
09c792c206 lvmlockd: fix check for no running lock manager
In some cases it was reporting there was no running
lock manager when there is.
2017-08-29 15:18:12 -05:00
David Teigland
f847fcd31a lvmlockd: print error about starting lock manager
In the case where lvmlockd is running, but no lock manager
is running, we should print a specific error message about
that situation.
2017-08-28 16:24:00 -05:00
Zdenek Kabelac
c8fdc5c087 cleanup: easier to read code
Split into lines for better reading.
2017-08-25 14:20:59 +02:00
Zdenek Kabelac
d79d919329 lvmlockd: log pthread_join errno code
Log possible errno with pthread_join (and one close() instance).
2017-08-25 14:20:59 +02:00
Zdenek Kabelac
da9a8fdedc lvmlockctl: fix check for failing close
On  close() failure it's -1.
2017-08-25 14:20:59 +02:00
Zdenek Kabelac
288e10cf8b lvmlockd: avoid double unlock of client_mutex
Avoid double unlocking of client_mutex and
and unlock client_mutex in 'else' branch
since it's already unlocked in 'if (cl->dead)' branch.
2017-08-25 14:20:59 +02:00
Zdenek Kabelac
b3b1e788e1 daemonize: more unified code
ATM we have several instances of daemonizing code.
Each has its 'special' logic so not completely easy
to unify them all into a single routine.

Start to unify them and use one strategy for rediricting
all input/outpus to /dev/null - use 'dup2' function for this
and open /dev/null before fork to make sure it's available.
2017-08-25 14:20:57 +02:00
David Teigland
e71c3ff187 lvmlockd: fix mutex unlock
Remove an unwanted pthread_mutex_unlock which would
lead to a double unlock.
2017-08-23 11:30:55 -05:00
David Teigland
46ddd5520c lvmlockd: add comment about temp ls name 2017-08-23 11:25:18 -05:00
Zdenek Kabelac
d4ce98de4d lvmlockd: shorter code
gcc warns here about storring 69 bytes in 64 byte array (losing
potentially 4 bytes from 'ls->name').

lvmlockd-core.c:2657:36: warning: ‘%s’ directive output may be truncated writing up to 64 bytes into a region of size 60 [-Wformat-truncation=]
  snprintf(tmp_name, MAX_NAME, "REM:%s", ls->name);
                                    ^~
lvmlockd-core.c:2657:2: note: ‘snprintf’ output between 5 and 69 bytes into a destination of size 64
  snprintf(tmp_name, MAX_NAME, "REM:%s", ls->name);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Replaced with slightly better code - but it still misses error path what
to do if the name would be truncated... - so added FIXME.

Also using all bytes for snprintf() buffer size
(as the size is with \0 included)
2017-08-22 10:23:31 +02:00
Zhang Huan
43305ae8da lvmlockd: reduce io impact for finding sanlock lv free lock offset
currently, lvcreate for sanlock find the free lock offset
from the beginning of the lvmlock every time.
after created thousands of lvs, it will issue thousands of read
ios for lvcreate to find free lock offset.
remeber the last free lock offset will greatly reduce the impact

Signed-off-by: Zhang Huan <zhanghuan@huayun.com>
2017-08-15 11:56:31 -05:00
David Teigland
8ecb5817c7 lvmlockd: global name doesn't apply to sanlock
When adopting locks, we shouldn't skip the special
dlm global lockspace name when using sanlock.
2017-08-07 10:46:03 -05:00
David Teigland
568c7ed6f1 lvmlockd: fix lm running check during adoption
When trying to adopt locks in startup, we want to ignore
a lock manager that isn't running, not fail.
2017-08-07 10:45:59 -05:00
Zdenek Kabelac
92b53a8077 configure: improve test for realtime clock
Check first if we need to even link -lrt  - since clock functions
are normally emebeded with recent  glibc (>=2.17)
Use standard  RT_LIBS name.
Avoid duplicate test for realtime clock with lvmlockd
Show better error message when realtime clock support is missing or
disabled.
Link  RT_LIBS explicitely with lvmlockd and lvmetad.
2017-08-01 14:03:54 +02:00
Zdenek Kabelac
00fdf01d9d makefiles: cleanups 2017-08-01 11:53:32 +02:00
Zdenek Kabelac
2232e82d25 makefiles: fixing linking
Avoid adding -g more then once for debug builds.
Avoid enabling  DEBUG_MEM when we build multithreaded tools.
Link executables with -fPIE -pie and --export-dynamic LDFLAGS
Introduce PROGS_FLAGS to add option to pass flags for external libs.
Link  lvm2 internally library only when really used.
Link DAEMON_LIBS with daemons.
Pass VALGRIND_CFLAGS internally
Set shell failure mode on couple places.
2017-08-01 11:53:30 +02:00
Zdenek Kabelac
1fd8785ff3 tidy: drop unneeded return 2017-07-20 11:20:22 +02:00
Zdenek Kabelac
0bf836aa14 tidy: prefer not using else after return
clang-tidy: avoid using  'else' after return - give more readable code,
and also saves indention level.
2017-07-20 11:18:29 +02:00
Zdenek Kabelac
567aa60fa1 lvmetad: cleanup
Avoid hashing insertion when object with same content is already there.
2017-07-17 12:33:17 +02:00
Zdenek Kabelac
f7e62bc55c cleanup: drop extra compare
dm_free() already validates for NULL itself.
2017-07-17 12:32:18 +02:00
Zdenek Kabelac
55f9e2f399 cleanup: shorten dump output code
Save couple lines on code with simpler code.
2017-07-17 12:32:18 +02:00
Zdenek Kabelac
f6c2ee57fa cleanup: drop const from allocated value
Avoid using const for casting to non-const.
2017-07-17 12:32:18 +02:00
Zdenek Kabelac
ba9820b142 numbers: strtod or strtoul need reset of errno
API for strtod() or strtoul() needs reset of errno, before it's being
called. So add missing resets in missing places and some also some
errno validation for out-of-range numbers.
2017-07-17 12:32:18 +02:00
Zdenek Kabelac
28e319ddc0 clvmd: fix valgrind memory report
Avoid reading already released memory and do a continue directly.

Invalid read of size 1
   at 0x1201B0: main_loop (clvmd.c:931)
   by 0x11F640: main (clvmd.c:666)
 Address 0x72ddef0 is 32 bytes inside a block of size 224 free'd
   at 0x4C30D18: free (vg_replace_malloc.c:530)
   by 0x54D6FD1: dm_free_wrapper (dbg_malloc.c:357)
   by 0x122E6E: process_work_item (clvmd.c:2034)
   by 0x123003: lvm_thread_fn (clvmd.c:2085)
   by 0x590A3A8: start_thread (pthread_create.c:465)
   by 0x5C3C7FE: clone (in /usr/lib64/libc-2.25.90.so)
 Block was alloc'd at
   at 0x4C2FB6B: malloc (vg_replace_malloc.c:299)
   by 0x54D6EF1: dm_malloc_aux (dbg_malloc.c:286)
   by 0x54D6F1C: dm_zalloc_aux (dbg_malloc.c:291)
   by 0x54D6F96: dm_zalloc_wrapper (dbg_malloc.c:345)
   by 0x11F89C: local_rendezvous_callback (clvmd.c:731)
   by 0x1203D2: main_loop (clvmd.c:964)
   by 0x11F640: main (clvmd.c:666)
2017-07-17 12:30:01 +02:00
Zdenek Kabelac
d7f92ea8ee clvmd: fix valgrind warning
Initialize mutex upfront any debugging and fix this report:

Mutex reinitialization: mutex 0x485d20, recursion count 0, owner 1.
   at 0x4C38480: pthread_mutex_init_intercept (drd_pthread_intercepts.c:821)
   by 0x4C38480: pthread_mutex_init (drd_pthread_intercepts.c:830)
   by 0x11F359: main (clvmd.c:562)
mutex 0x485d20 was first observed at:
   at 0x4C38F63: pthread_mutex_lock_intercept (drd_pthread_intercepts.c:885)
   by 0x4C38F63: pthread_mutex_lock (drd_pthread_intercepts.c:898)
   by 0x11E920: debuglog (clvmd.c:254)
   by 0x11F1D8: main (clvmd.c:527)
2017-07-17 12:29:57 +02:00
Zdenek Kabelac
919fa89482 lvmetad: fix memory leaks
Hash tables need to release no longer needed inserted data.
2017-07-17 12:27:53 +02:00
David Teigland
c995e40b63 lvmlockd: use DM_UUID_LEN for buffer size 2017-07-07 15:00:15 -05:00
Huan Zhang
bffae6c985 lvmlockd: miss adopt orphaned resources
1. dm_uuid is 68 byte length, but buf is 64 which
   will cause miss match uuid from lv lock manager
2. no lv lock_type path in dm config, use lock_args instead

Signed-off-by: Zhang Huan <zhanghuan@chinac.com>
2017-07-07 14:58:14 -05:00
Alasdair G Kergon
f2eda36cfa clvmd: Fix client list corruption
Centralise editing of the client list into _add_client() and
_del_client().  Introduce _local_client_count to track the size of the
list for debugging purposes.  Simplify and standardise the various ways
the list gets walked.

While processing one element of the list in main_loop(),
cleanup_zombie() may be called and remove a different element, so make
sure main_loop() refreshes its list state on return.  Prior to this
patch, the list edits for clients disappearing could race against the
list edits for new clients connecting and corrupt the list and cause a
variety of segfaults.

An easy way to trigger such failures was by repeatedly running shell
commands such as:
  lvs &; lvs &; lvs &;...;killall -9 lvs; lvs &; lvs &;...

Situations that occasionally lead to the failures can be spotted by
looking for 'EOF' with 'inprogress=1' in the clvmd debug logs.
2017-07-01 01:34:38 +01:00
Alasdair G Kergon
af789fd6d0 clvmd: add client id to debug log messages
Use standard format to make it easier to find the client to which each
debug log message refers.
2017-07-01 01:17:40 +01:00
Alasdair G Kergon
17ed254091 clvmd: add debuglog mutex
Log messages issued by different threads occasionally got intertwined.
2017-07-01 00:58:39 +01:00
Zdenek Kabelac
a533892cd3 coverity: checked_return of close
Check (or make quiet) close() ret code.
NOTE: there is another duplicated code of daemonize function which
should be converted to libdaemon.
2017-06-28 14:42:11 +02:00
Zdenek Kabelac
664e947726 coverity: add some error path for failed allocs
Coverity reports some unchecked allocations.
2017-06-27 00:27:36 +02:00
Zdenek Kabelac
feed61f3fa libdm: use rounded float for percent print
Use new added  dm_percent_to_round_float to enhance print
of percentage values.
2017-06-24 17:44:42 +02:00
Zdenek Kabelac
0016b79c8b dmeventd: improve more raid status reporting
When we want to report primary leg failure, check for intial 'a',
since otherwice 'Aa idle' is normally visible.

Also reset array of bit flags marking dead devices, once
plugin detects raid is in sync.
2017-06-24 00:06:12 +02:00
Zdenek Kabelac
653bdedb83 raid: plugin does not to use --config
Functionality of ignore suspend devices is already granted by:

lvm2_disable_dmeventd_monitoring() -> init_run_by_dmeventd() ->
init_ignore_suspended_devices().

In fact plugins should never use --config because it has
some unpleasant technical issues.
2017-06-23 23:32:40 +02:00
Jonathan Brassow
4c0e908b0a RAID (lvconvert/dmeventd): Cleanly handle primary failure during 'recover' op
Add the checks necessary to distiguish the state of a RAID when the primary
source for syncing fails during the "recover" process.

It has been possible to hit this condition before (like when converting from
2-way RAID1 to 3-way and having the first two devices die during the "recover"
process).  However, this condition is now more likely since we treat linear ->
RAID1 conversions as "recover" now - so it is especially important we cleanly
handle this condition.
2017-06-14 08:39:50 -05:00
Bryn M. Reeves
9a094350e0 dmfilemapd: do not wait if file has been truncated 2017-06-13 19:46:41 +01:00
Bryn M. Reeves
0cb628dfe2 dmfilemapd: update file block count at daemon start
The file block count stored in the filemap_monitor was lazily
initialised at the time of the first check. This causes problems
in the case that the file has been truncated between this time and
the time the daemon started: the initial block count and current
block count match and the daemon fails to detect a change.

Separate the setting of the block count from the check and make a
call to update the value at the start of _dmfilemapd().
2017-06-13 19:46:41 +01:00
Tony Asleson
61420309ee lvmdbusd: Prevent stall when update thread gets exception
If during the process of fetching current lvm state we experience an
exception we fail to call set_result on the queued_requests we were
processing.  When this happens those threads block forever which causes
the service to stall infinitely.  Only clear the queued_requests after
we have called set_result.
2017-06-02 12:39:04 -05:00
Tony Asleson
699ccc05ca lvmdbusd: Add background command to flight recorder
We were not adding background tasks to flight recorder.  Add the meta
data to the flight recorder when we start the command and update the meta
data when the command is finished.  Locking was added to meta data to
prevent concurrent update and returning string representation as these can
happen in two different threads.
2017-06-02 12:32:51 -05:00
Tony Asleson
192d142e1c lvmdbusd: cmdhandler.py vg_reduce, remove extranous '--all'
vgreduce previously allowed --all and --removemissing together even though
it only actual did the remove missing.  The lvm dbus daemon was passing
--all anytime there was no entries in pv_object_paths.  This change supplies
--all if and only if we are not removing missing and the pv_object_paths
is empty.

Vgreduce has and continues to enforce the invalid combination of supplying a
device list when you specify --all or --removemissing so we do not need
to check for that invalid combination explicitly in the lvm dbus service as
it's already covered.

Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1455471
2017-06-02 12:32:43 -05:00
Marian Csontos
8e99a46d09 lvmdbusd: Fix missed rename
Introduced By: e53454d6 (2.02.169)
Related: RHBZ#1348688
2017-05-18 12:52:46 +02:00
Marian Csontos
16c6d9f11a lvmdbusd: Fix notify_dbus mangling config option
If config option is passed by caller, dbusd appends to the option not to
the value, and also without using delimiter.

Bug: RHBZ#1451612
2017-05-17 15:35:20 +02:00
Zdenek Kabelac
455a4de090 dmeventd: restore multiple warnings
With recent updates for thin pool monitoring in version 169
we lost multiple WARNINGs to be printed in syslog, when
pool crossed  80%, 85%, 90%, 95%, 100%.

Restore this logic as we want to keep user informed more
then just once when 80% boundary is passed.
2017-05-10 15:39:36 +02:00
Bryn M. Reeves
a9940d16fe dmfilemapd: always initialise 'same' local variable (Coverity)
Fix a regression introduced in 70bb726 that allows a local variable
in the monitored file checking routine to be accessed before its
assignment when the file has already been unlinked.
2017-05-08 17:10:25 +01:00
Tony Asleson
405a3689bc lvmdbusd: Correct PV lookups
When a user does a Manager.PvCreate they can specify the block device using a
device path that may be different than what lvm reports is the device path.  For
example a user could use:

/dev/disk/by-id/wwn-0x5002538500000000 instead of /dev/sdc

In this case the pvcreate will succeed, but when we query lvm we don't find the
newly created PV. We fail because it's device path is returned as /dev/sdc.  This
change re-uses an internal lookup which can accommodate this and correctly find
the newly created PV.

Corrects https://bugzilla.redhat.com/show_bug.cgi?id=1445654
2017-05-05 10:30:06 -05:00
Bryn M. Reeves
7fbeea30e5 dmfilemapd: clear filemap_monitor before calling _parse_args()
If the wrong number of arguments are given, main() will attempt
to free the uninitialised pointer in fm.path.
2017-05-05 11:53:44 +01:00
Marian Csontos
7da13bbf7b dbus: Add --yes when --setphysicalvolumesize is used 2017-05-02 08:42:21 +02:00
Marian Csontos
5cf51fb2f7 dbus: log_debug needs qualifier
Adding qualifier makes the only unqualified log_debug occurence
consistent with other uses in the same file.

Other possible ways to fix this:

- using `from .utils import log_debug`
- moving the line below `from . import utils` line
2017-04-27 18:16:17 +02:00
Tony Asleson
0f31f10ac5 lvmdbusd: Improve error msg to include PV
Include PV device path when we believe it already exists.
2017-04-26 07:31:08 -05:00
Tony Asleson
e78329e281 lvmdbusd: Make sure we don't hang on lvcreate
If we happen to create a lv that has a previous signature we hang on y/n
prompt, add '--yes'.
2017-04-26 07:10:18 -05:00
Zdenek Kabelac
3e3f2a53ea dmfilemapd: free path
Cleanup resources before exit.
2017-04-12 21:33:47 +02:00
Zdenek Kabelac
dfc083d8d6 dmfilemapd: keep 1 byte for \0
Coverity noticed the buffer should have 1 byte for \0.
2017-04-12 21:33:47 +02:00
Jonathan Brassow
ba12a2e81a Typo: change loose to lose
loose (v): set free; release
lose (v) : be deprived of or cease to have or retain

We 'lose' redundancy or 'lose' meaning.
2017-04-12 10:28:19 -05:00
Bryn M. Reeves
5e74e43896 dmfilemapd: cast return value of strto* functions when needed 2017-03-30 10:02:31 +01:00
Bryn M. Reeves
834574cc27 dmfilemapd: use log_sys_error in _filemap_monitor_set_notify
Since _filemap_monitor_set_notify() is only called after daemon
start up it should not use the _early_log() macros. Instead, use
log_sys_error to log errors from the two syscalls in the function.
2017-03-30 10:01:57 +01:00
Bryn M. Reeves
bfc880994c dmfilemapd: cast sysconf() return value to int 2017-03-30 09:57:58 +01:00
Bryn M. Reeves
76b6cbea60 dmfilemapd: fix len type in _filemap_monitor_check_file_unlinked()
Make 'len' a size_t to match the return type of readlink(2).
2017-03-30 09:57:58 +01:00
Bryn M. Reeves
660bd4fe3e dmfilemapd: make filemap_monitor->nr_regions uint64_t 2017-03-30 09:57:58 +01:00
Zdenek Kabelac
dfdd6ccf3b vgrename: use long enough buffer for path
Use PATH_MAX when creating buffers for rename.
2017-03-30 00:38:41 +02:00
Bryn M. Reeves
13426092e9 dmfilemapd: free region table at end of _update_regions() (coverity) 2017-03-29 18:34:24 +01:00
Bryn M. Reeves
f66bc3dab0 dmfilemapd: fix off-by-one in fd comparison (coverity)
The function _filemap_monitor_check_file_unlinked() attempts to
test whether a fd value should be closed by comparison to zero:

        if ((fd > 0) && close(fd))
                log_error("Error closing fd %d", fd);

The test should be '>=' since 0 is a valid file descriptor.
2017-03-29 18:34:19 +01:00
Bryn M. Reeves
8658bbe3ee dmfilemapd: always close link check file descriptor (coverity)
Similar to 40fb91a, but for the file descriptor opened using the
link path reported by /proc/<self>/fd/<fd>.

The daemon opens a new file descriptor from /proc/<self>/fd when
checking for an unlinked file with mode=inode. Ensure that it is
always closed even if the same file test fails.
2017-03-29 18:34:11 +01:00
Bryn M. Reeves
7db51a6bed dmfilemapd: always terminate link buffer after readlink(2) (coverity) 2017-03-29 18:34:05 +01:00
Bryn M. Reeves
70bb726678 dmfilemapd: always close check file descriptor (coverity)
The daemon opens a new file descriptor from fm->path when checking
for an unlinked file with mode=inode. Ensure that it is always
closed even if the same file test fails.
2017-03-29 18:33:53 +01:00
Bryn M. Reeves
43fdbb8aeb dmfilemapd: use *argv[0] when testing for absolute path (coverity)
The path argument check in dmfilemapd incorrectly tests for:

  if (argv[0] == '/')

Rather than testing the 1st character in the string pointed to by
argv[0].
2017-03-29 18:33:42 +01:00
Bryn M. Reeves
b5252a51a5 dmfilemapd: pass correct dirp to closedir() 2017-03-29 16:54:44 +01:00
Bryn M. Reeves
481a522dd7 dmfilemapd: do not closdir(pid_d) unless it is open 2017-03-29 16:48:32 +01:00
Bryn M. Reeves
d71071a1af dmfilemapd: log closedir errors with log_sys_error() 2017-03-29 16:04:15 +01:00
Marian Csontos
a41d5b6491 dmfilemapd: Fix install target 2017-03-28 21:52:30 +02:00
Tony Asleson
862ca6e8b7 lvmdbusd: Rename ee to got_external_event
This variable is global, make it more descriptive.
2017-03-20 10:08:39 -05:00
Tony Asleson
b65a9230a3 lvmdbusd: Update state during pv move
Periodically update the state during pv move so that all the different
dbus objects reflect something close to reality during the process.
2017-03-20 10:08:39 -05:00
Tony Asleson
3ead4fb7ac lvmdbusd: Limit state refreshes for udev events
Udev events can come in like a flood when something changes.  It really
doesn't do us any good to refresh the state of the service numerous times
when 1 would suffice.  We had something like this before, but it was
removed when we added the refresh thread.  However, we have since learned
that we need to sequence events in the correct order and block dbus
operations if we believe the state has been affected, thus udev events are
being processed on the main work queue.  This change limits spurious
work items from getting to the queue.
2017-03-20 10:08:39 -05:00
Tony Asleson
7eeb093fdd lvmdbusd: Call add_no_notify for *move commands
Missed this in change when "add_no_notify" was added.  This was causing
extra external events to process when we did moves.
2017-03-20 10:08:38 -05:00
Tony Asleson
2dc71fc291 lvmdbusd: Only disable notify_dbus after getting external event
If we always disable the sending of notify dbus events then in the case
where all the users are lvm dbus users we will be in udev handling mode
until at least 1 external lvm command occurs.  Instead we will not disable
notify dbus until after we get at least 1 external event.  This makes the
service get into the correct mode of operation faster.
2017-03-20 10:08:38 -05:00
Heinz Mauelshagen
a37bb7b2a5 dmeventd: adjust mirror/raid DSOs to new repair design
Previous commit 506d88a2ec introduced disabling lvmetad on repairs.

Avoid calling lvscan and use of any --config options altogether
in the mirror and raid DSOs.

Related: rhbz1380521
2017-03-16 21:05:05 +01:00
David Teigland
73d028023a lvmetad: fix bug in snprintf of disable reason 2017-03-16 12:15:40 -05:00
David Teigland
506d88a2ec lvconvert: disable lvmetad for repair
Repairing missing devices does not work reliably
with lvmetad, so disable lvmetad before repair.
A standard lvmetad refresh (pvscan --cache) will
enable lvmetad again.
2017-03-16 11:50:36 -05:00
Heinz Mauelshagen
e5b6f2685a dmeventd: reintroduce fix mirror DSO to work with lvmetad
Commit 07ded8059c assumed that the mirror is blocked which is not the case.

It is accessible, degraded and in need of repair because some of its legs
(partially) failed.  Any auto-repair via dmeventd fails though because
of lvmetad not providing proper data about the failed PV(s).  That's why
this workaround got introduced in commit 76f6951c3e until we get to
the lvmetad interaction core issue.

Mind any mirror auto-repair failure is caused by such lvmetad interaction
problems not yet solved so disabling lvmetad works as a resort as elaborated
on in the related bz.

Reintroducing the interim solution.

Resolves: rhbz1380521
2017-03-16 14:19:06 +01:00
Zdenek Kabelac
07ded8059c mirror: revert 76f6951c3e
Effectively revert whole  76f6951c3e.
We need to figure out some other solution.

At this moment usage of --config  with 'repair' of blocked mirror
is 'freezing' combination.
2017-03-16 01:17:57 +01:00
Zdenek Kabelac
115fd205de mirror: avoid scanning
While mirror is blocked we can't try to scan device.
Regression introduce by previous commit
76f6951c3e.
2017-03-16 01:02:10 +01:00
Bryn M. Reeves
77a7ed065f dmfilemapd: ensure path argument is absolute
Require that the path argument to dmfilemapd be an absolute path
and document this in tool output, libdevmapper.h and dmfilemapd.8.

The check is also enforced by dm_stats_start_filemapd() to avoid
forking a new process with an invalid path argument.
2017-03-13 11:44:05 +00:00
Bryn M. Reeves
9b0aba5fe9 dmfilemapd: return error for wrong number of args
The initial check on argc incorrectly returns 1 when the wrong
number of arguments are present, causing a segfault in main()
when no arguments are given:

  # dmfilemapd
  Wrong number of arguments.
  usage: dmfilemapd <fd> <group_id> <path> <mode> [<foreground>[<log_level>]]
  Segmentation fault (core dumped)
2017-03-13 11:09:11 +00:00
Bryn M. Reeves
a9341b18ef daemons: use DEFAULT_PROC_DIR in dmfilemapd
Include defaults.h and use it to obtain the default procfs mount
point in dmfilemapd.
2017-03-10 17:26:47 +00:00
Bryn M. Reeves
b17eb054cd daemons: cleanup dmeventd references from dmfilemapd/Makefile.in 2017-03-10 17:26:47 +00:00
Tony Asleson
957e283de8 lvmdbusd: Simplify add_no_notify for loop 2017-03-09 16:39:47 -06:00
Tony Asleson
075f8e658c lvmdbusd: Correct utils.add_no_notify doc 2017-03-09 16:39:47 -06:00
Tony Asleson
00143753ee lvmdbusd: Add debug message to lvm ID lookup 2017-03-09 16:39:47 -06:00
Tony Asleson
0c0002d765 lvmdbusd: Correct src doc for Manager.UseLvmShell
It returns a boolean.
2017-03-09 16:39:47 -06:00
Tony Asleson
11cfc58976 lvmdbusd: Disable notify_dbus for service command use
Utilizing the --config option  we will utilize global/notify_dbus=0 so
that the service itself doesn't generate change events which it then needs to
process.
2017-03-09 16:39:47 -06:00
Tony Asleson
e53454d6de lvmdbusd: Use work queue for queries too
We need to place query operations in the queue to prevent the case where
a client knows of something before the service does.  For example if a
client creates a PV/VG/LV outside of the dbus API and then immediately
tries to lookup and use that resource in the lvm dbus service it should
be present.  By placing the queries in the work queue any previous
refresh operation will complete before we process the query.
2017-03-09 16:39:47 -06:00
Tony Asleson
470a1f1c50 lvmdbusd: Add FlightRecorderDump
Diagnostic method for dumping flight recorder to syslog.
2017-03-09 16:39:47 -06:00
Bryn M. Reeves
1ba11aa7ee dmfilemapd: always set the program_id after listing regions
The filemap daemon takes its program_id from the regions it is
managing: use DM_STATS_ALL_PROGRAMS when retrieving an initial
listing and then obtain the correct program_id from the group
leader.
2017-03-09 21:20:27 +00:00
Heinz Mauelshagen
76f6951c3e dmeventd: (workaround) fix mirror DSO to work with lvmetad
Automatic dmeventd repair of mirrors with active lvmetad configured
(mirror_image_fault_policy = "allocate") fails because the lvscan
run before the repair in the mirror DSO does not update the
lvmetad cache properly thus "lvconvert --repair ..." fails.

Need to scan the mirror LV before and after the repair
to have proper cache content after the repair finished.
The cache can't be relied on or the repair will fail.

Resolves: rhbz1380521
2017-03-09 20:41:07 +01:00
Bryn M. Reeves
8b78982297 daemons: add dmfilemapd
Add a daemon that can be launched to monitor a group of regions
corresponding to the extents of a file, and to update the regions as the
file's allocation changes.

The daemon is intended to be started from a library interface, but can
also be run from the command line:

  dmfilemapd <fd> <group_id> <path> <mode> [<foreground>[<log_level>]]

Where fd is a file descriptor open on the mapped file, group_id is the
group identifier of the mapped group and mode is either "inode" or
"path". E.g.:

  # dmfilemapd 3 0 vm.img inode 1 3 3<vm.img
  ...

If foreground is non-zero, the daemon will not fork to run in the
background. If verbose is non-zero, libdm and daemon log messages will
be printed.

It is possible for the group identifier to change when regions are
re-mapped: this occurs when the group leader is deleted (regroup=1 in
dm_stats_update_regions_from_fd()), and another region is created before
the daemon has a chance to recreate the leader region.

The operation is inherently racey since there is currently no way to
atomically move or resize a dm_stats region while retaining its
region_id.

Detect this condition and update the group_id value stored in the
filemap monitor.

A function is also provided in the the stats API to launch the filemap
monitoring daemon:

  int dm_stats_start_filemapd(int fd, uint64_t group_id, const char *path,
                              dm_filemapd_mode_t mode, unsigned foreground,
                              unsigned verbose);

This carries out the first fork and execs dmfilemapd with the arguments
specified.

A dm_filemapd_mode_t value is specified by the mode argument: either
DM_FILEMAPD_FOLLOW_INODE, or DM_FILEMAPD_FOLLOW_PATH. A helper function,
dm_filemapd_mode_from_string(), is provided to parse a string containing
a valid mode name into the appropriate dm_filemapd_mode_t value.
2017-03-09 18:50:15 +00:00
Zdenek Kabelac
05dd566a52 dmeventd: unify error handling
Always make sure the 'status' is release on 'error' path (thin pluging missed)
Make code looking same across all plugins.
2017-02-14 00:03:34 +01:00
Zdenek Kabelac
aa0c735e2c dmeventd: limit thin_command usage
Require usable command string to begining with '/'
So 'thin_command = "/some/path/command"' is the only supported variant
to internal 'lvm' command.
2017-02-13 09:43:53 +01:00
Zdenek Kabelac
0844b20f98 coverity: remove unneeded header files 2017-02-11 21:17:27 +01:00
Zdenek Kabelac
375e38709c cleanup: drop double const specifier
Remove duplicated 'const' specifier.
Reindent.
2017-02-11 20:30:16 +01:00
Zdenek Kabelac
2a9eda1229 mem: add extra mem pages for pthread stack
Some archs can use even 64K pages and then lvm2 runs into trouble if
the stack is 'too small' to fit extra page capturing stack overwrite.

So when lvm2 limits stack - add extra mem page - be it 4K or 64K.

Relates to ppc64le bug: https://bugzilla.redhat.com/1387279
2017-02-11 18:23:15 +01:00
Zdenek Kabelac
dae4f53acb clvmd: add mutex protection for cpg_ call
The library for corosync multicasting is not supporting multithread
usage - add local mutex to avoid parallel call of cpg_mcast_joined().
2017-02-05 17:55:37 +01:00
Tony Asleson
875ce04c61 lvmdbusd: cmdhandler.py, remove duplicate code
Move similar code to common functions, less is more!
2017-02-01 19:05:41 -06:00
Tony Asleson
3eccbb4b47 lvmdbusd: manager.py, remove duplicate code
Move similar code to common functions, less is more!
2017-02-01 18:57:01 -06:00
Tony Asleson
945842fa68 lvmdbusd: lvmdb.py, remove duplicate code
Move similar code to common functions, less is more!
2017-02-01 18:56:39 -06:00
Tony Asleson
681d69c70a lvmdbusd: pv.py, remove duplicate code
Move similar code to common functions, less is more!
2017-02-01 16:51:00 -06:00
Tony Asleson
a010cede6e lvmdbusd: vg.py, remove duplicate code
Move similar code to common functions, less is more!
2017-02-01 16:37:03 -06:00
Tony Asleson
83a1907586 lvmdbusd: lv.py, remove duplicate code
Move similar code to common functions, less is more!
2017-02-01 15:38:55 -06:00
Zdenek Kabelac
836eb122ce dmeventd_thind: set LVM_RUN_BY_DMEVENTD
Set LVM_RUN_BY_DMEVENTD envvar to expose the command is runing from
dmeventd environment.
2017-01-23 14:55:47 +01:00
Zdenek Kabelac
2e0605d6db dmeventd_thin: internal command without lvm prefix
Internal command processing needs to go without 'lvm ' prefix.
2017-01-21 17:42:19 +01:00
Zdenek Kabelac
85dab3963f dmeventd_thin: enable support for external command
With this commit we start to support configurable action
from thin-pool monitoring via  'dmeventd/thin_command'
2017-01-21 00:01:05 +01:00
Zdenek Kabelac
8c4f3633ac dmeventd_thin: new logic for calling commands
For more advanced support we need to ensure better logic for calling
external much more advanced script for maintanance of thin-pool.

So this new code ensures:

When thin-pool data or metadata is bigger then 50%,
then with each 5% increment, action is called.
This is independent from autoextend_threshold.
This action always happens when thin-pool is over threshold,
(so no action when it's exactly i.e. 60%).
The only exception is 100% full thin-pool - which invokes 'last'
action.

Since thin-pool occupancy may change also downward, code needs
to also handle possibly reduction of occupancy  of thin-pool.
So when usage drop from 90% to 50%, thin-pool will start to call
again action when it will pass 55% threshold.

This give external commands lot of option i.e. to call 'fstrim'
before actual resize is needed.
2017-01-20 23:58:56 +01:00
Zdenek Kabelac
8b95551ade dmeventd_thin: drop umounting on error path
Default internal logic will stop trying to do any 'rescue' action
when executed command fails.
This will be now fully in hands of external script if such
behaviour is needed.
2017-01-20 23:58:56 +01:00
Zdenek Kabelac
43e3268ada dmeventd_thin: rework failure handling
Instead of stopping monitoring after couple failing retries,
keep monitoring forever, just make larger delays between command
retries (ATM upto ~42 minutes).

So syslog is not spammed too often, yet commands have a chance to
be retried and succeed eventually...
2017-01-20 23:56:39 +01:00
Zdenek Kabelac
46c23dfb87 dmeventd_thin: SIGCHLD handler
To improve reaction time on when child is finished,
lets handle SIGCHLD in particular thread.
Let's hope kernel will route SIGCHLD to matching thread.
2017-01-20 23:55:51 +01:00
Zdenek Kabelac
bc7a1d70d4 dmeventd_thin: init command
When dmeventd configured command does not start with 'lvm ' prefix,
it's going to be an 'external' command.
In this case we split command by spaces to argv strings.
2017-01-20 23:55:50 +01:00
Zdenek Kabelac
14746a6c00 dmeventd_thin: add wait_pid
Add support handling command exit.
2017-01-20 23:55:50 +01:00
Zdenek Kabelac
2e935c0967 dmeventd_thin: add run_command
Implement forking of executable command.
When command is forked, dmeventd may continue monitor device.
2017-01-20 23:55:50 +01:00
Zdenek Kabelac
e5bef50827 dmeventd_thin: better warning logic
When fullness is passing WARN_THRESHOLD, print warning,
when it drops bellow and crossed again, we should print
warning again, but always only once.
2017-01-20 23:55:50 +01:00
Zdenek Kabelac
0d945ddbad dmeventd_thin: switch to struct percent
Later we can use stored percent values to pass them
to executed commands.
2017-01-20 23:55:50 +01:00
Zdenek Kabelac
eca964b554 dmeventd_thin: handling of internal command 2017-01-20 23:55:50 +01:00
Zdenek Kabelac
dd19b56985 thin: refresh status when error processing fails
When thin-pool processes event and 'lvextend --use-policies' fails
rather capture up-to-date new info as the fullness percentage may
have jumped noticable. This way we could use 'more' correct numbers
when checking for thresholds.
2016-12-22 23:37:07 +01:00
Tony Asleson
a7e1f973cc lvmdbusd: Use timeout_add instead
The function timeout_add_seconds has quite a bit of variability.  Using
timeout_add which specifies the timeout in ms instead of seconds.  Testing
shows that this is much more consistent which should improve clients that
are using shorter timeouts for the API and the connection.
2016-12-20 11:06:57 -06:00
Tony Asleson
75568294be lvmdbusd: Use cfg.reload() instead of dbo.refresh
We want to update the data and send out any signals as needed, not just
update the in memory database.
2016-12-20 11:06:57 -06:00
Tony Asleson
6fe6e8053a lvmdbusd: Remove un-needed main thread execution 2016-12-20 11:06:57 -06:00
Bryn M. Reeves
14c20ae217 lvmdbusd: add path.py to .gitignore 2016-12-10 17:59:07 +00:00
David Teigland
c459f23565 lvmetad: fix segfault in daemon_reply_simple
missing NULL termination
2016-12-09 15:22:30 -06:00
Zdenek Kabelac
f9c6c115d3 cleanup: easier code for raid plugin
Set bits only when then were not yet assigned.
2016-12-09 15:15:02 +01:00
Tony Asleson
c9848bb7ea lvmdbusd: Only allow 0..N for --blackboxsize argument 2016-11-30 16:47:39 -06:00
Tony Asleson
53ef496719 lvmdbusd: Emit signal on Job completion
Added a properties changed signal on the job dbus object so that client
can wait for a signal that the job is complete instead of polling or
blocking on the wait method.
2016-11-30 15:59:06 -06:00
Tony Asleson
1d52090953 lvmdbusd: Remove TODO on concurrent access to properties
As the code now uses a single thread to handle all changes to the
dbus model we no longer need to handle this potential race
condition.
2016-11-30 15:59:06 -06:00
Tony Asleson
37f05ccab1 lvmdbusd: Supress protected member access warning
We want _run to be protected so that users outside of the class
don't mistakenly use it.  It's for internal use only.
2016-11-30 15:59:06 -06:00
Tony Asleson
3bc69cb23c lvmdbusd: Add --blackboxsize command line argument
Allows the user to override the number of commands that get dumped
to the log when we encounter a lvm error.  Also useful during
development when you don't want to see the blackbox output.
2016-11-30 15:59:06 -06:00
Tony Asleson
b0757ac96e lvmdbusd: Remove unused variable
This variable has been un-used for a long time.
2016-11-30 15:59:06 -06:00
Tony Asleson
58f4d98af1 lvmdbusd: Remove debug log_error 2016-11-29 16:50:30 -06:00
Tony Asleson
617a819abc lvmdbusd: WS fix 2016-11-29 16:50:30 -06:00
Tony Asleson
d882edb32d lvmdbusd: Simplfy reading streams
Remove redundant code and make code paths the same for all streams.
2016-11-29 16:50:30 -06:00
Tony Asleson
25b5413f89 lvmdbusd: Make lvm shell read more robust
Make sure JSON is correct before we stop trying to read.
2016-11-29 16:50:30 -06:00
Zdenek Kabelac
4a59cfaa1d makefiles: drop file source and indent 2016-11-26 00:22:38 +01:00
Tony Asleson
064e24bc1e lvmdbusd: Only read whats buffered
When reading data from stdout & stderr we were reading until the
reading until we got None back which really isn't needed as the
read will return everything that is available.
2016-11-23 18:16:11 -06:00
Zdenek Kabelac
e38678be3f cleanup: single LVM_SYSTEM_DIR string 2016-11-23 17:55:03 +01:00
Zdenek Kabelac
5fd5cfe061 cleanup: use display_lvname and msg cleanup
Use display_lvname in tracing messages.
Add some missing 'dots' to messages.
2016-11-23 17:48:01 +01:00
Tony Asleson
0bbf631349 lvmdbusd: Remove extraneous finally
The print statement is incorrect.
2016-11-17 13:59:39 -06:00
Tony Asleson
3b4c5a2151 lvmdbusd: Fix source documentation 2016-11-17 13:58:17 -06:00
Tony Asleson
3a6c78e22a lvmdbusd: cmdhandler.py, fix imports
Use the correct py3 syntax for handling imports.  Also use print instead
of log_debug which in this context will produce no output.
2016-11-17 13:57:44 -06:00
Tony Asleson
19a0bf9df6 lvmdbusd: Remove debug JSON file 2016-11-17 11:45:11 -06:00
Tony Asleson
99806ec7ec lvmdbusd: Place Manager.UseLvmShell request on queue
We need to acquire a lock which can block us which in turn causes
the dbus request handling to block as well.  Place the request on
the work queue instead.
2016-11-17 11:35:16 -06:00
Tony Asleson
df92d330aa lvmdbusd: Extra report FD read on no data
Our expectation was that when using the lvm shell that when the lvm prompt
was read from stdout, that all other ouput had been written and flushed.
However, this doesn't appear to be the case.  Add extra read passes to
retrieve delayed report data.
2016-11-17 11:35:16 -06:00
Tony Asleson
a1805cf336 lvmdbusd: Bubble up invalid JSON
If we get invalid JSON, lets bubble that up to the user.
2016-11-17 11:35:16 -06:00
Tony Asleson
4f1171466f lvmdbusd: Remove un-used import 2016-11-17 11:35:16 -06:00
Tony Asleson
2fbb90e403 lvmdbusd: WS corrections
Fix mixed spaces & tabs and other ws issues.
2016-11-17 11:35:16 -06:00
Tony Asleson
dcdef9647b lvmdbusd: Add cfg.load to pv remove
The global dbus state does not match lvm with simply removing the pv
dbus object.
2016-11-17 11:35:16 -06:00
Tony Asleson
24fd1dd7a3 lvmdbusd: _lv_create, simplify return path 2016-11-17 11:35:16 -06:00
Tony Asleson
72a943f720 lvmdbusd: Long running operations use own thread
Anything that ends up using polld will be done in a unique thread
so that we don't hold off other operations from running concurrently.
2016-11-17 11:35:16 -06:00
Tony Asleson
fa444906bb lvmdbusd: Use one thread to fetch state updates
In preparation to have more than one thread issuing commands to lvm
at the same time we need to serialize updates to the dbus state and
retrieving the global lvm state.  To achieve this we have one thread
handling this with a thread safe queue taking and coalescing requests.
2016-11-17 11:35:16 -06:00
Tony Asleson
affe2cebf5 lvmdbus: Make bus name configurable
Add env LVM_DBUS_NAME to change what the bus name is.
2016-11-17 11:35:16 -06:00
Tony Asleson
461d340bd7 lvmdbusd: Remove the periodic timer task
This code is no longer needed because the back ground task has been
removed.  Will add back if we change the design and end up utilizing
multiple worker threads.
2016-11-03 18:38:10 -05:00
Tony Asleson
ee0c9e7b23 lvmdbusd: Take out background thread
There is no reason to create another background task when the task that
created it is going to block waiting for it to finish.  Instead we will
just execute the logic in the worker thread that is servicing the worker
queue.
2016-11-03 18:29:06 -05:00
Zdenek Kabelac
a9ee86ccf2 dmeventd: provide message context
Show device name with printed message.
Also use different level for printing error message and
standard informational message.
2016-11-03 17:49:07 +01:00
Zdenek Kabelac
0d95082aa9 cleanup: remove goto
Move goto path into 'if()' branch.
2016-11-03 17:49:07 +01:00
Zdenek Kabelac
9cbe4c1af9 log_info to log_very_verbose
Translate log_info() into log_very_verbose() which is macro
supposed to be used by our code.

log_info() is internal macro with eventually some 'symbolic' meaning
in syslogging daemons.
2016-11-03 17:49:07 +01:00
Zdenek Kabelac
cc19cc07f7 log_info to log_warn
Switch to log_warn level when we are reporting these message.
2016-11-03 17:49:07 +01:00
Zdenek Kabelac
e3775173b4 cleanup: log_info to log_error
Switch to more appropriate logging level.
2016-11-03 17:49:07 +01:00
Zdenek Kabelac
4b4d19e3aa dmeventd: separate dm and dmeventd logging
Ensure different logging function for dmeventd.c logging
and dm and lvm library.

We can recognize we want to show every log_info() and
log_notice() message from dmeventd.c code while not
exposing those from libdm/libdevmapper-event

Also switch to use log with errno - it's not changing
anything and doesn't bring any more features yet to dmeventd
logging but we just properly pass dm_errno_or_class properly
through the whole code stack for possible future use
(i.e. support of class logging for dmeventd).
2016-11-03 12:43:09 +01:00
Zdenek Kabelac
cd468b6218 dmeventd: debug only subsystemd with # sign
Reword the logging logic and try to restore previous logging
behavior for 'standalone' running daemon while preserving
debuggable feautures it has gained.

So actual rules:

dmeventd without any '-d' option will syslog all messages
from dmeventd.c it dmeventd plugins.

log_notice()==log_verbose()
log_info()==log_very_verbose()
But to show also log_debug() used has to give '-ddd'.

When user specified '-d, -dd, -ddd, -dddd' it
will also enable tracing of messages from libdm & lib
executed code - which is mainly useful for testing
i.e.: 'dmeventd -fldddd'
2016-11-03 12:43:09 +01:00
Zdenek Kabelac
33dd1f7747 dmeventd: abort on internal error
Provide same support for DM_ABORT_ON_INTERNAL_ERRORS we do have
with default libdm logging  as  dmeventd is libdm based tool.
2016-11-03 12:43:09 +01:00
Zdenek Kabelac
6af26273cb logging: add more log macros
Introduce macros:

log_level(), log_stderr(), log_once(), log_bypass_report()

For easier and more consisten way how to 'decoder' bits
of info from passed 'level'.

This patch fixes potential problem when 'level' of message
might not have always masked right bits.
2016-11-03 12:43:09 +01:00
Tony Asleson
96118a2508 lvmdbusd: Stop using threads for job wait
Instead of creating a thread to handle the case where a client
is calling job.Wait, we will utilize a timer.  This significantly
reduces the number of threads that get created and destroyed while
the service is running.
2016-11-02 16:39:13 -05:00
Tony Asleson
95abadd13c lvmdbusd: main.py: change debug msg text 2016-11-02 16:39:13 -05:00
Tony Asleson
60de09b00c lvmdbusd: Don't use dbus lib in worker thread
Simplify the code paths so that we don't utilize the dbus library code
when we are in worker thread context.

ref. https://bugs.freedesktop.org/show_bug.cgi?id=98521
2016-11-02 16:39:03 -05:00
Tony Asleson
38dd79307a lvmdbusd: Execute load in main thread
We will fetch the lvm state in non-main thread and only process the new
data with the main thread to prevent hanging the main thread event loop.

ref. https://bugs.freedesktop.org/show_bug.cgi?id=98521
2016-11-02 16:38:49 -05:00
Tony Asleson
24803bbaad lvmdbusd: Return results in main thread
Also introduce some additional new code to execute code other code
in main thread too.

ref. https://bugs.freedesktop.org/show_bug.cgi?id=98521
2016-11-02 16:38:00 -05:00
Tony Asleson
c8e8439b3d lvmdbusd: Use timer instead of thread
We had a thread sitting around for cleaning up other processes, changed to
a periodic timer task.
2016-11-02 16:35:45 -05:00
Tony Asleson
4fab833920 lvmdbusd: Remove log ouput when ec=0 & stderr != 0 bytes
lvm likes to log to stderr virtually all the time, this isn't
helpful.
2016-10-11 13:16:57 -05:00
Tony Asleson
2830f72288 lvmdbusd: Disable lvm abort on too much log output
The commit:
https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=34c55d98eefd88f85476c0f62f0649c706bde6f0

introduced an abort if lvm logs too much.  In the case of utilizing
lvm shell this is a pretty normal occurance, so we will disable
when we use lvm shell.
2016-10-11 13:16:57 -05:00
Tony Asleson
9f0195ec1e lvmdbusd: Ensure tmp dir gets cleaned up
Regardless of the outcome of starting up the lvm shell process, lets
ensure we clean up the temp directory and pipe.
2016-10-10 16:31:00 -05:00
Tony Asleson
2e941beb44 lvmdbusd: Ensure lvm shell still exists 2016-10-10 16:31:00 -05:00
Tony Asleson
a8bb8dfb08 lvmdbusd: Add LvCommon.DataPercent 2016-10-05 15:55:41 -05:00
Tony Asleson
d54ffcfcd1 lvmdbusd: Add LvCommon.MovePv
Needed for feature parity for lvm2app.
2016-10-05 15:28:42 -05:00
Tony Asleson
a3f24aaf5c lvmdbusd: Add properties to LvCommon
The following LvCommon properties were added so that the API
would have the same functionality as lvm2app has.

LvCommon.MetaDataSizeBytes
LvCommon.Attr
LvCommon.MetaDataPercent
LvCommon.CopyPercent
LvCommon.SnapPercent
LvCommon.SyncPercent
2016-10-05 13:59:38 -05:00
Zdenek Kabelac
9fe4f2337b cleanup: drop assign before use
Drop unneeded assigns singe vars are set later in code before
their first use (Coverity).
2016-10-03 17:49:55 +02:00
Zdenek Kabelac
00f883a4aa dmeventd: pthread_sigmask in single function
Integrate back _unblock_sigalrm() and check for error code of
pthread_sigmask() function so we do not use uninitialized
sigmask_t on error path (Coverity).
2016-10-03 17:47:28 +02:00
Zdenek Kabelac
ee04f1fcfd raid: dmeventd plugin use 64bit arithmetic
Coverity suggested to used 64bit unsigned ints instead of signed 32b int.
Assuming there is no user of >31legs raid array.
2016-10-03 17:44:25 +02:00
Tony Asleson
bd96036835 lvmdbusd: Use 'pv_missing' column instead of '[unknown]'
Previously using '[unknown]' for PV device path comparison which
is incorrect as it's not always true.
2016-09-28 12:07:48 -05:00
Tony Asleson
be5eb995d3 lvmdbusd: Remove extraneous parameter
Remove 'gen_new' parameter as it's not needed, correct documentation.
2016-09-28 12:05:44 -05:00
Tony Asleson
a93616cf66 lvmdbusd: Add diagnostic validate for look ups
Make sure that the lookup tables don't have extranoues stuff in
them.
2016-09-27 13:28:54 -05:00
Tony Asleson
063265eacd lvmdbusd: Allow PV device names to be '[unknown]'
When a PV device is missing lvm will return '[unknown]' for the device
path.  The object manager keeps a hash table lookup for uuid and for PV's
device name.  When we had multiple PVs with the same device path we
we only had 1 key in the table for the lvm id (device path).  This caused
a problem when the PV device transitioned from '[unknown]' to known as any
subsequent transitions would cause an exception:

Traceback (most recent call last):
  File "/usr/lib/python3.5/site-packages/lvmdbusd/request.py", line 66, in run_cmd
    result = self.method(*self.arguments)
  File "/usr/lib/python3.5/site-packages/lvmdbusd/manager.py", line 205, in _pv_scan
    cfg.load()
  File "/usr/lib/python3.5/site-packages/lvmdbusd/fetch.py", line 24, in load
    cache_refresh=False)[1]
  File "/usr/lib/python3.5/site-packages/lvmdbusd/pv.py", line 48, in load_pvs
    emit_signal, cache_refresh)
  File "/usr/lib/python3.5/site-packages/lvmdbusd/loader.py", line 80, in common
    cfg.om.remove_object(cfg.om.get_object_by_path(k), True)
  File "/usr/lib/python3.5/site-packages/lvmdbusd/objectmanager.py", line 153, in remove_object
    self._lookup_remove(path)
  File "/usr/lib/python3.5/site-packages/lvmdbusd/objectmanager.py", line 97, in _lookup_remove
    del self._id_to_object_path[lvm_id]
KeyError: '[unknown]'

when trying to delete a key that wasn't present.  In this case we don't add a
lookup key for the device path and the PV can only be located by UUID.

Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1379357
2016-09-27 13:28:54 -05:00
Tony Asleson
a882eb2b3b lvmdbusd: Clean up objectmanager.get_object_path_by_uuid_lvm_id
This method grew crufty with a number of stuble bugs.  Refactor and
simplify.
2016-09-27 13:28:54 -05:00