1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-11 05:18:09 +03:00
Commit Graph

8204 Commits

Author SHA1 Message Date
Amitay Isaacs
9bdd6814e4 ctdb-packaging: Update library versions to upstream versions
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2018-12-18 07:12:09 +01:00
Amitay Isaacs
59e244c9d0 ctdb-packaging: Match configure command as per spec file
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2018-12-18 07:12:09 +01:00
Amitay Isaacs
4443124fe8 ctdb-packaging: Call waf with python wrapper
This allows to build packages even when python3 is not available by
setting PYTHON variable.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2018-12-18 07:12:09 +01:00
Amitay Isaacs
9912709eca ctdb-build: Use open() instead of file() for python3
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2018-12-18 07:12:09 +01:00
Amitay Isaacs
1e061ff1e3 ctdb-tool: Avoid data uninitialized warnings
../../tools/ctdb.c: In function 'str_to_data':
../../tools/ctdb.c:624: warning: 'data.dsize' may be used uninitialized in this function
../../tools/ctdb.c:624: warning: 'data.dptr' may be used uninitialized in this function

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2018-12-18 07:12:09 +01:00
Martin Schwenke
1ed91f0e10 ctdb-tests: Do not force TEST_VAR_DIR to be absolute
This can result in Unix domain socket names that are too long.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue Dec 18 05:31:00 CET 2018 on sn-devel-144
2018-12-18 05:31:00 +01:00
Martin Schwenke
61b54193fe ctdb-event: Force symbolic link targets to be absolute
If CTDB_BASE is relative then symbolic link targets will be incorrect.

Don't force CTDB_BASE to be absolute because this can result in Unix
domain socket names that are too long.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-12-18 02:02:04 +01:00
Martin Schwenke
108aca0a9e ctdb-event: Declare and construct data_script only if needed
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-12-18 02:02:04 +01:00
Martin Schwenke
63a4c634a6 ctdb-tests: Force symbolic link targets to be absolute
If CTDB_BASE is relative then the symbolic link target will be
incorrect.

Don't force CTDB_BASE to be absolute because this can result in Unix
domain socket names that are too long.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-12-18 02:02:04 +01:00
Martin Schwenke
45f96c7346 ctdb-event: Force EVENTSCRIPTS_TESTS_VAR_DIR to be absolute
Event scripts (well, statd_callout) can change directory, causing
stubs to be unable to locate EVENTSCRIPTS_TESTS_VAR_DIR if it is
relative.

Don't force TEST_VAR_DIR to be absolute because this can result in
Unix domain socket names that are too long.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-12-18 02:02:04 +01:00
Martin Schwenke
ba83aa9d8b ctdb-event: Force script directory to be absolute
If TEST_VAR_DIR is relative then symbolic link targets will be
incorrect.

Don't force TEST_VAR_DIR to be absolute because this can result in
Unix domain socket names that are too long.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-12-18 02:02:03 +01:00
Martin Schwenke
da8aaf2aee ctdb-recoverd: Call an election when the recovery lock is lost
The lock may have been lost due to a failure in the underlying locking
mechanism.  This could be due to quorum loss or similar.  It is best
to call an election to confirm that this node should still be master.
At worst, the node will reelect itself, fail to take the lock and then
ban itself.  This is a suitable outcome for a node that has been
partitioned from others in the cluster.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-12-18 02:02:03 +01:00
Martin Schwenke
9d1d5fa4ac ctdb-doc: Add non-breaking space to lock_buckets documentation
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-12-18 02:02:03 +01:00
Martin Schwenke
93284ed032 ctdb-daemon: Divide by 2 when calculating hop count bucket
This provides finer resolution while still maintaining a reasonable
maximum.  In this case the top bucket contains any hop counts
>= 16384, compared to the current situation where the top bucket contains
hop counts >= 268435456.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-12-18 02:02:03 +01:00
Andrew Bartlett
5ddff307b4 build: Move python detection back into waf (instead of in configure and Makefile)
This avoids creating a mini-configure in the configure script.

Users wishing to use python2 to build need to specify PYTHON=
to both ./configure and make

After we merged the python3 change, it became clear that relying on systems prefixing
the correct python just causes trouble and make debugging harder, so only use $PYTHON
for the override, not the default case

This essentially reverts a660b7fb8e but
leaves the files more consistent.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2018-12-14 14:40:19 +01:00
Noel Power
a660b7fb8e PY3: switch current build to use python3
Make sure default make and configure for all now defaults
to building with python3.

To build a samba (or sub component e.g. talloc etc.) with python3
  ./configure && make

To build a samba (or sub component e.g. talloc etc.) with python2
  PYTHON=python ./configure && PYTHON=python make

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-10 10:38:25 +01:00
Christof Schmitt
7d271450f7 ctdb: Remove <file> parameter from pfetch usage info
The code does not implement saving the record data to a file, so update
the usage info accordingly.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Dec 10 05:02:13 CET 2018 on sn-devel-144
2018-12-10 05:02:12 +01:00
Christof Schmitt
81de8f0641 ctdb: Fix hex to int conversion in h2i
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-12-10 02:01:14 +01:00
Swen Schillig
353a947b4a ctdb: Adding memory pool for queue callback
The received packet is copied into a newly allocated memory chunk for further
processing by the assigned callback. Once this is done, the memory is free'd.
This is repeated for each received packet making the memory allocation / free
an expensive task. To optimize this process, a memory pool is defined which
is sized identically to the queue's buffer.
During tests it could be seen that more than 95% of all messages were sized
below the standard buffer_size of 1k.

Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Christof Schmitt <cs@samba.org>

Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Fri Dec  7 23:27:16 CET 2018 on sn-devel-144
2018-12-07 23:27:16 +01:00
Swen Schillig
382705f495 ctdb: Introduce buffer.offset to avoid memmove
The memmove operation is quite expensive, therefore,
a new buffer attribute "offset" is introduced to support
an optimized buffer processing.
The optimization is to "walk" through the buffer and process
each packet until the buffer is fully processed (empty)
without requiring any memmove.
Only if a packet is in-complete, the buffer content is moved
and the new data is read from the queue.
This way almost all memmove operations are eliminated.

Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Christof Schmitt <cs@samba.org>
2018-12-07 19:56:16 +01:00
Stefan Metzmacher
f87d6cbfff ctdb/wscript: make use of MODE_{644,744,755,777}
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-05 13:35:19 +01:00
Stefan Metzmacher
8ba0a9a1ab ctdb/wscript: use python 3.6 compatible functions
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-05 13:35:19 +01:00
Martin Schwenke
dd7574afd1 ctdb-daemon: Exit with error if a database directory does not exist
Since 4.9.0, the log messages can be confusing if a required database
directory does not exist.  Explicitly check for database directories,
logging a clear error and exiting if one is missing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13696

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Dec  3 06:56:41 CET 2018 on sn-devel-144
2018-12-03 06:56:41 +01:00
Olly Betts
28aeb86a9f Fix spelling mistakes
Signed-off-by: Olly Betts <olly@survex.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-30 03:35:13 +01:00
Andreas Schneider
ab5f26f3f3 ctdb: Use #ifdef instead of #if for config.h definitions
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-11-28 23:19:21 +01:00
Martin Schwenke
c1dd6382e3 ctdb-tests: Make the debug hung script test cope with unreadable stacks
Ideally this would just involve using "test -r".  However, operating
system security features may mean that kernel stacks are not readable
even though they appear to be.

Instead, try reading that stack of a process on the test node.  If
that succeeds then so should reading the stack of the "stuck" sleep
process in the test.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13684

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>

Autobuild-User(master): Tim Beale <timbeale@samba.org>
Autobuild-Date(master): Thu Nov 15 08:15:32 CET 2018 on sn-devel-144
2018-11-15 08:15:32 +01:00
Andreas Schneider
008b9652ca ctdb: Fix an out of bound array access
Found by cppcheck.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13680

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-14 05:07:15 +01:00
Andreas Schneider
2d512b278e debug: Use debuglevel_(get|set) function
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Nov  8 11:03:11 CET 2018 on sn-devel-144
2018-11-08 11:03:11 +01:00
Martin Schwenke
6e16e95f74 ctdb-daemon: Do not fork when CTDB_TEST_MODE is set
Explicitly background ctdbd instead.

This has the advantage of leaving stdin open.  ctdbd can then be
enhanced to exit when stdin closes, allowing better cleanup in a test
environment.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue Nov  6 10:30:14 CET 2018 on sn-devel-144
2018-11-06 10:30:14 +01:00
Martin Schwenke
01f6fbba4e ctdb-daemon: Switch interactive variable to a bool
popt uses an int in place of a bool, so declare an extra int and make
the conversion explicit.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:18 +01:00
Martin Schwenke
804bdf9719 ctdb-tests: Add local_daemons.sh onnode and socket commands
These aren't used by simple tests but they will be useful for
integrating ctdbd local daemons into other test suites and for
debugging.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:18 +01:00
Martin Schwenke
19de5f463d ctdb-tests: Use local_daemons.sh in local_daemons.bash
The etc-ctdb/ subdirectory containing the event script moves into the
top-level tests/ directory because the subdirectory is really now
owned by local_daemons.sh instead of simple/.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:17 +01:00
Martin Schwenke
46fd4f144e ctdb-tests: Add local_daemons.sh
This provides a separate script for handling local daemons.  It can be
used for testing outside of the CTDB simple test suite.  It is
installed as ctdb_local_daemons.

The logic is copied from ctdb/tests/simple/scripts/local_daemons.bash.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:17 +01:00
Martin Schwenke
25efb924bf ctdb-tests: Allow use of setup_ctdb_base() outside of test cases
Always create an empty event script directory.  If $TEST_SUBDIR is
unset then don't use it to look for etc-ctdb/.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:17 +01:00
Martin Schwenke
9679c9e9b6 ctdb-build: Don't set unused variable TEST_BIN_DIR
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:17 +01:00
Martin Schwenke
aa0c7ccfc8 ctdb-tests: Move setting of ctdb_dir and top_dir
These are only used in script_install_paths.sh.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:17 +01:00
Martin Schwenke
1d86fd537a ctdb-tests: Use $CTDB_SCRIPTS_TOOLS_BIN_DIR
Don't calculate this locally as _tools_dir.  Add it to PATH
unconditionally - this may result in duplicate entries in PATH but the
resulting code is simpler.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:17 +01:00
Martin Schwenke
7ac5e7ae44 ctdb-tests: Use $CTDB_SCRIPTS_TESTS_BINDIR
Don't calculate this locally as _test_bin_dir.  Just calculate
top_dir, source script_install_paths.sh and use
$CTDB_SCRIPT_TESTS_BINDIR.

Don't bother sanity checking if TEST_BIN_DIR is set.  It will go away
soon.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:17 +01:00
Martin Schwenke
bde73c7a04 ctdb-tests: Add new variable CTDB_SCRIPTS_TESTS_BINDIR
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:17 +01:00
Martin Schwenke
2cb82ef453 ctdb-tests: Change all cluster setup to use ctdb_test_init()
ctdb_test_init() now passes any arguments to setup_ctdb().

Update tests that have custom local daemon configuration to call
ctdb_test_init() directly.  Remove the redundant, initial call to
ctdb_test_init() to avoid starting the cluster an extra time.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:17 +01:00
Martin Schwenke
9a2910c60b ctdb-tests: Drop passing of test arguments to ctdb_test_init()
Arguments are currently ignored anyway.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:17 +01:00
Martin Schwenke
d762e52e65 ctdb-tests: Drop ctdb_restart_when_done()
This no longer does anything.  Integration test cases now start and
shut down the cluster.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:16 +01:00
Martin Schwenke
79db138de5 ctdb-tests: Drop dependency on variable ctdb_test_restart_scheduled
The remainder of the scheduled restart logic is about to be removed,
so produce debugging information any time the cluster is not healthy.

While here, reindent and drop the else since there is already an early
return before it.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:16 +01:00
Martin Schwenke
bc8df7191c ctdb-tests: Drop tests that only start and stop daemons
Integration test cases now start and shut down the cluster.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:16 +01:00
Martin Schwenke
412fb7b7d5 ctdb-tests: Move enabling of event scripts to setup_ctdb()
This is for the real cluster case.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:16 +01:00
Martin Schwenke
aa2ee4bea8 ctdb-tests: Improve signal handling trap
Interrupting a test run currently moves on to the next test.  It
should exit.

Follow the practice of exiting with 128 + signal number.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:16 +01:00
Martin Schwenke
92337234e5 ctdb-tests: Drop cleanup_handler()
Running testsuite-specific code here isn't a good option.

Daemons are now shut down in ctdb_test_exit(), even when testing is
interrupted.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:16 +01:00
Martin Schwenke
0e9ead8f28 ctdb-tests: Start daemons in ctdb_test_init(), stop them in ctdb_test_exit()
This makes tests self-contained.  They can also now be individually
looped, if necessary.

Most tests (all but 1 complex, more than 50% of simple) restart the
daemons anyway, so this simplification is worth it.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:16 +01:00
Martin Schwenke
e733e4cb74 ctdb-tests: Ignore SIGPIPE during simple test cleanup
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:16 +01:00
Martin Schwenke
6d9c89bfa3 ctdb-tests: Drop setting of unused variable scriptname
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:16 +01:00
Martin Schwenke
520568051c ctdb-tests: Drop use of confusing testfailures variable
Exit on first test failure instead of setting a variable.  The bizarre
logic in ctdb_test_exit() makes this worth dropping.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:16 +01:00
Martin Schwenke
9ebcebe519 ctdb-tests: Drop useless "ctdb version" test
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:16 +01:00
Martin Schwenke
43c26e1e64 ctdb-tests: Rationalise tunable simple tests
These 3 tests duplicate various checks and can easily be handled as a
single test.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:15 +01:00
Martin Schwenke
ba86eacb66 ctdb-tests: Rationalise ctdb stop/continue/disable/enable simple tests
The "continue" and "enable" tests are just extensions of the "stop"
and "disable" tests, so drop the latter 2.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:15 +01:00
Martin Schwenke
5fdac517fa ctdb-tests: Use wait_until_node_has_no_ips() in some tests
This strengthens those tests to ensure that released IPs aren't
replaced with others.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:15 +01:00
Martin Schwenke
eda1296d67 ctdb-tests: Add function wait_until_node_has_no_ips()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:15 +01:00
Martin Schwenke
44019b5577 ctdb-event: Only run talloc report if CTDB_INTERACTIVE is set
This is only really wanted for interactive testing when logging to
stderr.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:15 +01:00
Martin Schwenke
e952f0316b ctdb-event: Never fork to become daemon in eventd
This stops ctdbd from being able to shut down eventd, since the PID it
records will be invalid.  There's no need for eventd to fork.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:15 +01:00
Martin Schwenke
4e6bd42493 ctdb-daemon: Improve documentation for -i option
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:15 +01:00
Martin Schwenke
9c41481f21 ctdb-daemon: Don't set log_to_stdout for become_daemon()
ctdbd logs to stderr in interactive mode, not stdout.  This way stdout
is always closed.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:15 +01:00
Martin Schwenke
c84254d23d ctdb-daemon: Avoid unnecessarily spamming the logs when in test mode
Logging the logging location to syslog can be useful on production
systems when the configuration goes unexpectedly missing.  However, in
test mode this just adds noise to the logs on the test system.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:14 +01:00
Martin Schwenke
1bbc4fad43 ctdb-tools: Detect unknown node number
If there aren't enough addresses in the list then the shift will
silently fail and the printed address will be the unshifted value of
$1, which is incorrect/unexpected.  So, sanity check the node number.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:14 +01:00
Martin Schwenke
08469408c3 ctdb-tests: README updates
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:14 +01:00
Martin Schwenke
4f4a835c34 ctdb-tests: Remove export of CTDB_SOCKET
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:14 +01:00
Martin Schwenke
d246b1dadf ctdb-tests: Use path_socket() in dummy client
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:14 +01:00
Martin Schwenke
3b1e5977d8 ctdb-tests: Drop incorrect comment, unused function
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:14 +01:00
Martin Schwenke
82e589e388 ctdb-tests: Drop setting of CTDB_SOCKET and CTDB_PIDFILE
The local daemons ssh stub doesn't need to do this because the ctdbd
and the ctdb tool now only need CTDB_TEST_MODE and CTDB_BASE for local
daemon tests.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:14 +01:00
Martin Schwenke
d75fa2c3fd ctdb-daemon: Drop unused function ctdb_set_socketname()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:14 +01:00
Martin Schwenke
5f478b7c5f ctdb-daemon: Use path functions for socket and PID file
Drop the use of ctdb_set_sockname() because it complicates the memory
allocation and this is the only place it is used.  Just assign to the
relevant pointer.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:14 +01:00
Martin Schwenke
cd021596da ctdb-tests: Use path_socket() in test client tools
Just leak the memory allocated by path_socket().  This is only used in
short-lived test programs, so it isn't worth the hassle of plumbing a
talloc context through several layers to get here.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:14 +01:00
Martin Schwenke
cc3aedd307 ctdb-tools: Use path_socket() in ctdb tool
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:14 +01:00
Martin Schwenke
38566780d2 ctdb-tests: Use ctdb-path for fake_ctdbd directory setup
This needs to be done before any of the code changes are made,
including updating the ctdb tool.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:13 +01:00
Martin Schwenke
d4a1f897af ctdb-tests: Use ctdb-path-like values for local daemons socket and PID file
However, don't use ctdb-path itself because some tests use nested
instances of onnode.  The outermost instance would set CTDB_SOCKET and
any inner instance would pick up that value, regardless of CTDB_BASE.

This is a temporary measure to avoid breaking testing while use of the
path functions is added to ctdbd and the ctdb tool.  When this is
complete these variables can be removed altogether because the code
will just depend on CTDB_TEST_MODE and CTDB_BASE.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:13 +01:00
Martin Schwenke
32c2ec8fa2 ctdb-common: Allow path_socket() to use $CTDB_SOCKET
Use of CTDB_SOCKET is being generally removed.  However, this override
is being added to allow test code outside of ctdb/ to be able to
specify the socket, if desired.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:13 +01:00
Martin Schwenke
27df4f002a ctdb-recovery: Ban a node that causes recovery failure
... instead of applying banning credits.

There have been a couple of cases where recovery repeatedly takes just
over 2 minutes to fail.  Therefore, banning credits expire between
failures and a continuously problematic node is never banned,
resulting in endless recoveries.  This is because it takes 2
applications of banning credits before a node is banned, which
generally involves 2 recovery failures.

The recovery helper makes up to 3 attempts to recover each database
during a single run.  If a node causes 3 failures then this is really
equivalent to 3 recovery failures in the model that existed before the
recovery helper added retries.  In that case the node would have been
banned after 2 failures.

So, instead of applying banning credits to the "most failing" node,
simply ban it directly from the recovery helper.

If multiple nodes are causing recovery failures then this can cause a
node to be banned more quickly than it might otherwise have been, even
pre-recovery-helper.  However, 90 seconds (i.e. 3 failures) is a long
time to be in recovery, so banning earlier seems like the best
approach.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13670

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Nov  5 06:52:33 CET 2018 on sn-devel-144
2018-11-05 06:52:33 +01:00
Martin Schwenke
fbea9d3699 ctdb-daemon: Fix valgrind hit in event code
==25741== Syscall param write(buf) points to uninitialised byte(s)
==25741==    at 0x4939291: write (write.c:27)
==25741==    by 0x4868285: sys_write (sys_rw.c:68)
==25741==    by 0x13915D: sock_queue_trigger (sock_io.c:316)
==25741==    by 0x4DE6478: tevent_common_invoke_immediate_handler (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DE64A2: tevent_common_loop_immediate (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DEBE5A: ??? (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DEA2D6: ??? (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DE57E3: _tevent_loop_once (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x15D1BA: ctdb_event_script_args (eventscript.c:821)
==25741==    by 0x13B437: ctdb_start_daemon (ctdb_daemon.c:1315)
==25741==    by 0x110642: main (ctdbd.c:393)
==25741==  Address 0x57888a4 is 100 bytes inside a block of size 144 alloc'd
==25741==    at 0x48357BF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==25741==    by 0x4B9B7C0: talloc_named_const (in /usr/lib/x86_64-linux-gnu/libtalloc.so.2.1.14)
==25741==    by 0x15CCC6: eventd_client_write (eventscript.c:430)
==25741==    by 0x15CCC6: eventd_client_run (eventscript.c:556)
==25741==    by 0x15CCC6: ctdb_event_script_run (eventscript.c:649)
==25741==    by 0x15D198: ctdb_event_script_args (eventscript.c:812)
==25741==    by 0x13B437: ctdb_start_daemon (ctdb_daemon.c:1315)
==25741==    by 0x110642: main (ctdbd.c:393)
==25741==

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Oct 22 09:27:15 CEST 2018 on sn-devel-144
2018-10-22 09:27:15 +02:00
Amitay Isaacs
a190960380 ctdb-event: Check the return status of sock_daemon_set_startup_fd
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2018-10-22 06:04:20 +02:00
Amitay Isaacs
80549927bc ctdb-common: Set close-on-exec for startup fd
The startup_fd should not be propagated to the child processes created
from a daemon.  It should only be used in the daemon code to return the
status of the startup.  Another use of startup_fd is to notify the
parent if the daemon process has exited.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2018-10-22 06:04:20 +02:00
Martin Schwenke
c9e1603a5d ctdb-daemon: Exit if eventd goes away
ctdbd enters a broken state if eventd goes away.  A clean shutdown is
not possible because that involves running events.  Restarting eventd
is possible but this might mask a serious problem and it is possible
that eventd might keep on disappearing.  Just exit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-22 06:04:20 +02:00
Martin Schwenke
a3d12252fa ctdb-daemon: Return early when refusing to run an event script
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-22 06:04:20 +02:00
Martin Schwenke
80f3f7c188 ctdb-tests: Improve counting of database records
Record counts are sometimes incomplete for large databases when
relevant tests are run on a real cluster.

This probably has something to do with ssh, pipes and buffering, so
move the filtering and counting to the remote end.  This means that
only the count comes across the pipe, instead of all the record data.

Instead of explicitly excluding the key for persistent database
sequence numbers, just exclude any key starting with '_'.  Such keys
are not used in tests.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Oct  8 05:36:11 CEST 2018 on sn-devel-144
2018-10-08 05:36:11 +02:00
Martin Schwenke
52dcecbc92 ctdb-tests: Add extra debug to large database recovery test
This test sometimes fails, probably because the test is flakey.
Either the records aren't being added correctly or the counting of
records loses records.  Try to debug both possibilities.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:23 +02:00
Martin Schwenke
d67d8ed44a ctdb-tests: Shut down transaction_loop clients more cleanly
A transaction_loop client can exit with a transaction active when its
time limit expires.  This causes a recovery and causes problems with
the test cleanup, which detects unwanted recoveries and fails.

Set a flag when the time limit expires and exit cleanly before the
next transaction is started.

Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:23 +02:00
Martin Schwenke
2aa006a311 ctdb-tools: Have onnode pass -n option even when regular ssh not in use
ONNODE_SSH is really a test hook, so it doesn't need to support
completely random values.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:23 +02:00
Martin Schwenke
6ac5124b01 ctdb-tests: Support closing of stdin in local daemons ssh stub
Not sure this is needed but this makes it behave the same as ssh.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:23 +02:00
Martin Schwenke
0dfb3c87b5 ctdb-tests: Be more careful when building public IP addresses
The goal is to allow more local daemons by expanding the address range
rather than generating invalid addresses.

For IPv6, use a separate address space instead of an offset for the
2nd address.

For IPv4, use the last 2 octets with addresses starting at
192.168.100.1 and 192.168.200.1.  Avoid addresses with 0 and 255 in
the last octet by using a maximum of 100 addresses per "subnet"
starting at .1.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:23 +02:00
Martin Schwenke
36eb738877 ctdb-tests: Be more careful when building node addresses
The goal is to allow more local daemons by expanding the address range
rather than generating invalid addresses.

For IPv6, use all 4 trailing hex digits.

For IPv4, use the last 2 octets.  Although 127.0.0.0 is a /8 network,
avoid unexpected issues due to 0 and 255 in the last octet.  Use a
maximum of 100 addresses per "subnet" starting at .1.  Keep the first
group of addresses in 127.0.0.0/24 to continue to allow a reasonable
number of nodes to be tested with socket-wrapper.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:23 +02:00
Martin Schwenke
03dddc37b5 ctdb-tests: Don't format IPv4 octets as hex digits
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:22 +02:00
Martin Schwenke
0eabac5295 ctdb-tests: Be more efficient about starting/stopping local daemons
Don't loop, just use onnode all.

For shutting down, use onnode -p all.  This results in a significant
time saving for stopping many deamons because "ctdb shutdown" is now
synchronous.

onnode -p all can be used to start daemons directly because they
daemonize.  However, this does not work under valgrind because the
valgrind process does not exit, so onnode will wait forever for it.
In this case, use onnode without the -p option.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:22 +02:00
Martin Schwenke
a9ac33015b ctdb-tests: Do not use ctdbd_wrapper in local daemon tests
Run the daemon directly and shut it down using ctdb shutdown.

The wrapper waits for ctdbd to reach >=FIRST_RECOVERY runstate within
a timeout period and shuts ctdbd down if that doesn't happen.  This is
only really used to ensure that ctdbd doesn't exit early after an
apparently successful start.  There are no known cases where ctdbd
will continue running but fail to reach >=FIRST_RECOVERY runstate.

When ctdbd is started in tests, the test code will wait until ctdbd is
in a healthy state on all nodes before proceeding, so there is
effectively no change in behaviour.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:22 +02:00
Martin Schwenke
8bde6fa09c ctdb-tests: Don't remove non-existent test database directory
This directory is no longer used.  Lack of removal doesn't seem to
cause a problem.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:22 +02:00
Martin Schwenke
f2e4a5e9fa ctdb-tests: Drop unused function maybe_stop_ctdb()
There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:22 +02:00
Martin Schwenke
2cd6a00399 ctdb-tests: Explicitly check for local daemons when shutting down
This is clearer if the logic is explicit...  and...

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:22 +02:00
Martin Schwenke
90f6b0a1ed ctdb-tests: Drop functions daemons_start(), daemons_stop()
There are too many functions to start/stop daemons.  Simplify this.

Inline the functionality into ctdb_start_all() and ctdb_stop_all().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:22 +02:00
Martin Schwenke
f1ede41adf ctdb-tests: Don't used daemons_start()/daemons_stop() directly in tests
There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:22 +02:00
Martin Schwenke
4642a347d0 ctdb-tests: Rename _ctdb_start_all() -> ctdb_start_all()
There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:22 +02:00
Martin Schwenke
f57e5bbde7 ctdb-tests: Rename ctdb_start_all() -> ctdb_init()
There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:21 +02:00
Martin Schwenke
a66a96934a ctdb-tests: Drop ps_ctdbd()
This was used for debugging tests by ensuring that the arguments to
ctdbd were as expected.  It no longer outputs anything useful because
ctdbd is now started without arguments.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-10-08 02:46:21 +02:00
Amitay Isaacs
83b3c5670d ctdb-tests: Drop code for RECEIVE_RECORDS control
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2018-10-08 02:46:21 +02:00
Amitay Isaacs
2f89bd96fb ctdb-protocol: Drop marshalling code for RECEIVE_RECORDS control
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2018-10-08 02:46:21 +02:00