1
0
mirror of https://github.com/samba-team/samba.git synced 2025-02-03 13:47:25 +03:00

8528 Commits

Author SHA1 Message Date
Amitay Isaacs
963a639101 ctdb-tests: Add tests for cmdline_add() api
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Nov 14 12:03:46 UTC 2019 on sn-devel-184
2019-11-14 12:03:46 +00:00
Amitay Isaacs
e469d6c119 ctdb-common: Add api to add new section/commands to cmdline
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-11-14 10:38:34 +00:00
Amitay Isaacs
977a6f7fad ctdb-common: Change cmdline implementation to support multiple sections
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-11-14 10:38:34 +00:00
Amitay Isaacs
7a008c6b74 ctdb-tests: Update cmdline tests for section name
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-11-14 10:38:34 +00:00
Amitay Isaacs
b2b24c91fa ctdb-common: Add section to group commands in cmdline
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-11-14 10:38:34 +00:00
Amitay Isaacs
29948d7b1e ctdb-common: Generate usage message from cmdline_parse()
If any of the option parsing or command parsing fails, generate usage
message.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-11-14 10:38:34 +00:00
Martin Schwenke
e45feaf28d ctdb-tcp: Simplify freeing of transport data on shutdown
The type-checking is superfluous and gets in the way of readability.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Thu Nov 14 03:45:44 UTC 2019 on sn-devel-184
2019-11-14 03:45:44 +00:00
Martin Schwenke
750f3938e4 ctdb-daemon: Rename ctdb_context private_data to transport_data
This gives a casual reader a useful clue.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-11-14 02:20:46 +00:00
Martin Schwenke
53f8492caa ctdb-daemon: Rename ctdb_node private_data to transport_data
This gives a casual reader a useful clue.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-11-14 02:20:46 +00:00
Volker Lendecke
a6d99d9e5c ctdb-tcp: Close inflight connecting TCP sockets after fork
Commit c68b6f96f26 changed the talloc hierarchy such that outgoing TCP sockets
while sitting in the async connect() syscall are not freed via
ctdb_tcp_shutdown() anymore, they are hanging off a longer-running structure.
Free this structure as well.

If an outgoing TCP socket leaks into a long-running child process (possibly the
recovery daemon), this connection will never be closed as seen by the
destination node. Because with recent changes incoming connections will not be
accepted as long as any incoming connection is alive, with that socket leak
into the recovery daemon we will never again be able to successfully connect to
the node that is affected by this leak. Further attempts to connect will be
discarded by the destination as long as the recovery daemon keeps this socket
alive.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14175
RN: Avoid communication breakdown on node reconnect

Signed-off-by: Martin Schwenke <martin@meltin.net>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-11-14 02:20:46 +00:00
Amitay Isaacs
816205027a ctdb-ib: Fix build errors for infiniband transport
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Nov 13 13:31:10 UTC 2019 on sn-devel-184
2019-11-13 13:31:10 +00:00
Andrew Bartlett
92ce387ed0 build: Remove workaround for missing os.path.relpath in Python < 2.6
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: David Mulder <dmulder@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-11-13 08:42:30 +00:00
Volker Lendecke
f5f89b1b99 ctdb: Use TALLOC_FREE() in a few places
We have a macro for NULLing out the pointer

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Nov  8 01:35:11 UTC 2019 on sn-devel-184
2019-11-08 01:35:11 +00:00
Martin Schwenke
bf99f82077 ctdb-tests: Make process exists test more resilient
This can fail as follows:

--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--
Running test ./tests/UNIT/tool/ctdb.process-exists.003.sh (02:26:30)
--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--
ctdb.process-exists.003      - ctdbd process with multiple connections on node 0
Setting up fake ctdbd
<10||0|
OK
<10|PID 26107 exists
|0|
OK
==================================================
Running "ctdb -d NOTICE process-exists 26107 0x1234567812345678"
PASSED
==================================================
Running "ctdb -d NOTICE process-exists 26107 0xaebbccdd12345678"
Registered SRVID 0xaebbccdd12345678
--------------------------------------------------
Output (Exit status: 1):
--------------------------------------------------
PID 26107 with SRVID 0xaebbccdd12345678 does not exist
--------------------------------------------------
Required output (Exit status: 0):
--------------------------------------------------
PID 26107 with SRVID 0xaebbccdd12345678 exists

FAILED
connection to daemon closed, exiting
==========================================================================
TEST FAILED: ./tests/UNIT/tool/ctdb.process-exists.003.sh (status 1) (duration: 0s)
==========================================================================

This happens when dummy_client has not registered the SRVID (for its
10th connection) before the 2nd simple_test.

Change the initial wait to ensure that the SRVID is registered.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Nov  6 02:46:24 UTC 2019 on sn-devel-184
2019-11-06 02:46:24 +00:00
Martin Schwenke
dd9d5ec5c8 ctdb-tests: Improve code quality in ctdb_init()
Improve quoting and indentation.  Print a clear error if the cluster
goes back into recovery and doesn't come back out.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-11-06 01:22:30 +00:00
Martin Schwenke
3b5ed00054 ctdb-tests: No longer retry starting the cluster
Retrying like this hides bugs.  The cluster should come up first time,
every time.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-11-06 01:22:30 +00:00
Martin Schwenke
bf47bc18bb ctdb-tcp: Drop tracking of file descriptor for incoming connections
This file descriptor is owned by the incoming queue.  It will be
closed when the queue is torn down.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14175
RN: Avoid communication breakdown on node reconnect

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-11-06 01:22:30 +00:00
Martin Schwenke
d0baad257e ctdb-tcp: Avoid orphaning the TCP incoming queue
CTDB's incoming queue handling does not check whether an existing
queue exists, so can overwrite the pointer to the queue.  This used to
be harmless until commit c68b6f96f26664459187ab2fbd56767fb31767e0
changed the read callback to use a parent structure as the callback
data.  Instead of cleaning up an orphaned queue on disconnect, as
before, this will now free the new queue.

At first glance it doesn't seem possible that 2 incoming connections
from the same node could be processed before the intervening
disconnect.  However, the incoming connections and disconnect occur on
different file descriptors.  The queue can become orphaned on node A
when the following sequence occurs:

1. Node A comes up
2. Node A accepts an incoming connection from node B
3. Node B processes a timeout before noticing that outgoing the queue is writable
4. Node B tears down the outgoing connection to node A
5. Node B initiates a new connection to node A
6. Node A accepts an incoming connection from node B

Node A processes then the disconnect of the old incoming connection
from (2) but tears down the new incoming connection from (6).  This
then occurs until the originally affected node is restarted.

However, due to the number of outgoing connection attempts and
associated teardowns, this induces the same behaviour on the
corresponding incoming queue on all nodes that node A attempts to
connect to.  Therefore, other nodes become affected and need to be
restarted too.

As a result, the whole cluster probably needs to be restarted to
recover from this situation.

The problem can occur any time CTDB is started on a node.

The fix is to avoid accepting new incoming connections when a queue
for incoming connections is already present.  The connecting node will
simply retry establishing its outgoing connection.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14175

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-11-06 01:22:30 +00:00
Martin Schwenke
e62b3a05a8 ctdb-tcp: Check incoming queue to see if incoming connection is up
This makes it consistent with the reverse case.  Also, in_fd will soon
be removed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14175

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-11-06 01:22:30 +00:00
Björn Jacke
a456c2bb02 ctdb/utils/smnotify/smnotify.c: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:39 +00:00
Björn Jacke
1e73161bdd ctdb/utils/scsi_io/scsi_io.c: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:39 +00:00
Björn Jacke
f3754b6487 ctdb/server/ctdb_daemon.c: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:39 +00:00
Björn Jacke
5d2a257c2e ctdb/server/ctdb_client.c: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:39 +00:00
Björn Jacke
7722bd80fc ctdb/server/ctdb_call.c: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:38 +00:00
Björn Jacke
9fa37484c3 ctdb/include/ctdb_private.h: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:38 +00:00
Björn Jacke
493705dc27 ctdb/ib/ibwrapper_test.c: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:38 +00:00
Björn Jacke
540325d3cb ctdb/ib/ibw_ctdb.c: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:38 +00:00
Björn Jacke
b9d7b85afd ctdb/doc/readonlyrecords.txt: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:38 +00:00
Björn Jacke
cb867b29c9 ctdb/doc/ctdb.1.xml: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:38 +00:00
Björn Jacke
93859b3394 ctdb/doc/ctdb-tunables.7.xml: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:38 +00:00
Björn Jacke
37398acca5 ctdb/doc/ctdb-statistics.7.xml: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:38 +00:00
Björn Jacke
1b51b44487 ctdb/common/srvid.h: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:38 +00:00
Björn Jacke
c71a1df18a ctdb/client/client.h: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:38 +00:00
Martin Schwenke
6de5706b4d ctdb-tests: Add vacuuming tests
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Thu Oct 24 05:28:21 UTC 2019 on sn-devel-184
2019-10-24 05:28:21 +00:00
Martin Schwenke
49262a6bc4 ctdb-tests: Add handling of process clean-up on a cluster node
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-10-24 04:06:44 +00:00
Martin Schwenke
b9654085f5 ctdb-tests: Factor out function check_cattdb_num_records()
This can be use in multiple vacuuming tests.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-10-24 04:06:44 +00:00
Martin Schwenke
5a6d319eea ctdb-tests: Add ctdb-db-test tool
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-10-24 04:06:44 +00:00
Martin Schwenke
439ef65d29 ctdb-client: Factor out function client_db_tdb()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-10-24 04:06:44 +00:00
Martin Schwenke
41a41d5f3e ctdb-daemon: Implement DB_VACUUM control
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-10-24 04:06:43 +00:00
Martin Schwenke
d462d64cdf ctdb-vacuum: Only schedule next vacuum event if vacuuuming is scheduled
At the moment vacuuming is always scheduled.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-10-24 04:06:43 +00:00
Martin Schwenke
13cedaf019 ctdb-daemon: Factor out code to create vacuuming child
This changes the behaviour for some failures from exiting to simply
attempting to schedule the next run.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-10-24 04:06:43 +00:00
Martin Schwenke
5539edfdbe ctdb-vacuum: Simplify recording of in-progress vacuuming child
There can only be one, so simplify the logic.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-10-24 04:06:43 +00:00
Martin Schwenke
496204feb0 ctdb-protocol: Add marshalling for control DB_VACUUM
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-10-24 04:06:43 +00:00
Martin Schwenke
a896486b62 ctdb-protocol: Add marshalling for struct ctdb_db_vacuum
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-10-24 04:06:43 +00:00
Martin Schwenke
b314835341 ctdb-protocol: Add new control CTDB_CONTROL_DB_VACUUM
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2019-10-24 04:06:43 +00:00
Amitay Isaacs
d0cc9edc05 ctdb-vacuum: Avoid processing any more packets
All the vacuum operations if required have an event loop to ensure
completion of pending operations.  Once all the steps are complete,
there is no reason to process any more packets.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-24 04:06:43 +00:00
Amitay Isaacs
680df07630 ctdb-daemon: Avoid memory leak when packet is deferred
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-24 04:06:43 +00:00
Amitay Isaacs
c6427dddf5 ctdb-recoverd: No need for database detach handler
The only reason for recoverd attaching to databases was to migrate
records to the local node as part of vacuuming.  Recovery daemon does
not take part in database vacuuming any more.

The actual database recovery is handled via the recovery_helper and
recovery daemon should not need to attach to the databases any more.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-24 04:06:43 +00:00
Amitay Isaacs
fc81729dd2 ctdb-recoverd: Drop VACUUM_FETCH message handling
This is now implemented in the ctdb daemon using VACUMM_FETCH control.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-24 04:06:43 +00:00
Amitay Isaacs
498932c0e8 ctdb-vacuum: Replace VACUUM_FETCH message with control
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-24 04:06:42 +00:00