1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-27 14:04:05 +03:00

4550 Commits

Author SHA1 Message Date
Martin Schwenke
823edbf6fe scripts: Ensure even external scripts get tagged in logs as "ctdbd"
Our practice is to search logs for "ctdbd:".  We want to make sure we
find everything.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 5940a2494e9e43a83f2bca098bd04dfc1a8f2e93)
2013-04-22 13:58:36 +10:00
Martin Schwenke
fb8be43d6d eventscripts: Ensure directories are created
Previous commits stopped the top level of the script from creating
certain directories but some functions assume that required
directories exist.

Create those directories instead.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 0076cfc4666e5a96eb2c8affb59585b090840e00)
2013-04-22 13:58:36 +10:00
Martin Schwenke
903f4c394c scripts: Clean up update_tickles() and handling of associated directory
Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 700cf95a1f29b4b88460a00a55d57a9e397011e0)
2013-04-19 13:13:36 +10:00
Martin Schwenke
100a0eed90 scripts: Use $CTDB_SCRIPT_DEBUGLEVEL instead of something more complex
The current logic is horrible and creates an unnecessary file.  Let's
make the script debug level independent of ctddb's debug level.

* Have debug() use $CTDB_SCRIPT_DEBUGLEVEL directly

* Remove ctdb_set_current_debuglevel()

* Remove the "getdebug" command from ctdb stub in eventscript unit
  tests

* Update relevant eventscript unit tests to use
  $CTDB_SCRIPT_DEBUGLEVEL

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 85efa446c7f5c5af1c3a960001aa777775ae562f)
2013-04-19 13:13:36 +10:00
Martin Schwenke
f54dab03d5 scripts: Ensure service command is in $PATH in ctdb-crash-cleanup.sh
Move the use of the service command below inclusion of functions file,
which sets $PATH.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit d254d03f69cbdc3e473202b759af6e1392cbb59c)
2013-04-19 13:12:36 +10:00
Martin Schwenke
d24077922f initscript: Remove duplicate setting of $ctdbd
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit e7a4b7e35a1e4b826846e2494a3803abb57065ee)
2013-04-18 13:22:12 +10:00
Martin Schwenke
7ba42d2c89 util: Removed unused declaration of ctdbd_start()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 1e989894764e4cd1d551c44784d91cb295cd790d)
2013-04-18 13:22:12 +10:00
Martin Schwenke
7ccde44d30 include: Move ctdb_start_daemon() from ctdb_client.h to ctdb_private.h
It really is internal.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit abb64f62efaa70df4b87c030b96300eafd98e6a3)
2013-04-18 13:22:12 +10:00
Martin Schwenke
1f5bfde553 scripts: ctdb-crash-cleanup.sh uses initscript to see if ctdbd is running
"ctdb ping" can time out.  How many times should we try?

Instead, depend on the initscript to implement something sane.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 90cb337e5ccf397b69a64298559a428ff508f196)
2013-04-18 13:22:12 +10:00
Martin Schwenke
38366b6b53 initscript: Use a PID file to implement the "status" option
Using "ctdb ping" and "ctdb status" is fraught with danger.  These
commands can timeout when ctdbd is running, leading callers to believe
that ctdbd is not running.  Timeouts could be increased but we would
still have to handle potential timeouts.

Everything else in the world implements the "status" option by
checking if the relevant process is running.  This change makes CTDB
do the same thing and uses standard distro functions.

This change is backward compatible in sense that a missing
/var/run/ctdb/ directory means that we don't do a PID file check but
just depend on the distro's checking method.  Therefore, if CTDB was
started with an older version of this script then "service ctdb
status" will still work.

This script does not support changing the value of CTDB_VALGRIND
between calls.  If you start with CTDB_VALGRIND=yes then you need to
check status with the same setting.  CTDB_VALGRIND is a debug
variable, so this is acceptable.

This also adds sourcing of /lib/lsb/init-functions to make the Debian
function status_of_proc() available.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 687e2eace4f48400cf5029914f62b6ddabb85378)
2013-04-18 13:22:12 +10:00
Martin Schwenke
dcf1ac34ab ctdbd: Add --pidfile option
Default is not to create a pid file.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 996e74d3db0c50f91b320af8ab7c43ea6b1136af)
2013-04-18 13:21:59 +10:00
Martin Schwenke
7aa0a49cbd util: ctdb_fork() should call ctdb_set_child_info()
For now we pass NULL as the child name.  Later we'll give ctdb_fork()
and friends an extra argument and pass that through.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit ba8866d40125bab06391a17d48ff06a4a9f9da89)
2013-04-18 13:18:29 +10:00
Martin Schwenke
4ede763f3b util: New functions ctdb_set_child_info() and ctdb_is_child_process()
Must be called by all child processes.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 59b019a97aad9a731f9080ea5be14d0dbdfe03d6)
2013-04-18 13:18:29 +10:00
Michael Adam
f07d47a56d tests: add a comment to recovery db corruption test
The comment explains that we use "ctdb stop" and "ctdb continue"
but we should use "ctdb setcrecmasterrole off".

Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 06ac62f890299021220214327f1b611c3cf00145)
2013-04-17 21:16:45 +10:00
Amitay Isaacs
41e2a2e272 tests: Add a test for subsequent recoveries corrupting databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit b1577a11d548479ff1a05702d106af9465921ad4)
2013-04-17 21:16:45 +10:00
Amitay Isaacs
8cbf3211b7 tests: Support waiting for "recovered" state in tests
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 2438f3a4944f7adbcae4cc1b9d5452714244afe7)
2013-04-17 21:16:45 +10:00
Michael Adam
f1fe9ddf42 ctdb_call: don't bump the rsn in ctdb_become_dmaster() any more
This is now done in ctdb_ltdb_store_server(), so this
extra bump can be spared.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit cad3107b12e8392f786f9a758ee38cf3a3d58538)
2013-04-17 21:16:32 +10:00
Michael Adam
fd01c464d1 Fix a severe recovery bug that can lead to data corruption for SMB clients.
Problem:
Recovery can under certain circumstances lead to old record copies
resurrecting: Recovery selects the newest record copy purely by RSN. At
the end of the recovery, the recovery master is the dmaster for all
records in all (non-persistent) databases. And the other nodes locally
hold the complete copy of the databases. The bug is that the recovery
process does not increment the RSN on the recovery master at the end of
the recovery. Now clients acting directly on the Recovery master will
directly change a record's content on the recmaster without migration
and hence without RSN bump.  So a subsequent recovery can not tell that
the recmaster's copy is newer than the copies on the other nodes, since
their RSN is the same. Hence, if the recmaster is not node 0 (or more
precisely not the active node with the lowest node number), the recovery
will choose copies from nodes with lower number and stick to these.

Here is how to reproduce:

- assume we have a cluster with at least 2 nodes
- ensure that the recmaster is not node 0
  (maybe ensure with "onnode 0 ctdb setrecmasterrole off")
  say recmaster is node 1
- choose a new database name, say "test1.tdb"
  (make sure it is not yet attached as persistent)
- choose a key name, say "key1"
- all clustere nodes should ok and no recovery running
- now do the following on node 1:

1. dbwrap_tool test1.tdb store key1 uint32 1
2. dbwrap_tool test1.tdb fetch key1 uint32
   ==> 1
3. ctdb recover
4. dbwrap_tool test1.tdb store key1 uint32 2
5. dbwrap_tool test1.tdb fetch key1 uint32
   ==> 2
4. ctdb recover
7. dbwrap_tool test1.tdb fetch key1 uint32
   ==> 1
   ==> BUG

This is a very severe bug, since when applied to Samba's locking.tdb
database, it means that for SMB clients on clustered Samba there is
the potential for locking out oneself from previously opened files
or even worse, data corruption:

Case 1: locking out

- client on recmaster opens file
- recovery propagates open file handle (entry in locking.tdb) to
  other nodes
- client closes file
- client opens the same file
- recovery resurrects old copy of open file record in locking.tdb
  from lower node
- client closes file but fails to delete entry in locking.tdb
- client tries to open same file again but fails, since
  the old record locks it out (since the client is still connected)

Case 2: data corruption

- clien1 on recmaster opens file
- recovery propagates open file info to other nodes
- client1 closes the file and disconnects
- client2 opens the same file
- recovery resurrects old copy of locking.tdb record,
  where client2 has no entry, but client1 has.
- but client2 believes it still has a handle
- client3 opens the file and succees without
  conflicting with client2
  (the detached entry for client1 is discarded because
   the server does not exist any more).
=> both client2 and client3 believe they have exclusive
  access to the file and writing creates data corruption

Fix:

When storing a record on the dmaster, bump its RSN.

The ctdb_ltdb_store_server() is the central function for storing
a record to a local tdb from the ctdbd server context.
So this is also the place where the RSN of the record to be stored
should be incremented, when storing on the dmaster.

For the case of the record migration, this is currently done in
ctdb_become_dmaster() in ctdb_call.c, but there are other places
such as in recovery, where we should bump the RSN, but currently
don't do it.

So moving the RSN incrementation into ctdb_ltdb_store_server fixes
the recovery-record-resurrection bug.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit feb1d40b21a160737aead22e398f3c34ff3be8de)
2013-04-17 21:16:17 +10:00
Michael Adam
579d591015 logging: fix comment typo
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 4c0cbfbe8b19f2e6fe17093b52c734bec63dd8b7)
2013-04-17 12:44:26 +02:00
Michael Adam
b1a6289b44 ctdbd: unimplement the unused SET_DMASTER control
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 2e92deef5221ee651028ef87138b3113f1fece91)
2013-04-17 12:44:08 +02:00
Michael Adam
ca1f3de8b4 recoverd: remove bogus comment "qqq" from "add prototype new banning code"
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 9f01b8db72780acf2f88f1392bc0a796dd4c6176)
2013-04-17 12:43:48 +02:00
Michael Adam
eaad7c1e62 build: silence building of porting_test
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit e96acf19b4d1e0f951ab92b88869a01ff06398be)
2013-04-17 12:43:33 +02:00
Amitay Isaacs
ae5e2244ad traverse: Ensure backward compatibility for CTDB_CONTROL_TRAVERSE_ALL
This makes sure that CTDB_CONTROL TRAVERSE_ALL is compatible with older versions
of CTDB (i.e. 1.2.39 and 1.2.40 branches).

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 5808f0778b39b79ab7a5c7f53ad27947131386ec)
2013-04-17 12:31:14 +02:00
Amitay Isaacs
9e0f8fa09c traverse: Add CTDB_CONTROL_TRAVERSE_ALL_EXT to support withemptyrecords
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit e691df43d20871468142c8fb83f7c7303c4ec307)
2013-04-17 12:30:59 +02:00
Amitay Isaacs
43860e4ba0 tests: Fix typo in variable name
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 043e18a8324ccb2c8ddd7b323ebedb5b0de1298d)
2013-04-15 15:37:39 +10:00
Amitay Isaacs
e062555da0 tools/ltdbtool: Fix handling of -e option
Also, include description of -e option in usage.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 35264e42ade4676468cf7713fa339c784e932953)
2013-04-08 14:25:34 +10:00
Amitay Isaacs
77a29b3733 recoverd/takeover: Use IP->node mapping info from nodes hosting that IP
When collating IP information for IP layout, only trust the nodes that are
hosting an IP, to have correct information about that IP.  Ignore what all the
other nodes think.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 1c7adbccc69ac276d2b957ad16c3802fdb8868ca)
2013-04-08 11:14:32 +10:00
Amitay Isaacs
d931e73fb8 statd-callout: Make sure statd callout script always runs as root
In RHEL 6+, rpc.statd runs as "rpcuser" instead of root as on RHEL 5. This
prevents CTDB tool commands talking to daemon since "rpcuser" cannot access
CTDB socket.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Pair-Programmed-With: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit fe8c4880b371492a38554868d4ca10918c54e412)
2013-04-08 11:14:28 +10:00
Amitay Isaacs
41f0252da2 client: Set the socket non-blocking only after connect succeeds
If the socket is set non-blocking before connect, then we should catch
EAGAIN errors and retry. Instead of adding a random number of retries,
better to wait for connect to succeed and then set the socket to
non-blocking.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 524ec206e6a5e8b11723f4d8d1251ed5d84063b0)
2013-04-05 13:21:41 +11:00
Amitay Isaacs
b1b3d64380 Revert "client: handle transient connection errors"
This reverts commit dc0c58547cd4b20a8e2cd21f3c8363f34fd03e75.

There is a simpler solution that retrying random number of times. Do not set
socket non-blocking till connect succeeds.

(This used to be ctdb commit 74acc2c568300ef42740cf11299a1b2507047f60)
2013-04-05 13:21:02 +11:00
Volker Lendecke
d82336f1f3 common/messaging: Use the jenkins hash in ctdb_message
This give a better hash distribution

(This used to be ctdb commit f7f8bde2376f8180a0dca6d7b8d7d2a4a12f4bd8)
2013-04-05 13:13:08 +11:00
Volker Lendecke
a37033bfc9 common/messaging: use tdb_parse_record in message_list_db_fetch
This avoids malloc/free in a hot code path.

(This used to be ctdb commit c137531fae8f7f6392746ce1b9ac6f219775fc29)
2013-04-05 13:12:58 +11:00
Amitay Isaacs
9937adf0ca common/messaging: Abstract db related operations inside db functions
This simplifies the use of message indexdb API and abstracts tdb related code
inside the API.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit bf7296ce9b98563bcb8426cd035dbeab6d884f59)
2013-04-05 13:00:43 +11:00
Amitay Isaacs
8788e6318c common/messaging: Don't forget to free the result returned by tdb_fetch()
This fixes a memory leak in the messaging code.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 20be1f991dd75c2333c9ec9db226432a819f57ba)
2013-04-05 13:00:16 +11:00
Amitay Isaacs
96ad89f438 common/messaging: Free message list header if all message handlers are freed
This makes sure that even if the srvids are not deregistered, the header
structure is freed when the last message handler has been freed as a result of
client going away.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 4e1ec7412866f2d31c41de1bec0fbf788c03051b)
2013-04-05 12:59:25 +11:00
Sumit Bose
d2ed7429b7 build: Fix for tevent autoconf check
The list of include files is the 4th argument of AC_CHECK_DECLS.

(This used to be ctdb commit 85b777196289646ca37e06ebbf1f7a684d0aabc5)
2013-03-27 12:20:17 +11:00
Amitay Isaacs
dd050cd4ba util: Add hex_decode_talloc() to decode hex string into a binary blob
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 307416afda707b687f5e89e8438e45c154a4c806)
2013-03-25 17:45:23 +11:00
Amitay Isaacs
7f88fe3d05 logging: Do not ignore stdout/stderr from the exec'd children
To log debugging information from child processes that are started
with vfork and exec, do not set close_on_exec on STDOUT and STDERR for
that process.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 08c53ee609b80f87450a7a1d7dd24fbcdf5ab7bc)
2013-03-25 17:41:37 +11:00
Michael Adam
257af5b62a server:persistent: fix a debug message (copy'n'paste error)
Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 87c89b7c2a14e2ee79a3efc7e8125842bc04bf23)
2013-03-12 14:02:08 +01:00
Volker Lendecke
3ffd57f0c9 fix a typo
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 98abd344342a011a8599411deae79f94abc09541)
2013-03-12 14:02:08 +01:00
Amitay Isaacs
d4407a6516 common/io: For scheduling immediate events use tevent_schedule_immediate
tevent_schedule_immediate() is much more efficient at handling events that need
to be processed immediately rather than creating timed events with
timeval_zero().

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 11734be353a1e246163eda631d35dfe55d1d6fb1)
2013-03-06 15:32:37 +11:00
Amitay Isaacs
5d7efb4cf1 ctdbd: Add an index db for message list for faster searches
When CTDB is busy with lots of smbd, CTDB was spending too much time in
daemon_check_srvids() which searches a list of srvids in the registered
message handlers.  Using a hash based index significantly improves the
performance of search in a linked list.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 3e09f25d419635f6dd679b48fa65370f7860be7d)
2013-03-06 15:32:33 +11:00
Martin Schwenke
8cd6a67b8b tools/ctdb: delip no longer fails if IP can not be moved
Moving the IP is an optimisation so should not cause failure.

Refactor and simplify the retry-move-IP into new function
try_moveip().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 5402f85dde045576cbaf64e01c68e28ed52204e8)
2013-02-27 16:01:55 +11:00
Michael Adam
12d07dd1c6 server:persistent: fix a comment typo.
Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 6455ce5e4980a63d56ed30f7059869c8356c12ea)
2013-02-22 11:37:03 +01:00
Martin Schwenke
2476d8a9fd recoverd: update_capabilities() should use connected nodes
... as the comment says... not just active nodes.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 4f71dca8df19a63f198e2d6d59e605b49ec5e803)
2013-02-20 14:51:24 +11:00
Martin Schwenke
7f727e5d32 client: Refactor node listing functions to use list_of_nodes()
This reduces repetition.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit f505020a5720faa4ecc6414e0bfaa6b3c0e47291)
2013-02-20 14:44:38 +11:00
Martin Schwenke
dab2f6817d client: New generic node listing function list_of_nodes()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit a73bb56991b8c07ed0e9517ffcf0dc264be30487)
2013-02-20 14:44:38 +11:00
Amitay Isaacs
a2abdc1353 common/io: Rewrite socket handling code to read all available data
This improves the processing of packets considerably.  It has been
observed that there can be as many as 10 packets in the socket buffer and
the current code of reading a single packet from a socket at a time is
not very optimal.  This change reads all the bytes from socket buffer and
then parses to extract multiple packets.  If there are multiple packets,
set up a timed event to process next packet.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit d788bc8f7212b7dc1587ae592242dc8c876f4053)
2013-02-19 17:18:21 +11:00
Martin Schwenke
f9bbbaef7e doc: Fix typo in ctdbd manpage
Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 855ab348901edb3ec1327499a43f509d279b8182)
2013-02-15 13:12:26 +11:00
Amitay Isaacs
1d3eebbca4 ctdbd: Fix the PullDBPreallocation size to 10MB as intended
In 1f262deaad0818f159f9c68330f7fec121679023, Ronnie changed recovery code
to allocate chunks of 10MB in traverse_pulldb() and traverse_recdb().  The
tunable PullDBPreallocation size was set to 100MB.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit e204fac03412520e877ab04363b3ece02667c55b)
2013-02-14 09:40:35 +11:00