IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
On systems with 32 bit offsets, expansion and fcntl locking on these records
will fail anyway. SAMBA already does '#define _FILE_OFFSET_BITS 64' in
config.h (on my 32-bit x86 Linux system at least) to get 64 bit file offsets.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 252f7da702)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 2d768f664e6db65b3b7e0c732f33ee2b806892f9)
This reverts commit 401f421fa003d9515df15e759b50b56e0c67d69c.
Conflicts:
include/ctdb_private.h
server/ctdb_tunables.c
(This used to be ctdb commit b883d19a495a41a22db37f9c2cf6250fee529de0)
Date: Tue Dec 15 15:53:30 2009 +1030
eventscript: hack to avoid overloading valgrind
Now we fork one child per script, when running under valgrind the
load
gets quite high. This is because valgrind does a lot of work after
exit,
and we don't wait for the children to finish; we start the next one
when
the child reports status via the pipe.
This fix is ugly, but simple.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 6ed34d5320c39d8a55f2a36ad4c1ab574e0b0796)
I saw once where the master ctdbd logging structure was talloc freed
which caused issues.
So only free the structure if it is NOT the master structure.
This needs to be looked into in more detail.
(This used to be ctdb commit bcf494b81f4277dc75f05faccf0c446bd15f6e2b)
or else we can crash if we receive log messages from a child but the log structure has been freed()
(This used to be ctdb commit ea9e39369379939abf6a4076fa2014c10c1a9ad0)
Subject: eventscript: fix spinning at 100% cpu when child exits.
ctdbd was spinning reading 0 from a pipe, as soon as the first
eventscript finishes.
This was caused by the intersection between a78b8ea7168e "Run only one
event for each epoll_wait/select call" and 32cfdc3aec34 "eventscript:
ctdb_fork_with_logging()". Unavoidable mid-air collision, since both
worked fine and both were developed simultaneously.
When the script exits, we have two pipes open to it: one for any
stdout/stderr for logging (ctdb_log_handler), and one for the result
(ctdb_event_script_handler). The latter frees everything, including
the log fd and event structure.
We used to get one callback to ctdb_log_handler, which got a harmless
0-length read, then one to ctdb_event_script_handler which cleaned up.
Now we only do one callback per poll, we need the logging function to
clean itself up so we can make process.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 211ea7907e8e96041aa6f7d086551d64d065a8a3)
since we no longer ban nodes when dodgy scripts continue to hang.
We now only mark nodes as unhealthy if monitor events fail or timeout. Never ban.
(This used to be ctdb commit 5c8e56fc7a518e115bceac257867739283cf6a1e)
In other news, did you know ctime() returns a \n-terminated string?
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 1b4e7bb548976b99f122142b040494b6f9911962)
Commit c1ba1392fe "eventscript: get rid of ctdb_control_event_script_finished
altogether" was wrong: there is one case where we want to free the script
without transferring their status to last_status. This happens because we
always kill an running monitor command when we run any other command.
This still isn't quite right (and never was): the callback will be called
with status value 0, which might flip us to HEALTHY if we were unhealthy.
This is conveniently fixed in my next set of patches :)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 0ea0e27d93398df997d3df9d8bf112358af3a4a5)
there is no rational need for a setting where we permanently mark nodes as disabled everytime an eventscript fails
(This used to be ctdb commit 68a8ee99b128a5ec883600735626bdb3bbc9c503)
This reverts commit 8aef46d2aab3efb322dda51eaa202653cefd5222.
This special recovery logic is wrong now with the transaction rewrite.
The treatment of persistent databases will later be rewritten to use the
database sequence number.
Michael
(This used to be ctdb commit c5a0aef668a63f927d6184612b13ce316eb4a0be)
This patch improves the handling of the fetch_lock operation on non-persistent
databases that ctdb clients have to do very frequently.
The normal flow how this goes is the following:
1. Client does a local fetch_lock on the database
2. Client looks if the local node is dmaster.
If yes, everything is fine
If no, continue here
3. Client unlocks the local record
4. Client issues a "get me the record" call to ctdbd
5. ctdbd goes out and fetches the dmaster role
6. ctdbd tells the client to retry
7. Client starts over again
The problem is between step 6 and 7: Before the client has had the chance to
retry (i.e. catch the record with a fetch_locked), another node might have come
asking ctdbd to migrate away the record again. This is a real problem, I've
seen >20 loops of this kind in real workloads.
This patch does the following: Whenever ctdb receives a record as result of
step 5, it puts the key on a "holdback list". As long as a key is on this list,
a request to migrate away the dmaster is put on hold. It is the client's duty
to issue the "CTDB_CONTROL_GOTIT" control when it has successfully done step 2
after having asked ctdb to fetch the record. This will release the key from the
"holdback list" and re-issue all dmaster migration requests.
As a safeguard against malicious clients, once a second (default 1000msecs,
tunable "HoldbackCleanupInterval" in milliseconds) ctdbd goes over the list of
held back keys, deletes them and releases all held back migration requests.
(This used to be ctdb commit 5736e17c139c9a8049e235429aeae0c6c9d0e93d)
Make it return success for make test.
This is temporarily disabled until the rewrite of the
transaction code (in samba and the daemon) using the global
lock feature has been ported to the ctdb client code.
Michael
(This used to be ctdb commit 78ca29352aa39f4ef4e41096b92d55cb2e0d348a)
This is a simplified version of the trans2 commit control:
It just rolls out the marshall buffer to all active nodes.
It is the main ctdbd part of the re-implementation of the
persistent transactions. The client code is changed to
take a global lock to start a transactions and store into
the marshal buffer instead of writing to the local tdb
under a local transaction.
The old transaction implementation is going to be
removed in a later commit.
Michael
(This used to be ctdb commit f66428f9d2013080a414404c1ba6117888352fd6)
Date: Wed, 9 Dec 2009 22:45:12 +0100
Subject: [PATCH] Revert an accidential commit
(This used to be ctdb commit af6656f2844d8fd72204a70358c9d589dbe1bd34)
Writes without transaction are not possible any more on
persistent databases.
Michael
(This used to be ctdb commit 59f46d7261dfdbdef900bf95dd9eb28ad22a46b2)
This is useless now that persistent write operations without
transaction are forbidden.
Michael
(This used to be ctdb commit b022863d44026c19d5aae54aa485b670bea0540e)
This is useless now that persistent writes without transactions are forbidden.
Michael
(This used to be ctdb commit 9ac82311d796e1fab31f8de62b8ccc754445093c)
This is like the 53_ctdb_transaction test, but it additionally
runs a loop with recoveries while the transactions are running.
When called like this, the transaction loops run for 10 minutes:
CTDB_TEST_TIMELIMIT=600 tests/scripts/run_tests tests/simple/54_ctdb_transaction_recovery.sh
The default timelimit is 30 seconds.
Michael
(This used to be ctdb commit 2ff2679e8f3d50ebf735f2c420898a84268bdc95)
This gets just too noisy on a busy system.
And it is purley informational anyways...
Michael
(This used to be ctdb commit 7f64a00c76203fdf6673c3f862a4bfd17fb848d7)
This might be a bit less efficient, but experience in winbind has shown that
event callbacks can trigger changes in the socket state in very hard to
diagnose ways.
(This used to be ctdb commit a78b8ea7168e5fdb2d62379ad3112008b2748576)
syslog.h says:
LOG_NOTICE 5 normal but significant condition
LOG_INFO 6 informational
several vacuuming related logs logged at NOTICE level although I don't see
any real significance, these are just informational messages for me
Signed-off-by: Christian Ambach <christian.ambach@de.ibm.com>
(This used to be ctdb commit 142111983c103e90ccccbe26fd580c4eb28e949f)
add the __location__ macro to the logs to get a better idea
in which loop the problem occured
Signed-off-by: Christian Ambach <christian.ambach@de.ibm.com>
(This used to be ctdb commit dccb549fd6a6e338063699544e52f2a1a6a966b5)
when checking link status for an interface, first
check if this interface is in fact a bond device
(by the precense of a /proc/net/bonding/IFACE file)
and use that file for checking status.
Othervise assume ib* is an infiniband interface which we donnt know how
to check, or otherwise it is an ethernet interface and ethtool should
hopefully work.
(This used to be ctdb commit 8cc6c5de3d7abb0b72eaa6e769e70963b02d84cb)
We also no longer return an error before scripts have been run; a special
zero-length data means we have never run the scripts.
"ctdb scriptstatus all" returns all event script results.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 9b90d671581e390e2892d3a68f3ca98d58bef4df)
We're going to need this so ctdb can query non-monitor status.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 53bc5ca23ca55a3ac63a440051f16716944a2a51)
Ronnie suggested this; seems like a very good idea.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 93153bca68926401dc9ae7fd77ed3f17be923344)
We always have to call it before freeing the state; we should just do
this work in the destructor itself.
Unfortunately, the script state would already be freed by the time
the state destructor is called, so we make the script state a child of
ctdb, and talloc_free() it manually on the one path which doesn't use
the destructor.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit c1ba1392fe52762960e896ace0aca0ee4faa94d5)
Rather than only tranferring to last_status for monitor events, do
it for every event (ctdb->last_status is now an array).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit c73ea56275d4be76f7ed983d7565b20237dbdce3)
We only need ctdb->current_monitor so we can kill it when we want to run
something else; we don't need to use it here as we always know what script
we are running.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 4cf1b7c32bcf7e4b65aec1fa7ee1a4b162cac889)
The only difference between the exposed an internal structure now is
that the name and output fields were pointers. Switch to using
ctdb_scripts_wire/ctdb_script_wire internally as well so marshalling
is a noop.
We now reject scripts which are too long and truncate logging to the
511 characters we have space for (the entire output will be in the
normal ctdbd log).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit fd2f04554e604bc421806be96b987e601473a9b8)
We're going to allow fetching status of all script runs, so this
name is no longer appropriate.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit f5cb41ecf3fa986b8af243e8546eb3b985cd902a)
This neatens the code slightly. We also use the name 'current' in
ctdb_event_script_handler() for uniformity.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit e9661b383e0c50b9e3d114b7434dfe601aff5744)
This brings us closer to the wire format, by using a simple array
and a 'current' iterator.
The downside is that a 'struct ctdb_script' is no longer a talloc
object: the state must be passed to our log fn, and the current
script extracted with &state->scripts->scripts[state->current].
The wackiness of marshalling is simplified, and as a bonus, we can
distinguish between an empty event directory
(state->scripts->num_scripts == 0) and and error (state->scripts ==
NULL).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 76e8bdc11b953398ce8850de57aa51f30cb46bff)
This unifies almost everything: the state->current pointer points to
the struct ctdb_script where we record start, finish, status and
output.
We still only marshall up the monitor events; the rest disappear when
the state structure is freed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit c476c81f3e3d8fc62f2e53d82fce5774044ee9ce)
We rename ctdb_monitor_script_status to ctdb_script, and instead of
allocating them as the scripts are executed, we allocate them up front
and keep a "current" interator.
This slightly simplifies the code, though it means we only marshall up
to the last successfully run script.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit b2a300768536d10bd867a987ad4cf1c5268c44bc)
A new helper functions which sets up an event attached to the child's
stdout/stderr which gets routed to the logging callback after being
placed in the normal logs.
This is a generalization of the previous code which was hardcoded to
call ctdb_log_event_script_output.
The only subtlety is that we hang the child fds off the output buffer;
the destructor for that will flush, which means it has to be destroyed
before the output buffer is.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 32cfdc3aec34272612f43a3588e4cabed9c85b68)
The current logging logic assumes that any stdout/stderr belongs to
the currently running monitor script output. This isn't quite right
anyway, and we'd like to capture stderr output of other script
invocations.
So we move towards multiple struct ctdb_log_state by handing it
directly to ctdb_log_handler to use, rather than having it assume
ctdb->log. We need a ctdb pointer inside the log struct now though.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 497766cf186442de00fb324343150442457be858)
The child no longer uses ctdb_ctrl_event_script_init or
ctdb_ctrl_event_script_finished, and the others are redundant: it
doesn't need to tell us it's starting a script when it only runs one.
We move start and stop calls to the parent, and eliminate the RPC
infrastructure altogether.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 391926a87a7af73840f10bb314c0a2f951a0854c)
We do the same thing in two places: fire off a child from the initial
ctdb_event_script_callback_v() and also from the ctdb_event_script_handler()
when it's done.
Unify this logic into fork_child_for_script().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 814704a3286756d40c2a6c508c1c0b77fa711891)
We rename child_run_scripts() to child_run_script(), because it now
runs a single script rather than walking the list. When it's
finished, we fork the next child from the ctdb_event_script_handler()
callback.
ctdb_control_event_script_init() and ctdb_control_event_script_finished()
are now called directly by the parent process; the child still calls
ctdb_ctrl_event_script_start() and ctdb_ctrl_event_script_stop() before
and after the script.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 0fafdcb8d3532a05846abaa5805b2e2f3cee8f47)
This means all the state about running the scripts is in that structure,
which helps in the next patch.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 020fd21e0905e7f11400f6537988645987f2bb32)
We put a "scripts" member in ctdb_event_script_state, rather than using
a special struct for monitor events. This will fit better as we further
unify the different events, and holds the reports from the child process
running each monitor script.
Rather than making the monitor state a child of current_monitor_status_ctx,
we just point current_monitor directly at it. This means we need to reset
that pointer in the destructor for ctdb_event_script_state.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 9a2b4f6b17e54685f878d75bad27aa5090b4571f)
We have monitor_event_script_ctx and other_event_script_ctx, and
current_monitor_status_ctx in struct ctdb_context. This seems more
complex than it needs to be.
We use a single "event_script_ctx" as parent for all event script
state structures. Then we explicitly reparent monitor events under
current_monitor_status_ctx: this is freed every script invocation to
kill off any running scripts anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 0d925e6f2767691fa561f15bbb857a2aec531143)
Simple refactoring in preparation for switching to one-child-per-script.
We also call the functions run by the child process "child_".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit bfee777faff75e9bed4aedc1558957483616a6d3)
This is the start of a move towards finer-grained reporting, with one
child per script. Simple code motion to do sanity check and get the
list of scripts before fork().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 816b9177f51ae5b21b92ff4a404f548fe9723c96)
If we've timed out, but we've not timed out more than
ctdb->tunable.script_ban_count, we pretend we haven't.
There's a logic bug in the way this is done: if we were unhealthy before,
this would set us to "healthy" again (status == 0). I don't think this
would happen in real life, but it's a little surprising.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit e6488c0e05bab5c4c2c0a6370930b0b27e5ed56e)
Currently the timeout handler in eventscript.c does the banning if a
timeout happens. However, because monitor events are different, it has
to special case them.
As we call the callback anyway in this case, we should make that handle
-ETIME as it sees fit: for everyone but the monitor event, we simply ban
ourselves. The more complicated monitor event banning logic is now in
ctdb_monitor.c where it belongs.
Note: I wrapped the other bans in "if (status == -ETIME)", though they
should probably ban themselves on any error. This change should be a
noop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 9ecee127e19a9e7cae114a66f3514ee7a75276c5)
eventscript.c uses this now, but our next patch makes others use it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit a305cb7743c24386e464f6b2efab7e2108bb1e7e)
If we time out just as the child exits, we currently will report an
uninitialized cb_status field. Set it to -ETIME as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 024386931bda9757079f206238ae09bae4de6ea2)
This completes our "problem with script" reporting; we never set cb_status
to -1 on error. Real errnos are used where the failure is a system call
(eg. read, setpgid), otherwise -EIO is used if we couldn't communicate with
the parent.
The latter case is a bit useless, since the parent probably won't see
the error anyway, but it's neater.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 1269458547795c90d544371332ba1de68df29548)
If we break, we avoid cut & paste code inside the loop. Need to initialize
ret to 0 for the "no scripts" case.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit ec36ced9446da7e3bf866466d265ee8e18f606c1)
Rather than ignoring deleted event scripts (or pretending that they were "OK"),
and discarding other stat errors, we save the errno and turn it into a negative
status.
This gives us a bit more information if we can't execute a script (eg.
too many symlinks or other weird errors).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 5d894e1ae5228df6bbe4fc305ccba19803fa3798)
This unifies code paths and simplifies things: we just hand -ENOEXEC to
ctdb_ctrl_event_script_stop().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit eadf5e44ef97d7703a7d3bce0e7ea0f21cb11f14)
We currently assume 127 == script removed. The script can also return 127;
best to re-check the execution status in this case (and for 126, which will
happen if the script is non-executable).
If the script is no longer executable/not present, we ignore it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 0a53d6b5ac81daf0efa32f35e7758ede2a5bdb63)
This is used later in the "script vanished" check.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 8ddb97040842375daf378cbb5816d0c2b031fa65)
As we start to use errno more, it's a huge pain if talloc_free() can blatt
it (esp. destructors).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 76a0ca77feba14e1e1162c195ffbdf516e62aa4d)
This starts the move toward more expressive encoding of return values:
positive values mean the script ran, negative means we had a problem with
the script (and the value is the errno).
This does timeout, but changes the ctdb tool to recognize it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 0eb1d0aa14e68b598d9e281c8a02b8f94a042fd9)
This simplifies the code a little: last_status is now read to go
(it's only used by the scriptstatus command at the moment).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 6be931266a4e41fd0253f760936ad9707dd97c47)
This needs to be done after the control-dispatcher:
In the TRANS2_COMMIT control, the client->db_id needs
to be set before bailing out, since otherwise the
next TRANS2_COMMIT_RETRY will fail...
Michael
(This used to be ctdb commit 59faf3f923a5989b5ee94ef02a12827412775bae)
To cope with timeouts when recoveries and transactions collide.
Maybe 100 is too high.
Michael
(This used to be ctdb commit c23d804165e84bdf95ba960c953c736d361011d7)
So that it is correctly handled by recoveries.
Also explicitly set the dmaster field to the current node's pnn.
Michael
(This used to be ctdb commit 03a5bb727b9db1ba952632f08ceb5355f0df842d)
The decision mechanism which records of a persistent db
are to be pulled into the recdb during recovery is now
as follows:
* Usually a record with the higher rsn than that already
stored is taken. (Just as for normal tdbs.)
* If a transaction is running on some node, then those
nodes copies of all records are taken and are not
overwritten later by other nodes' copies.
In order to keep track of whether a record's copy was obtained
from a node with a transaction running, the recovery mechanism
misuses the ctdb tdb header field 'lacount' in the recdb.
It is cleared later when pushing out the recdb database to the
other nodes.
This way, an incomplete transaction is not spoiled when
a recovery interrupts and the replay should usually succeed
(possibly after a few retries).
Michael
(This used to be ctdb commit 8aef46d2aab3efb322dda51eaa202653cefd5222)
It is important to keep track of the dmaster (i.e. the node that last committed
a transaction containing changes to this node).
Michael
(This used to be ctdb commit fe68972eb9cf3aa1f16ba1aacf57ade5d66e647c)
and further down to pull_remote_database(), pull_one_remote_database(),
and push_recdb_database().
This is in preparation of special handling of persistent databases
during recoveries.
Michael
(This used to be ctdb commit 90abc4ac7c16e854cf6e8f96b60a77bc92e35e07)
The githash can be specified through the environment variable "GITHASH"
that can contain a commit hash or a tag name, e.g.
The call syntax is now
[GITHASH=xyz] [USE_GITHASH=yes/no] [DEBIAN_MODE=yes/no] maketarball.sh
Michael
(This used to be ctdb commit 41aa9bdfa2934f564bdc14374362437dfad0045f)
It is unlikely we will need something this verbose for normal troubleshooting.
This allows us to keep a significantly longer time interval of log messages
in the 500k slots available in the ringbuffer.
(This used to be ctdb commit cc99c05c0c6484ad574039a454e6133852cb41fa)
in memory instead of dynamically allocated ones so that we reduce the pressure
on malloc/free.
(This used to be ctdb commit c5cbb95512f034abeec515579983bf7ac55eadd9)
Commit 50c2caed57c0 removed a gratuitous talloc_steal from the code in
ctdb_control_event_script_finished(), but not ctdb_event_script_timeout().
Easiest to call ctdb_control_event_script_finished() at the bottom of the
timeout routine.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 17fa252d0d6981fbae8083a818f26d5ce9c5102e)
This controls is only used by samba when samba wants to check if a subrecord held by a <node-id>:<smbd-pid> is still valid or if it can be reclaimed.
If the node is banned or stopped, we kill the smbd process and return that the process does not exist to the caller. This allows us to recover subrecords from stopped/banned nodes where smbd is hung waiting for the databases to thaw.
bz58185
(This used to be ctdb commit 157807af72ed4f7314afbc9c19756f9787b92c15)
Add the mapping to the list everytime we accept() a new client connection
and set it up to remove in the destructor when the client structure is freed.
(This used to be ctdb commit f75d379377f5d4abbff2576ddc5d58d91dc53bf4)
and store this in the client structure.
There is no need to rely on the hack that samba sends some special message
handle registrations that encodes the pid in the srvid any more.
This might not work on AIX since I recall some issues to get the pid in
this way on that platform.
(This used to be ctdb commit b4a7efa7e53e060a91dea0e8e57b116e2aeacebf)
Wolfgang saw a talloc complaint about using freed memory in ctdb_tcp_read_cb.
His fix was to remove the talloc_free() in that function, which causes
loops when a socket is closed (as it does not get removed from the event
system), eg:
netcat 192.168.1.2 4379 < /dev/null
The real bug is that when we have more than one pending packet in the
queue, we loop calling the callback without any safeguards should that
callback free the queue (as it tends to do on invalid packets). This
can be reproduced by sending more than one bogus packet at once:
# Length word at start: 4 == empty packet (assumed little endian)
/usr/bin/printf \\4\\0\\0\\0\\4\\0\\0\\0 > /tmp/pkt
netcat 192.168.1.2 4379 < /tmp/pkt
Using a destructor we can check if the callback frees us, and exit
immediately. Elsewhere, we return after the callback anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 4d0523dd94fb07e860b3e8118691f93d1ef8d0fa)
It is triggered by setting DEBIAN_MODE=yes in the environment.
This creates a tarball suitable for use in debian packages.
The differences from the standard tarball are these:
* The tar ball file is called ctdb_VERSION.orig.tar.gz
* The base directory in the tar ball is ctdb-VERSION.orig/
Michael
(This used to be ctdb commit 83e7c161efa93cd7acdfc803142b4fb3bfde7538)
The functions file no longer causes a side-effect by doing a shift.
It also doesn't set a convenience variable for $1.
All eventscripts now explicitly use "$1" in their case statement, as
does the initscript. The absence of a shift means that the
takeip/releaseip events now explicitly reference $2-$4 rather than
$1-$3.
New function ctdb_standard_event_handler handles the status and
setstatus events, and exits for either of those events. It is called
via a default case in each eventscript, replacing an explicit status
case where applicable.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 3d55408cbbb3bb71670b80f3dad5639ea0be5b5b)
and until we have gone through a full re-recovery timeout without triggering
any pending recoveries before we start up the services and start monitoring
the node.
(This used to be ctdb commit 821333afb458358f90446062b0242790695e5060)
Apart from lots of cleanup work, this also fixes a bug where the share
checks didn't used to cope with directory names containing spaces.
The previous commit also loaded the config incorrectly.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 3c93336ab92c2e4829ff4dc360045bfa6df21d50)
Now we're doing checking, we might as well make sure the commands from
"ctdb eventscripts" are valid.
This gets rid of the "UNKNOWN" event type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 1d24a3869fe89fc9a109fd9e9b69df5fc665a5f6)
if the node is BANNED.
Dont even let them attach at all
to the database
Revert "temporarily try allowing clients to attach to databases even if
the node is banned/stopped or inactive in any other way."
This reverts commit 227fe99f105bdc3a4f1000f238cbe3adeb3f22f0.
(This used to be ctdb commit 10a3680fb3917ecafc824e73872eace321026172)
Now we're doing checking, we might as well make sure the commands from
"ctdb eventscripts" are valid.
This gets rid of the "UNKNOWN" event type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 66b22980b14601f29fe8cc64bd8f29883c7ca1c0)
This is not as good as a compile-time check, but at least we count the
number of arguments are correct.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 83b7b233cb4707e826f6ba260bd630c8bc8f1e76)
This is not as good as a compile-time check, but at least we count the
number of arguments are correct.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit a6d353519932eee48f9241ad8887b692882906c9)
Finally, we remove the call name (eg. "monitor" or "start") from the
options field of the struct: it now contains only extra options.
This is clearer, and mainly involves adding some %s to debug statements.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 33fb0e7ba047ca73969b59bccf70a04a17c25a0a)
Finally, we remove the call name (eg. "monitor" or "start") from the
options field of the struct: it now contains only extra options.
This is clearer, and mainly involves adding some %s to debug statements.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit b0648c7f08eba87ec3c9714e2525c9b621bfb4ef)
This means we can get rid of more strcmp; they can simply use the
state->call value instead.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 6c79fa33e26cc4f0873577f8e122b1495b4c427e)
This means we can get rid of more strcmp; they can simply use the
state->call value instead.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 834c93b3e1b8f4151b8a2cd82c2dd8bacc17f66c)
Rather than doing strcmp everywhere, pass an explicit enum around. This
also subtly documents what options are available. The "options" arg
is now used for extra arguments only.
Unfortunately, gcc complains on empty format strings, so we make
ctdb_event_script() take no varargs, and add ctdb_event_script_args(). We
leave ctdb_event_script_callback() taking varargs, which means callers
have to do "%s", "".
For the moment, we have CTDB_EVENT_UNKNOWN for handling forced scripts
from the ctdb tool.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 8001488be4f2beb25e943fe01b2afc2e8779930d)
Rather than doing strcmp everywhere, pass an explicit enum around. This
also subtly documents what options are available. The "options" arg
is now used for extra arguments only.
Unfortunately, gcc complains on empty format strings, so we make
ctdb_event_script() take no varargs, and add ctdb_event_script_args(). We
leave ctdb_event_script_callback() taking varargs, which means callers
have to do "%s", "".
For the moment, we have CTDB_EVENT_UNKNOWN for handling forced scripts
from the ctdb tool.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 470822b329f9d3ca9bef518b56e9ce28d5fedda2)
Everyone uses the same timeout value, so just remove it from the API.
If we ever need variable timeouts, that might as well be central too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 533c3e053293941d2a9484b495e78d45f478bb08)
Everyone uses the same timeout value, so just remove it from the API.
If we ever need variable timeouts, that might as well be central too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit fe8027309c1f7b987cd368fa98f9b28741baa786)
ctdb_event_script_v doesn't take varargs. ctdb_run_event_script is
a better name, and fix comment.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 466beafadb37011fe273de8810ab0012e92a1fd8)
1) ctdb_event_script_v doesn't take varargs. ctdb_run_event_script is
a better name, and fix comment.
2) Fix indentation on allowed_scripts.
3) Comment on run_eventscripts_callback is wrong; it's the callback
for any ctdb forced event.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit e7d57d7ae678b24dab3364a348838c6a3398942c)
In 15bc66ae801b0c69, Ronnie fixed a double-free race. The problem was that
ctdb_run_eventscripts() hands a context to ctdb_event_script_callback() to
hang its data off, which gets freed in the callback. This particularly
hurt in ctdb_event_script_timeout.
There's nothing wrong with this, but obviously we should make the callback
call last of all. At the time, ctdb_event_script_timeout() carefully
extracted everything from the struct ctdb_event_script_state before
calling ->callback.
This was cleaned up in 64da4402c6ad485f (Ronnie again), and now state
was referred to after the callback again. But the same change introduced
a direct use-after-free bug which caused an occasional oops.
So in our last episode (eda052101728cf92) Volker fixed this, and Michael
committed it.
But we still have the double free bug which 15bc66ae801b0c69 was supposed
to fix! Let's try to fix this in a more permanent way, but always doing
the callback from the destructor. This means we need to hold the status,
and don't send the KILL signal if ->child is set to 0.
Finally, add a comment about freeing ourselves in run_eventscripts_callback
and the structure definition.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit b90bdb07c1f6913ddbf11bde9684bdc8af61c549)
In 15bc66ae801b0c69, Ronnie fixed a double-free race. The problem was that
ctdb_run_eventscripts() hands a context to ctdb_event_script_callback() to
hang its data off, which gets freed in the callback. This particularly
hurt in ctdb_event_script_timeout.
There's nothing wrong with this, but obviously we should make the callback
call last of all. At the time, ctdb_event_script_timeout() carefully
extracted everything from the struct ctdb_event_script_state before
calling ->callback.
This was cleaned up in 64da4402c6ad485f (Ronnie again), and now state
was referred to after the callback again. But the same change introduced
a direct use-after-free bug which caused an occasional oops.
So in our last episode (eda052101728cf92) Volker fixed this, and Michael
committed it.
But we still have the double free bug which 15bc66ae801b0c69 was supposed
to fix! Let's try to fix this in a more permanent way, but always doing
the callback from the destructor. This means we need to hold the status,
and don't send the KILL signal if ->child is set to 0.
Finally, add a comment about freeing ourselves in run_eventscripts_callback
and the structure definition.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 20b15de068d042b292725945927ceda1b01d07c0)
Write the whole int through the pipe, rather than quietly cutting it
off. Also, use -2 as the result if the read fails; -1 comes from many
paths if the child fails before running the script.
Add a comment about why we don't need to check the write.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 6804f880436645b52c09a78fa300377fa8058d0e)
Write the whole int through the pipe, rather than quietly cutting it
off. Also, use -2 as the result if the read fails; -1 comes from many
paths if the child fails before running the script.
Add a comment about why we don't need to check the write.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit c715746c2f40eb9b21dbf011d16f1f1b0b53fdf9)
Apart from lots of cleanup work, this also fixes a bug where the share
checks didn't used to cope with directory names containing spaces.
The previous commit also loaded the config incorrectly.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 35a60a63a9b5c7d98dde514ae552239506b691c9)
The test depended on the exit code of "ctdb gettickles", which always
succeeds. This change wraps the command in a function that checks
whether the tickle we're interested in is registered.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit c4b05a731e1bee8f5b46529773a4f5389b2b6064)
command.
Use the existing context used for non-monitor events
Multiple concurrent uses of "ctdb eventscript ..." could otherwise lead to a SEGV
(This used to be ctdb commit 80a8d728e9680040e00d24361dfc9367dd372a56)
This is the first stage of an experimental change to eventscripts.
Ronnie and I did a few hours of factorisation of 40.vsftpd and applied
many of the changes to 41.httpd. Other eventscripts were also
modified.
At this stage this is completely untested.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 364e70b763f0ccd7714d15723ad3ea4d7e2968a1)
This allows running the actual monitoring asynchronously from ctdbd
and only using "status" to pick up the actual results.
(This used to be ctdb commit 1908bac812650ca25151051f5d86815e0b8ed319)
The NFS test sleeps for MonitorInterval to give CTDB time to record an
NFS tickle. However, this isn't always long enough. This changes the
test to wait until a monitor event has actually occurred.
The CIFS test assumes that Samba is able to register a tickle with
CTDB before it notices that netstat has registered the tickle and can
use onnode to ask CTDB about it. That is an incorrect assumption -
sometimes we can get to the point of asking CTDB about the tickle
before Samba and CTDB have processed it. This adds a timeout loop
that makes the CIFS test wait until the tickle has been registered or
fail after 10 seconds.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 20a9d35933d89dc7eb710075f360686a49d78609)
use killtcp and kill both directions of the nfs connections.
we used to kill only one direction since the other direction was unkillble
but recent kernels allow us to kill both
(This used to be ctdb commit 8001ae580bcc28d45f6026b529d7ffc247cbba34)
dont allow UNHEALTHY nodes to become natgw master, unless all nodes
are unhealthy
(This used to be ctdb commit e8e7129ff1371065fbd75e1aea844d6d04a96fa9)
This just sleeps for twice the value of EventScriptTimeout
in the monitor action. It is not run by default, but
can be activated by setting CTDB_RUN_TIMEOUT_MONITOR
in /etc/sysconfig/ctdb .
Michael
(This used to be ctdb commit 1a3ecdee85b82bb3234a92ae6bcdeb92238eb7ee)
It's much nicer for post-mortem debugging to have a body to examine.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 058e21d96c3c02759833fd5ddfe7b43e6a5f5740)
this to handle the case where all links do have a physical layer, but where all slaves have been disabled using ifdown
(This used to be ctdb commit bf50709630df000583f2b0ef0edc177c01d60eaf)
setting USE_GITHASH=no in the environment makes
makerpms.sh omit the git hash
Michael
(This used to be ctdb commit 209ff041596e39688186c99995863ed3e816b8e4)
The vacuum process treats persistent databases the same as
non-persistent and thus ignores the extra state for transactions.
This way, it breaks the api-level transactions.
Michael
(This used to be ctdb commit f98fefbc566eefbfcc660646af6e25256ab82b13)
This reverts commit 4e9a3a5dc232bac12ab387ea0cf4f1b279bed5c1.
Transaction commit should not be allowed to fail.
This is a real error.
Michael
(This used to be ctdb commit 825c506da76d7afd0714b75b8c8727874183a618)
The gap that remained is between checking whether a transaction commit
is in progress and taking the lock. Now we first take the lock and then
check whether a transaction commit is in progress. If so, we release the
lock, wait for one second and retry.
Michael
(This used to be ctdb commit b95524c08bf12914120cb6c818ecc1c99738fe37)
We should really leave it up to the administrator to decide
whether ctdb should be started automatically at boot-time.
Michael
(This used to be ctdb commit c1d8496f9fd5e8046f3d990264258dfb054f3b32)
use a udp socket on the ctdbd port to send messages to teh syslog child process for loggign.
we need this when syslog becomes "slow", like very slow, and on boxes where syslog is limited to 100 lines per second and starts to block after that
(This used to be ctdb commit 1446f4c247310e2ff2d522055bd8927d1a78d017)
* add __location__
* wrap overly long line
* print unsigned ints as unsigned (reqid, opcode, destnode)
Michael
(This used to be ctdb commit 6b47ea111867c845974aa2687a658ebca2854816)
Otherwise a node can lock itself out, e.g. when a commit control times out...
Michael
(This used to be ctdb commit cb432e30351d5e5a41e98da3c7b1c2a4d400a3a2)
In ctdb_transaction_commit(), when the trans2_commit control fails, there
is a race condition in the 1 second sleep between the local transaction_cancel
and the call to ctdb_replay_transaction(): The database is not locked, and
neither is the transaction_lock record. So another client can start and possibly
complete a new transaction in this gap, but only on the same node: The locking
of the transaction_lock record on a different node which involves migration of
the record to the other node has been disabled by introduction of the
transaction_active flag on the db which closes precisely this gap from the start
of the commit until the call to TRANS2_FINISH or TRANS2_ERROR.
But this mechanism does not cover the case where a process on the same node
tries to start a transaction: There is no obstacle to locking the transaction_lock
record because the record does not need to be migrated.
This commit closes this race condition in ctdb_transaction_fetch_start()
by using the new ctdb_ctrl_transaction_active() call to ask the local
ctdb daemon whether it has a transaction running on the database.
If so, the check is repeated until the running transaction is done.
This does introduce an additional call to the local ctdbd when starting
transactions, but it does close the (hopefully) last race condition.
Michael
(This used to be ctdb commit 02ee9dfd3c6b09f5c5172a7e38738c20b7f0aecd)
This aske the daemon wheter a transaction is currently active on a
given DB on that node. More precisely this asks for the transaction_active
flag in the ctdb_db_context that is set in the CTDB_TRANS2_COMMIT
control and cleared in the CTDB_TRANS2_ERROR or CTDB_TRANS2_FINISHED controls.
This will be useful for fixing race conditions in the transaction code.
Michael
(This used to be ctdb commit 8d430ae6968dfe566614379436fc3c56003fcd88)
Commit 25e82a8a667a54c6921ef076c63fdd738dd75d19 changed wait_until()
to protect the command it runs from "set -e" by running it in a
subshell. This breaks uses where the command is expected to set
global variables. For example, wait_until_get_src_socket lost the
value of $out from its call to get_src_socket().
The fix is to not be lazy and use a sub-shell!
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 39642e745254d93d74dde907787503854fe6ca4a)
When a single script is finished, also log the name of the script, the duration it took and the return status.
In the loop where we signal back to the main daemon that the script finished, do this once every 100ms instead of once every 1 second
(This used to be ctdb commit 6a1f7a7b1b3a0b8f89998db8fdad83bbb4e9b5a5)
add a global variable holding the pid of the main daemon.
change the tracking of time() in the event loop to only check/warn when called from the main daemon
(This used to be ctdb commit a10fc51f4c30e85ada6d4b7347b0f9a8ebc76637)
All event scripts use only the relative path, so we should
here.
Also PATH includes /sbin and /usr/sbin...
metze
(This used to be ctdb commit 20678e1506db1f96b58c326ee91339e797c07c22)
The problem was this:
When the monitor event fails, the node->flags get updated,
and an update (containing the old and new flags) is sent to
the recovery master.
If the recovery master sends the update to itself (the same process),
it was compairing the node->flags variable with the received new flags.
This check always found both flag values to be equal
and never sets the rec->need_takeover_run variable to true.
There were two problem, first the push_flags_handler() function
didn't pass the received old flags.
And the ctdb_control_modflags() function ignored the received old flags.
metze
(This used to be ctdb commit 8ec633b64a05a2d903c2b9639909f15f6375548f)
log an error if the clock jumps backwards
also log an error if the clock jumps >5 seconds forward (we assume here we will get at least one event every 5 seconds)
(This used to be ctdb commit 11193e1e192bee6f579bdf1303153571a82711d7)
make ctdb_queue_length() cheaper by using a counter variable instead of counting the number of packets each time.
(This used to be ctdb commit 331c6e3afd96d8b5e191153a631efdbdabb6ea33)
The way to use this is from a client to :
1, first create a message handle and bind it to a SRVID
A special prefix for the srvid space has been set aside for samba :
Only samba is allowed to use srvid's with the top 32 bits set like this.
The lower 32 bits are for samba to use internally.
2, register a "notification" using the new control :
CTDB_CONTROL_REGISTER_NOTIFY = 114,
This control takes as indata a structure like this :
struct ctdb_client_notify_register {
uint64_t srvid;
uint32_t len;
uint8_t notify_data[1];
};
srvid is the srvid used in the space set aside above.
len and notify_data is an arbitrary blob.
When notifications are later sent out to all clients, this is the payload of that notification message.
If a client has registered with control 114 and then disconnects from ctdbd, ctdbd will broadcast a message to that srvid to all nodes/listeners in the cluster.
A client can resister itself with as many different srvid's it want, but this is handled through a linked list from the client structure so it mainly designed for "few notifications per client".
3, a client that no longer wants to have a notification set up can deregister using control
CTDB_CONTROL_DEREGISTER_NOTIFY = 115,
which takes this as arguments :
struct ctdb_client_notify_deregister {
uint64_t srvid;
};
When a client deregisters, there will no longer be sent a message to all other clients when this client disconnects from ctdbd.
(This used to be ctdb commit f1b6ee4a55cdca60f93d992f0431d91bf301af2c)
The waiters reference the locakwait handle in order to remove itself from the li
nked list which caused a SEGV.
We dont actually need to remove ourselves from this list here since
if the parent freeze_handle holding the list is freed, then all waiters are rele
ased as well, and the only place we actually need to relink the waiter is in ctd
b_freeze_lock_handler, where we want to respond back to the clients and release
the waiters but we still want to keep the freeze_handle hanging around.
(This used to be ctdb commit e01ab46bafad09a5e320d420734db129d35863bc)
With the new vacuuming code, dont treat an invalid dmaster as fatal. Let it update to the new value insetad.
(This used to be ctdb commit 5b70fa8cfd5916d3c212823ad5cc1b251ae175ed)
The timeout for waiting for state changes isn't very predictable. It
is "about" MonitorInterval seconds... but can be longer given the
duration of eventscript runs and other things. So, we change the
timeout to MonitorInterval + EventScriptTimeout, hoping it never takes
that long.
Move the eventscript installation/removal from the old fake-tests into
a function in the functions file. Implement supporting functions to
create/remove/check-for various files that it handles. Also add a
function that uses all of this that waits for the next monitor event
(but only if all other monitor events pass).
The final check in the skip share check tests uses the above and waits
for a monitor event, and then checks that the node is still healthy.
Also enhance the wait_until function to handle a command starting with
'!' (as a separate word) to make it easy to wait for a file not to
exist.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 25e82a8a667a54c6921ef076c63fdd738dd75d19)
This can race with teh eventascript handling that does a :
list all scripts, sort them, then execute them
so trap status code 127 which means the script could not be executed (or /bin/sh does not exist) and treat it as not to cause the node to become unhealthy
(This used to be ctdb commit befabc917edb036ca81f5216f65a6d62b26ee83e)
Add a new tunable to control the maximum queue size we allow to a blocked client before we start discarding REQ_MESSAGES instead of queueing them for delivery.
This avoids having queued up very very large number of MESSAGES that samba semds
between eachother to nodes that are blocked/banned/stopped for extended periods
.
(This used to be ctdb commit f76d6fed8f9630450263b9fa4b5fdf3493fb1e11)
we also skip this check if we are already in the process of performing an ip reallocation and not only when we are performing a full recovery.
(This used to be ctdb commit 1a09b02767f3928d3c5db0e0afc59bb938e4a445)
Monitor events sometimes happen a little bit more than MonitorInterval
seconds apart. This changes some timeouts to MonitorInterval + 1
seconds.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 6ef4364b3349145b2fec23e0431cd6df6dcadd41)
This function has been broken since it was updated to work with the
"stopped" state (probably commit
67c5bfb5f02c9d45a32d976021ede4fb2174dfe9). Although ${var#:*:0}
removes the shortest matching prefix of $var, '*' can match substrings
that include ':' if '0' isn't where you expect. So we were making
unexpected matches and incorrectly returning true for some cases.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 11137bc2d492a62a26ec9f9f62ff362e81643f66)
This facilitates tracing of tests.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 1f906bd3476e7cebf217e35b5477d6a7bb615a0c)
make sure to always create the vactun database and get rid of some annoying log messages
(This used to be ctdb commit 54f9c314a0354f1039208fe6ac7dc159b6db8750)
Add a tuneable so that when scripts starts to hang/timeout, we can make the node unhealthy instead of banned
(This used to be ctdb commit 2e9fc6f0609833c6d8146196011ef780669d615d)
The databases can become frozen a while before we do the actual recovery
since we have the re-recovery timeout.
There is no point in doing much monitoring if we are waiting for a recovery,
or if we are banned.
This will eliminate some annoying log entries where certain tests will fail if the databases are locked.
(This used to be ctdb commit ff824676fab94168707aada7423ae766bc0f711c)
Leave the node as UNHEALTHY this stops clients from accessing the node until
the reclock file can be accessed again
(This used to be ctdb commit f5e9f3007c10a937158bc8cdfabf33c984cf9c50)
so we can spot if there are leaks.
plug two leaks for filedescriptors related to when sending ARP fail
and one leak when we can not parse the local address during tcp connection establish
(This used to be ctdb commit ddd089810a14efe4be6e1ff3eccaa604e4913c9e)
print a full "pstree -p" to the log.
Example :
|-ctdbd(29826)-+-ctdbd(29862)
| `-ctdbd(31897)-+-00.ctdb(31898)---sleep(31908)
change the default timeout to 60 seconds for eventscripts
(This used to be ctdb commit a3406c10d70f89d332eab25d481083142dff987d)
The -f option allows an alternate nodes file to be specified,
overriding the CTDB_NODES_FILE environment variable.
The -n option allows hostnames to be used instead of node numbers.
Using a range of hostnames is invalid, so hostnames can't contain
hyphens ('-') - sorry! You can use this option without a nodes file
by specifying "-f /dev/null".
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 46474e5f21fd97dd765c616647ff46055a9970e7)
master to perform an explicit ip reallocation.
This is more reliable and faster than having the recovery dameon track these
changes, and since we now have an explicit method to ask the recovery daemon
to perform an explicit ip reallocation, we should use this.
(This used to be ctdb commit 3807681e74f4bfe92befdae6ed616ff5f1a99880)
this allows us to configure and enable nfs at runtime without having to restart ctdbd
(This used to be ctdb commit f6e39d35713475defaa08a623e194f3f2f8f7d53)
This patch ensures that we lock all non-notify related databases first and
then the notify databases to avoiud a deadlock where samba needs to lock records on two databases at once (and notify being the second database).
Newer versions of samba would instead use the set-db-prio control to set this explicitely on a database per database basis instead of relying on hardcoded database names. This patch will be reverted in the future when all updated versions of samba has been pushed out.
(This used to be ctdb commit 70e7781df1f118a0e2632a9c634f3fd388fa6c8c)
transactions we start across all tdb databased during the recovery.
this allows us to properly clean up and delete these tdb transactions on a
recovery failure.
(This used to be ctdb commit b2ce8b900a7d00944c84e0574fea5b371064a06d)
There are 2 problems with this code:
* The loop in ctdb_check_directories_probe() breaks on filenames
containing whitespace.
The fix to protect them is to pass "$@" to this function and have it
operate on "$@".
Note that there's still a problem with whitespace in filenames in
the 50.samba eventscript. To fix this ctdb_check_directories_probe
should read the filenames from stdin. Another time...
* The check for '%' in filenames in ctdb_check_directories_probe()
ends up involving several forks. On a modern machine this can cost
a couple of minutes when checking a large number of directories.
The fix is to use a case statement.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit eb1fecaef9aa5cb85dff7d4f7af8a9878deabed8)
Each recovery that involves IP reassignments results in a restart of
vsftpd in the "recovered" event. Currently, we can have several
recoveries in quick succession and the "monitor" event following each
can fail because vsftpd isn't ready yet. This results in cumulative
failures, so the node is marked unhealthy, even though vsftpd has
never had a proper opportunity to become ready.
This resets the fail count after each recovery.
While we're here, also move the delete of the restart flag file into
the body of the conditional.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 318abeb4b913a8d846e7eaf4cf5c2a67b61ce974)
database priorities will be used to control in which order databases are locked during recovery in.
(This used to be ctdb commit 67741c0ee01916d94cace8e9462ef02507e06078)
this prevents a situation where the remove node may cause spurious ip reallocations.
(This used to be ctdb commit dd122351efaeef5475cdec111eb900110d83ec35)
This is useful when we are moving addresses using moveip in the cluster since otherwise if we collide with the recovery daemons own check we could cause a recovery
(This used to be ctdb commit 9c63858c0b22c81eaccb9865a414af0bbb2833d4)
Time can actually go backwards during this test if ntpd happens to
adjust it little bit. So we should cope...
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 23ae9e9863ea90c6fb3f105403fd098041fa73f4)
test -z really needs its argument to be quoted. Simplified a status
test.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit fe26da7780545b1ecc0a7da5bc1cf8beaeea94cc)
Change the monitor event in 40.vsftpd so it only fails if there are 2
successive failures connecting to port 21. This reduces the
likelihood of unhealthy nodes due to vsftpd being restarted for
reconfiguration due to node failover or system reconfiguration.
New eventscript functions ctdb_counter_init, ctdb_counter_incr,
ctdb_counter_limit. These are used to count arbitrary things in
eventscripts, depending on the eventscript name and a tag that is
passed, and determine if a specified limit has been hit. They're good
for counting failures!
These functions are used in 40.vsftpd and also in 01.reclock - the
latter used to do the counting without these functions.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit cfe63636a163730ae9ad3554b78519b3c07d8896)
Remove the explicit vacuum/repack commands from the 00.ctdb eventscript
and implement this in the ctdb daemon.
Combine vacuuming and repacking into one
cheap read traverse to enumerate all candidate records
and one write traverse that both repacks the database and also deletes the record locally where we are lmaster and where the records have already been deleted remotely.
this code also adds initial autotuning heuristics for the vacuum intervals and how many records to delete in each iteration.
minor stylish changes made by ronnie s
(This used to be ctdb commit 95a3ee551241aa164967991fe5efe078e1714bde)
if the reclock file has been set, then this script will test that the
reclock file can actually be accessed.
if the file does not exist, or if the attempts to stat the file hangs,
the node will be marked unhealthy after the third failed monitoring event
and after the tenth failure, ctdb itself will shutdown.
(This used to be ctdb commit 2cb04747887674def299e574fccb827c1c3194e7)
change this to provide absolution to all nodes once they have participated in a recovery session.
(This used to be ctdb commit f66d17fb2e81a35d5adb3754e1cc902f76b4590a)
This reverts commit affa6f47432507e84b7e76b88a2c27fff8e6e2e4.
Transaction commit should not be allowed to fail.
This is a fatal error.
Michael
(This used to be ctdb commit 4364419a486c1995bea56dab603cc4960e7c8e7a)
This reverts commit 7a6134e684c9ac4763bf198ef1410867b6082c94.
Transaction commit should not be allowed to fail.
This is a fatal error.
Michael
(This used to be ctdb commit 74e416108df6934f45ca646d709785dd76ab3c35)
There are two races in concurrent transactions on a single node.
One in starting a transaction, and one with committing (replaying).
This commit closes the first race by storing the pid in the
transaction-lock record and comparing the own pid against it
as a measure to prevent starting a second transaction when
a second node has come inbetween and changed the pid in the lock
record.
Michael
(This used to be ctdb commit 84e5a55a900b01903b80e23045edfc726d8d77a1)
This happens normally when someone explicitely triggers a recovery using "ctdb recover"
(This used to be ctdb commit 3085170be8460e59996a3eee4e29fec9ddbcf0f8)
