1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-27 03:21:53 +03:00
Commit Graph

5423 Commits

Author SHA1 Message Date
Amitay Isaacs
959b9ea0ef ctdb-recoverd: Process all the records for vacuum fetch in a loop
Processing one migration request at a time is very slow and processing
a batch of records can take longer than VacuumInterval.  This causes
subsequent vacuum fetch requests to be dropped.  The dropped records
can accumulate quickly and will cause the vacuum database traverse to
be quite expensive.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Dec  5 17:06:58 CET 2014 on sn-devel-104
2014-12-05 17:06:58 +01:00
Amitay Isaacs
257311e337 ctdb-vacuum: Do not delete VACUUM MIGRATED records immediately
Such records should be processed by the local vacuuming daemon to ensure
that all the remote copies have been deleted first.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-12-05 14:43:07 +01:00
Amitay Isaacs
dbb1958284 ctdb-vacuum: Use non-blocking lock when traversing delete tree
This avoids vacuuming getting in the way of ctdb daemon to process
record requests.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-12-05 14:43:07 +01:00
Amitay Isaacs
d35f512cd9 ctdb-vacuum: Use non-blocking lock when traversing delete queue
This avoids vacuuming getting in the way of ctdb daemon to process
record requests.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-12-05 14:43:07 +01:00
Amitay Isaacs
e4597f8771 ctdb-vacuum: Stagger vacuuming child processes
This prevents multiple child processes being forked at the same time
for vacuuming TDBs.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-12-05 14:43:07 +01:00
Amitay Isaacs
a0628e317d ctdb-vacuum: Track time for vacuuming in database statistics
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-12-05 14:43:07 +01:00
Martin Schwenke
7f377cf26e ctdb-scripts: Fix stack dumping when debugging hung scripts
There are parentheses missing that stop the default pattern from
matching commands with trailing garbage (e.g. "exportfs.orig").

A careful check of POSIX (and running GNU sed with --posix) suggests
that "\|" isn't a supported way of specifying alternation in a regular
expression.  Therefore, it is clearer to switch to extended regular
expressions so that this has a chance of being portable (even though
the point is to print /proc/<pid>/stack, which only works on Linux).

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue Nov 18 06:37:45 CET 2014 on sn-devel-104
2014-11-18 06:37:44 +01:00
Martin Schwenke
4cd5be87da ctdb-scripts: Try to restart statd after every 10 failures
Also add and update tests for statd stack dumps.  Update the existing
60.ganesha statd test to do more iterations.  Duplicate the result as
a new test for 60.nfs.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-11-18 04:17:10 +01:00
Martin Schwenke
f51672f514 ctdb-scripts: Add rpc.statd stack dumping to Ganesha restart
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-11-18 04:17:10 +01:00
Martin Schwenke
968401ccdc ctdb-scripts: Dump stack traces for hung mountd, rquotad, statd processes
Add a corresponding new unit test for statd.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-11-18 04:17:10 +01:00
Martin Schwenke
1f49e1ab5b ctdb-scripts: Add optional program name argument to nfs_dump_some_threads()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-11-18 04:17:10 +01:00
Martin Schwenke
2ebc305be6 ctdb-scripts: Factor out new function program_stack_traces()
In the process, fix a bug where an extra trace would be printed.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-11-18 04:17:10 +01:00
Amitay Isaacs
d04bfc6ec6 ctdb-daemon: Improve error handling for running event scripts
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Nov 14 03:06:12 CET 2014 on sn-devel-104
2014-11-14 03:06:12 +01:00
Martin Schwenke
d558281205 ctdb-includes: Reorder includes.h more logically
For now replace.h needs to be before tdb.h, since tdb.h needs
stdbool.h.  This is fixed upstream but not yet in various packaged
versions.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Oct 31 03:46:40 CET 2014 on sn-devel-104
2014-10-31 03:46:40 +01:00
Martin Schwenke
94f248d1cc ctdb-includes: Remove unused idtree.h
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-31 01:26:04 +01:00
Martin Schwenke
7fce187a0d ctdb-includes: Clean up includes.h
Remove unused #defines.  CTDB doesn't use these.  _PUBLIC_ is defined
in replace.h anyway.

Remove unnecessary #includes.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-31 01:26:03 +01:00
Amitay Isaacs
a54db687ac ctdb: Rename CTDB_VERSION to CTDB_PROTOCOL
CTDB_VERSION really is the ctdb protocol version.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
2014-10-28 05:42:05 +01:00
Amitay Isaacs
0d5ecaa2e3 ctdb: Rename ctdb socket variable from CTDB_PATH to CTDB_SOCKET
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
2014-10-28 05:42:04 +01:00
Martin Schwenke
1a8d431936 ctdb-logging: Add logging via UDP logging using RFC5424
Some implementations may not understand RC3164 format messages on the
UDP socket, so add support for RFC5424 message format.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
8ed3ff456c ctdb-logging: Add logging via UDP to 127.0.0.1:514 to syslog backend
This has most of the advantages of the old logd with none of the
complexity of the extra process.  There are several good syslog
implementations that can listen on the UDP port.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
a6e770ec28 ctdb-logging: Add non-blocking Unix domain logging to syslog backend
Format messages as per RFC3164.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
1d1cd04cb9 ctdb-logging: New option CTDB_LOGGING, remove CTDB_LOGFILE, CTDB_SYSLOG
Remove --logfile and --syslog daemon options and replace with
--logging.

Modularise and clean up logging initialisation code.  The
initialisation API includes an app_name argument that is currently
unused - this will be used in extensions to the syslog backend.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
2974554356 ctdb-logging: Replace logd code with a basic syslog(3) implementation
It is much simpler for most cases to have a syslog backend that
doesn't need a separate CTDB-specific logging daemon.  This loses the
lossy, non-blocking mode provided by logd.  However, a corresponding
feature with a completely different implemention (not requiring an
extra daemon) will be re-added into the syslog backend.  In an ideal
world the new implementation would be added first but unfortunately
that is hard to do because the logd code is hooked in at more than one
place.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
38c8e15690 ctdb-logging: Simplify file logging timestamping with timeval_str_buf()
If nothing else, this is slightly more portable.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
739324eead ctdb-logging: Separate out syslog and file logging backends
This makes the code cleaner and allows the syslog backend to be easily
modified without affecting other code.  Also do some extra clean-up,
including whitespace fixups.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
a22c8ca056 ctdb-logging: Rework debug level parsing
Put declarations into ctdb_logging.h, factor out some common code,
clean up #includes.

Remove the check so see if the 1st character of the debug level is
'-'.  This is wrong, since it is trying to check for a negative
numeric debug level (which is no longer supported) and would need to
be handled in the else anyway.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
d9d572a23c ctdb-logging: Remove ctdb element from struct ctdb_log_state
This is set but otherwise not used.  This allows the 1st argument to
ctdb_set_logfile() to be generalised to a TALLOC_CTX.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
dfd502c80c ctdb-logging: Remove log member from struct ctdb_context
This is only used by logging code and there is already a file-level
variable for this.  struct ctdb_context already contains too many
things.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
7d391b7466 ctdb-logging: Rename ctdb_log_handler() to ctdb_child_log_handler()
Now it is obvious that it has something to do with child processes.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
f4fc9a153c ctdb-logging: Remove debug levels DEBUG_ALERT and DEBUG_CRIT
Internally map them to DEBUG_ERR to limit code churn.

This reduces the unwieldy number of debug levels used by CTDB.  ALERT
and CRIT aren't of much use as separate errors, since everything from
ERR up should always be logged.  In future just ERR can be used.

This also improves compatibility with Samba's debug.c system priority
mapping.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
0eabbb8c2b ctdb-logging: Remove DEBUG_EMERG
It isn't used and shouldn't be.  CTDB can't make the system unusable.

Update associated test to ensure that EMERG isn't attempted.  Actually
test all remaining debug levels and modernise the test a bit.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
85cc4efbff ctdb-test: Remove unused function check_ctdb_logfile()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Martin Schwenke
96106e085a ctdb-logging: Add missing newline when logging to file
This got lost with the transition to the new Samba debug code.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-10-28 05:42:04 +01:00
Amitay Isaacs
470af88147 ctdb-tools: Fix heap-use-after-free problem
Found by address sanitizer.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Oct 17 12:56:02 CEST 2014 on sn-devel-104
2014-10-17 12:56:02 +02:00
Martin Schwenke
d568be6989 ctdb-build: Make some steps conditional on standalone build
CTDB's current wscript assumes that it is only used as part of a
standalone build.  However, it will soon be part of a unified build so
make some steps conditional.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-10-08 07:19:04 +02:00
Martin Schwenke
6b98ddc54f ctdb-build: Rename define BINDIR to CTDB_HELPER_BINDIR
This avoids a clash with Samba's BINDIR and also makes it easier to
move the helpers to somewhere else (e.g. libexec) in the future.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-10-08 07:19:04 +02:00
Martin Schwenke
88b24858d8 ctdb-build: Fix handling of public headers
Add the header munging, add/package ctdb_version.h, create
directory include/public.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Mon Oct  6 14:56:07 CEST 2014 on sn-devel-104
2014-10-06 14:56:06 +02:00
Martin Schwenke
59c3025706 ctdb-build: Change from ctdb-util to samba-util
Remove local lib/util and lib/tdb-wrap. Update wscript, packaging and
includes.h.

The only potentially surprising thing here is a fake samba-util
subsystem that just depends on samba-util-core.  As explained in a
comment:

  When a combined build is implemented, CTDB will wanted to build
  against samba-util rather than samba-util-core.  Similarly, other
  Samba subsystems expect samba-util.  So, for a standalone build,
  just define a fake samba-util subsystem that pulls in
  samba-util-core.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-10-06 12:34:33 +02:00
Martin Schwenke
43266be945 ctdb-build: Add generation of Samba-style version.h
This is needed by fault.c.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-10-06 12:34:33 +02:00
Martin Schwenke
ba3e2ecf67 ctdb-build: Move generation of ctdb_version.h earlier
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-10-06 12:34:33 +02:00
Martin Schwenke
b4589b954e ctdb-logging: Update to use Samba style debug.h/debug.c
Samba's debug subsystem has changed a lot, so CTDB's logging needs
to be rewritten to be compatible.

The new debug.h/debug.c can't just be pulled in because it has some
extra dependencies into Samba's lib/util.  For now, to support the
smallest possible patch, implement a minimal subset of Samba's
debug.[ch] that just supports the DEBUG_CALLBACK logtype.

Define a callback for each logging method.

Check later to see if debug_extra (or similar) can somehow be
implemented using debug classes.

The timestamp on CTDB CLI tool and test program DEBUG() output goes
away, so update the unit test code to cope.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-10-06 12:34:33 +02:00
Martin Schwenke
3105737c4b ctdb-logging: Change LogLevel to DEBUGLEVEL
For compatibility with current Samba debug.[ch].

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-10-06 12:34:33 +02:00
Martin Schwenke
1c8c0be6f9 ctdb-tests: Make the fake log timestamp string easy to modify
Use a variable to allow easy change of this string in case future
logging changes modify the timestamp format or do not support
timestamping.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-10-06 12:34:32 +02:00
Martin Schwenke
b544073653 ctdb-logging: Remove log ringbuffer
As far as we know, nobody uses this and it just complicates the
logging subsystem.

Remove all ringbuffer code and documentation.  Update the local
daemons startup code correspondingly.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-10-06 12:34:32 +02:00
Jeremy Allison
fc8e1056fe lib: util [ctdb]: Signal handling - change CatchChild() and CatchChildLeaveStatus() to return the previous handler.
Bug #10831 - SIGCLD Signal handler not correctly reinstalled on old library code use - smbrun etc.

https://bugzilla.samba.org/show_bug.cgi?id=10831

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-09-30 20:40:16 +02:00
Andrew Bartlett
ceb26257dd lib/util: Do not duplicate the protocol list, use smb_constants.h
This avoids the two lists getting out of sync, and only applies to a Samba build due to the surrounding #ifdef

Andrew Bartlett

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-09-29 08:28:07 +02:00
Amitay Isaacs
0f92de8463 ctdb-daemon: Fix the usage for lock helper
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Sep 25 17:16:31 CEST 2014 on sn-devel-104
2014-09-25 17:16:31 +02:00
Amitay Isaacs
57310f80c9 ctdb-recoverd: If obtaining recovery lock fails, try again
When ctdb daemon starts up, it considers itself the recovery master
and tries to do first recovery.  However, it's possible that there is
already a recovery master and the current node has not yet heard from it.
So do not ban ourselves immediately if ctdb_recovery_lock() fails when
doing first recovery.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-09-25 14:48:11 +02:00
Amitay Isaacs
f1e281cd47 ctdb-scripts: Fix the regular expresssion for parsing /proc/locks
The major and minor device numbers are hexadecimal not decimal.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Sep 25 07:19:59 CEST 2014 on sn-devel-104
2014-09-25 07:19:59 +02:00
Amitay Isaacs
c64369cba2 ctdb-locking: Reset ttimer before doing an early return
When timer expires, timeout handler routine sets lock_ctx->ttimer
to a newly created timer event.  However, when a node is INACTIVE,
timeout handler returns early with lock_ctx->ttimer set to the previous
timer event.  This timer event gets freed when the callback returns and
lock_ctx->ttimer remains set to already freed timer event.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-09-25 04:54:10 +02:00