1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-21 18:04:06 +03:00

1185 Commits

Author SHA1 Message Date
Martin Schwenke
014a1eb540 ctdb-scripts: CTDB_BASE must be set when including functions file
Also fix an unused test to set CTDB_BASE.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2015-09-15 08:56:21 +02:00
Martin Schwenke
c58d582ffb ctdb-scripts: Factor out possible creation of rt_tables file
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2015-09-15 08:56:21 +02:00
Martin Schwenke
677467682a ctdb-scripts: Move remaining state files into CTDB_SCRIPT_VARDIR
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2015-09-15 08:56:21 +02:00
Martin Schwenke
3aa11b175a ctdb-scripts: New internal variable CTDB_SCRIPT_VARDIR
Clearly identifies where all state files from scripts should go and
allows that location to be easily changed.  This patch should not
change any behaviour (outside of eventscript unit tests, where a
clearer location is now used).

CTDB_VARDIR should no longer be overridden.  Continue to set
CTDB_DBDIR and similar to override database location in unit tests.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2015-09-15 08:56:21 +02:00
Martin Schwenke
d064620f51 ctdb-scripts: Properly set CTDB_VARDIR in scripts at install time
Have wscript do path substitution.

No need to export this and CTDB_ETCDIR here, but test scripts will
still need to do so.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2015-09-15 08:56:21 +02:00
Martin Schwenke
e7a139bffe ctdb-scripts: Drop internal file-level variables referencing $CTDB_VARDIR
Using $CTDB_VARDIR at file scope is dangerous because it doesn't
respect the configuration.  Uses of these variables are simple so just
drop the variables and use $CTDB_VARDIR inside functions where it is
safe.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2015-09-15 08:56:20 +02:00
Martin Schwenke
143ec9e4c2 ctdb-scripts: Drop file removal involving $ctdb_managed_dir
This directory has been under $CTDB_VARDIR for a long time, so is
already removed.  The command is also potentially dangerous if
$ctdb_managed_dir is not set.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2015-09-15 08:56:20 +02:00
Martin Schwenke
353c27deab ctdb-scripts: Drop functions ctdb_checkstatus() and ctdb_setstatus()
They're not used outside of ctdb_standard_event_helper().

As a consequence, make ctdb_standard_event_helper() do nothing.  It is
harder to remove because it is used in many places, perhaps by
external eventscripts where it has been copied from existing scripts.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2015-09-15 08:56:20 +02:00
Martin Schwenke
8c2948eba5 ctdb-scripts: Drop 62.cnfs eventscript
To get a similar effect just do something like this:

  mmaddcallback ctdb-disable-on-quorumLoss \
    --command /usr/bin/ctdb \
    --event quorumLoss --parms "disable"

  mmaddcallback ctdb-enable-on-quorumReached \
    --command /usr/bin/ctdb \
    --event quorumReached --parms "enable"

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2015-09-15 08:56:20 +02:00
Martin Schwenke
9d49dfac5f ctdb-scripts: New internal variable CTDB_NFS_CALLOUT_STATE_DIR
60.nfs sets this to a directory where the NFS callout can store state.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Sep 14 08:36:34 CEST 2015 on sn-devel-104
2015-09-14 08:36:34 +02:00
Martin Schwenke
4164d7bf31 ctdb-scripts: Add default filesystem usage warnings
Always check filesystem usage for the database directories.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Sat Aug 29 20:08:48 CEST 2015 on sn-devel-104
2015-08-29 20:08:47 +02:00
Martin Schwenke
0f28ccf87a ctdb-scripts: Add default system memory usage warnings
CTDB should warn by default if too much system memory or swap is used.

The tests have also been tweaked.  In particular, the filesystem-only
tests need to initialise the memory information to avoid errors where
meminfo isn't set.

Document the defaults, warning against disabling them.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
2c601f1895 ctdb-scripts: Enable system monitoring eventscript by default
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
b18e4ae0c9 ctdb-scripts: Throttle system resource monitoring warnings
They are only printed when the percentage usage changes.  This should
stop the logs from being filled with warnings.

Add a test for the throttling.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
e6b5163bc1 ctdb-scripts: Don't shutdown CTDB when memory monitoring fails
Marking the node unhealthy should cause Samba processes to close,
possible freeing a stack of memory.  If not, then it is somebody
else's problem.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
b6a0e4b856 ctdb-scripts: New consistent system memory and swap monitoring
New variables CTDB_MONITOR_MEMORY_USAGE and CTDB_MONITOR_SWAP_USAGE.
Both take a pair of <warn_threshold>:<unhealthy_threshold> where each
theshold is specified as a percentage.

This adds a callout to check_thresholds() that is run when the
unhealthy threshold is reached.

Add some combination tests.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
02fa6c3d10 ctdb-scripts: Factor out new function check_thresholds()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
b7b6e25b3e ctdb-scripts: Memory monitoring uses thresholds expressed as percentages
CTDB_MONITOR_FREE_MEMORY and CTDB_MONITOR_FREE_MEMORY_WARN are now
percentages that specify thresholds of acceptable memory usage.

Memory/swap usage in tests also specified as percentages.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
bd2845d7eb ctdb-scripts: Use MemAvailable if it is in /proc/meminfo
Otherwise calculate, as before.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
99b8ef5121 ctdb-scripts: Only use /proc/meminfo for memory checks, not "free"
No need to use 2 different sources of information for similar checks.
Also, output of free has been changed, whereas /proc/meminfo is a
kernel API, which will not change.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
ab58c7abd9 ctdb-scripts: Move system memory checking to 05.system
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
fa1050690b ctdb-scripts: New configuration variable CTDB_MONITOR_FILESYSTEM_USAGE
This allows both errors (i.e. unhealthy) and warnings for different
thresholds.  It replaces CTDB_CHECK_FS_USE.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
8f713c87c1 ctdb-scripts: Don't fail monitoring if sanity checks fail
Just log some warnings.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
6b4a46e574 ctdb-scripts: Move filesystem monitoring into a function, clean it up
Drop obvious comments.  Use die() for less lines of code.  Use a case
statement to avoid forking unnecessary processes for each filesystem
being checked.  Drop parentheses around percentages in messages.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
47f7d1b1c8 ctdb-scripts: Rename 40.fs_use to 05.system
Will put all the system monitoring in here, simplifying 00.ctdb.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-29 17:06:25 +02:00
Martin Schwenke
7d04778c82 ctdb-scripts: Improve error handling for 50.samba testparm failure
Also add tests.  Update testparm stub to fake error and timeout.  Add
timeout stub.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-08-07 05:33:29 +02:00
Martin Schwenke
b0bc4d2cab ctdb-scripts: Move 60.nfs Ganesha callout to doc/examples/
We don't expect to maintain an up-to-date copy.  NFS Ganesha team
might provide patches.

Also move the Ganesha .check file

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-21 07:54:09 +02:00
Martin Schwenke
dd88c2ab8c ctdb-scripts: Support RPC checks for tcp6 and udp6
This adds new configuration variable CTDB_RPCINFO_LOCALHOST6.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-21 07:54:09 +02:00
Martin Schwenke
b6a3c1decd ctdb-scripts: Implement registration in nfs-linux-kernel-callout
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
fa6f22d7ae ctdb-scripts: Add registration for CTDB_NFS_CALLOUT operations
This is an optimisation to avoid forking the callout for operations
that are not implemented.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
bb7093ab09 ctdb-scripts: Add portmapper NFS .check file
Unhealthy after 1 failed attempt to contact the portmapper.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
a02bdb97f9 ctdb-scripts: Move NFS support functions to 60.nfs
Now that there is only a single NFS eventscript, other eventscripts no
longer need to load all of this.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
a3a443dcf4 ctdb-scripts: Drop configuration variable CTDB_NFS_DUMP_STUCK_THREADS
This is now handled by passing the desired number of threads to the
command specified in the dump_stuck_threads variable in .check files.

Remove unused function nfs_dump_some_threads().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
f3a4c4f10b ctdb-scripts: Remove unused function startstop_ganesha()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
6586651508 ctdb-scripts: Remove 60.ganesha, replace with callout for 60.nfs
This isn't a straightforward move of code from 60.ganesha to the
callout.  Simplifications have been made to allow better
interoperation with the new NFS checking logic.

The following configuration variables have been removed:

  CTDB_GANESHA_REC_SUBDIR

    Edit NFS ganesha callout to change this location

  CTDB_NFS_SERVER_MODE, NFS_SERVER_MODE

    Use CTDB_NFS_CALLOUT instead

  CTDB_NFS_SKIP_KNFSD_ALIVE_CHECK, CTDB_SKIP_GANESHA_NFSD_CHECK

    Disable the corresponding .check file instead

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
713ec21750 ctdb-scripts: Extend NFS .check files with service_check_cmd variable
$service_check_cmd specifies a command to run instead of the regular
rpcinfo-based check.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
d332013123 ctdb-scripts: Remove functions startstop_nfs() and startstop_nfslock()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
9c87d1dd29 ctdb-scripts: Parameterise 60.nfs with $CTDB_NFS_CALLOUT
The goal is to have a single NFS eventscript.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
49c4d1900c ctdb-scripts: Remove old NFS checking code
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
74428e5c14 ctdb-scripts: Switch NFS checks to new style
Note that the 60.ganesha RPC checks need to be identical to those in
the nfs-checks.d/ directory.  This is because the NFS unit test
infrastructure checks output against what should be produced by the
checks in nfs-checks.d/.  This is a minor issue, since one of the aims
of this work is to remove the need for a separate 60.ganesha.

In most cases configuration variable CTDB_NFS_DUMP_STUCK_THREADS is
now ignored.  This is now handled by passing the desired number of
threads to the command specified in the service_debug_cmd variable in
a .check file.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
3161d611bb ctdb-scripts: Add new NFS service checking infrastructure
Provides a new extensible format for .check files, using simple
variables instead of the unwieldy extended test(1) syntax now used.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
dfeb5b84fd ctdb-scripts: Factor out new function ctdb_counter_get()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
9f4f1c51fc ctdb-scripts: Move "ERROR:" prefix out of ctdb_check_rpc()
There will be warnings in addition to errors.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:18 +02:00
Martin Schwenke
1a9687f948 ctdb-scripts: Clean up ctdb_check_rpc()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:17 +02:00
Martin Schwenke
6f3ee81d17 ctdb-scripts: NFS RPC checks should be simple and consistent
Change status, nlockmgr, mountd, rquotad to be unhealthy after 6
rpcinfo check failures and do a verbose restart after every 2
failures.  Change 60.ganesha for consistency, since 60.ganesha tests
are broken and depend on the consistency.

Apart from the consistency aspect, the check infrastructure will soon
be simplified so that it only allows the equivalent of "unhealthy" and
"verbose restart:b" actions.

Update tests to have a corresponding numbers of iterations.  Run 1
extra iteration in most tests to check there are no unexpected
behaviour changes after the designated number of iterations completes.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:17 +02:00
Martin Schwenke
bc71251433 ctdb-scripts: Support monitoring of interestingly named VLANs on bonds
VLAN interfaces on bonds with a name other than <iface>.<id>@<iface>
are not currently supported.  That is, where the VLAN name isn't based
on the underlying bond name.  Such VLAN interfaces can be created with
the "ip link" command, as opposed to the "vconfig" command, or by
renaming a VLAN interface.

This is improved by determining the underlying interface name for a
VLAN from the output of "ip link".

No serious attempt is made to support VLANs with '@' in their name,
although this seems to be legal.  Why would you do that?

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-14 09:57:16 +02:00
Martin Schwenke
87c5c96b76 ctdb-scripts: Fix regression in VLAN interface support
Commit 6471541d6d2bc9f2af0ff92b280abbd1d933cf88 broke support for VLAN
interfaces.  Releasing a public IP address depends on
ip_maskbits_iface() and for a VLAN interface this will return an
interface of the form <vlan>@<iface>, which can't be fed back into
"ip" commands.

Update ip_maskbits_iface() to drop the '@' and everything after it.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reported-by: Jan Schwaratzki <jschwaratzki@ddn.com>
2015-07-14 09:57:16 +02:00
Martin Schwenke
0a65013b9d ctdb-scripts: Use an "if" statement instead of "&&"
If statd-callout is unwanted, so is removed, then this code fails.
Change to an "if" so that it succeeds as intended.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-09 06:23:21 +02:00
Martin Schwenke
0c609c9505 ctdb-scripts: Only write to /proc route flush files if they exist
On IPv4-only or IPv6-only systems one of these files will not exist.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-01 04:18:29 +02:00
Martin Schwenke
27674c413d ctdb-scripts: Create the directory containing the recovery lock
This will handle the most obvious cases.  It won't handle the case
where the directory is missing and the recovery lock location is
updated at run-time.  However, this is a good improvement.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2015-07-01 04:18:28 +02:00