IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
This triggers a child process to be created to perform the actual potentially blocking calls that are required.
(This used to be ctdb commit 7d575ee92c95bc4aab78a33bc1aac7ff0811ab3a)
Once the contexts are freed, the deferred calls are re-issued to the input packet processing functions again.
This is needed when/if a CALL can not currently be processed by the main engine due to the record being locked down for revoking of all delegations.
The data is passed through several layers of callbacks, and finally a timed event callback to ensure that the processing of the packet will be restarted again at the topmost eventloop, avoinding event loop nesting.
(This used to be ctdb commit cc6f78efcfa3b8caeffbd68018e6dfbf81488dce)
Assume all databases will support readonly mode for now and se thte flag for all databases. At later stage we will add support to control on a per database level whether delegations will be supported or not.
(This used to be ctdb commit 502f86f79944df4bac9094f716e54110c511dc24)
This function differs from the old FETCH in that this function will also fetch the record header and not just the record data
(This used to be ctdb commit c7196d16e8e03bb2a64be164d15a7502300eae0e)
Move struct ctdb_public_ip_list to ctdb_private.h and put some
definitions for some functions from ctdb_takeover.c there. This
allows those functions to be called from unit tests.
Add ctdb_takeover_tests.c and the Makefile support to build it.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 9d34be0233edf3bc022345c0494c4b2a4d7f8480)
The current non-deterministic IP allocation algorithm balances IPs
across the whole cluster. It does not consider different
interfaces/VLANs/subnets, so these different groups of IPs aren't
generally well balanced.
This adds the LCP2 algorithm for IP allocation and allows it to be
enabled by setting the "LCP2PublicIPs" tunable to 1.
The LCP2 algorithm calculates the imbalance of a node by totalling the
squares of the distances between each IP on the node. The IP distance
is defined as the length longest common prefix (LCP) of bits that is
found when comparing 2 IPs. The imbalance of a cluster is the maximum
imbalance for any node. At each step the algorithm selects an
allocation to the IP/node combination that results in the choosing the
allocation that best reduces the imbalance of the cluster.
The implementation splits out the IP allocation part of
ctdb_takeover_run() into new function ctdb_takeover_run_core(), and
then extracts out the basic IP assignment code into new functions
basic_allocate_unassigned() and basic_failback(). 3 new functions
lcp2_init(), lcp2_allocate_unassigned() and lcp2_failback() implement
the LCP2 algorithm, and are hooked into ctdb_takeover_run_core().
Signed-off-by: Martin Schwenke <martin@meltin.net>
(This used to be ctdb commit 61fc7fbd0235469df22deb6581c6bd47e30bc0be)
This is a flag that shall signa that a record has been automatically generated by ctdb
and not by an explicit client store operation. This will be used in the ctdb_ltdb_fetch
operation which stores an empty record with default initial header before trying to
migrate the record from the dmaster when the record does not exist in the local tdb.
(This used to be ctdb commit 46381a3cb58ccc11422af8f7798c80ea8d72294f)
This is realized by adding a ctdb_ltdb_store_fn function pointer to the db
context and filling it in the attach procedure for non-persistent dbs.
(This used to be ctdb commit df49ec44de80affa5ccc637dec12a20a26e8706e)
This is for the control dispatcher to check whether the input data has
a required minimum size.
(This used to be ctdb commit 2038e745db33cc5c3b4e2db8a00a57ede03906a2)
This will control how many fast-path vacuuming runs wil have to
be done, before a full vacuuming will be triggered, i.e. one with
a db-traversal.
(This used to be ctdb commit 0d997ec7e61a7bee2cb05456f9c7d5e6f7a44797)
This list will be filled by the client using a new
delete control. The list will then be used to implement
a fast-path vacuuming that will traverse this list instead
of traversing the database.
(This used to be ctdb commit 9bbedf786b26bb074f668b31f29a9032af958673)
This is to be used internally. The purpose is to flag a record
as been migrated by a VACUUM_MIGRATION, which is triggered by
a VACUUM_FETCH message as part of the vacuuming. The local store
routine will base its decision whether to delete or to store
the record (among other things) upon the value of this flag.
This flag should never be stored in the local database copies.
(This used to be ctdb commit dd2449c422f323f9b5485e45107a9cc5acc09e08)
This is to be used when the CTDB_SRVID_VACUUM_FETCH message
triggers the migration of deleted records to the lmaster.
The lmaster can then delete records that have not been
migrated with data instead of storing them.
(This used to be ctdb commit 455cc6616e10b7f09589f9b87cb60f591bb502b0)
This function walks all databases and checks for running trans3 commits.
It sends replies to all of them (with error code) and ends them.
To be called when a recovery finishes.
(This used to be ctdb commit 70ba153b532528bdccea70c5ea28972257f384c1)
One of which signals that the record has never been migrated to/from a node
while containing data.
This property "has never been migrated while non-zero" is important later
to provide heuristics on which records we might be able to purge
from the tdb files cheaply, i.e. without having to rely on the full-blown
database vacuum.
These records are belived to be very common and the pattern would look like
this :
1, no record exists at all.
2, client opens a file
3, samba requests the record for this file
4, an empty record is created on the LMASTER
5, the empty record is migrated to the DMASTER
6, samba writes a <sharemode> to the record locally and the record grows
7, client finishes working the file and closes the file
8, samba removes the sharemode and the record becomes empty again.
9, much later : vacuuming will delete the record
At stage 8, since the record has never been migrated onto a node wile being
non-zero it would be safe, and much more efficient to just delete the record
completely from the database and hand it back to the LMASTER.
The flags occupy the same uint32_t as was previously used for laccessor/lacount
in the header. For now, make sure the flags only define/use the top 16 bits
of this field so that we are sure we dont collide with bits set to one
from previous generations of the ctdb cluster database prior to this
change in semantics of this word.
This is a rework of Michaels patch :
commit 2af1a47cbe1a608496c8caf3eb0c990eb7259a0d
Author: Michael Adam <obnox@samba.org>
Date: Tue Nov 30 17:00:54 2010 +0100
add a DEFAULT record flag and a MIGRATED_WITH_DATA record flag.
(This used to be ctdb commit e075670dee8e6ecaba54986f87a85be3d0528b6b)
This concept didnt work out and it is really just as expensive as a full migration
anyway, without the benefit of caching the data for subsequence accesses.
Now, migrate the records immediately on first access.
This will be combined with a "cheap vacuum-lite" for special empty records to
prevent growth of databases.
Later extensions to mimic read-only behaviour of records will include proper shared read-only locking of database records, making the laccessor/lacount read-only access to the data obsolete anyway.
By removing this special case and handling of lacount laccessor makes the codapath where shared read-only locking will be be implemented simpler, and frees up space in the ctdb_ltdb header for use by vacuuming flags as well as read-only locking flags.
(This used to be ctdb commit 155dd1f4885fe142c6f8bd09430f65daf8a17e51)
Add a dlist to track all active lockwait child processes.
Everytime creating a new lockwait handle, check if there is already an
active lockwait process for this database/key and if so,
send the new request straight to the overflow queue.
This means we will only have one active lockwaic child process for a certain key,
even if there were thousands of fetch-lock requests for this key.
When the lockwait processing finishes for the original request, the processing in d_overflow() will automagically process all remaining keys as well.
Add back a --nosetsched argument to make it easier to run under gdb
(This used to be ctdb commit 3e9317a2e1f687b04bf51575d47fcd4faa6e6515)
Once we have more than 200 children waiting on a particular db, don't create
any more. Just put them on an overflow queue, and when a child gets a lock
search that queue to see if others were after the same lock (they probably
were).
(This used to be ctdb commit 5e614e8cfd1e9a4b13035a0e400b7a60a745b510)
scheduler for the child.
Use ctdb_fork() from callers where we dont want the child to be running
at real-time privilege.
(This used to be ctdb commit 58795a4c9e0624e20fa3e0023b65127053edd103)
Revert this patch:
commit 482c302d46e2162d0cf552f8456bc49573ae729d
We may need to use real-time processes for the main daemon and the recovery daemon to handle the cases where systems come under very high loads.
(This used to be ctdb commit 08bef9dcab6e4da15fc783f8624e5ed09aa060b5)
Add a new command "ctdb stats [num]" that prints the [num] most recent statistics intervals collected.
(This used to be ctdb commit e6e16fcd5a45ebd3739a8160c8fb5f44494edb9e)
define a new symbol to represent this range similarly to NFSD and ISCSID
Keep the old symbol name to be backward compatible with software using
these headers.
(This used to be ctdb commit 2ce34e50d057ba95249117a581658a5ad7e8eb60)
This function returns a pointer to a nodemap structure.
The returned structure must later be freed by calling ctdb_free_nodemap().
Move the definition of ctdb_sock_addr from ctdb_client.h to ctdb_protocol.h
Move the definition of the node flags, ctdb_node_and_flags and ctdb_node_map from ctdb_private.h to ctdb_protocol.h
Add both sync and async example for ctdb_getnodemap to the test application libctdb/tst.c
(This used to be ctdb commit 31c10eb2b337fd7d8a97a1f9e69b0e7570fec71d)
This is called everytime a reallocation is performed.
While STARTRECOVERY/RECOVERED events are only called when
we do ipreallocation as part of a full database/cluster recovery,
this new event can be used to trigger on when we just do a light
failover due to a node becomming unhealthy.
I.e. situations where we do a failover but we do not perform a full
cluster recovery.
Use this to trigger for natgw so we select a new natgw master node
when failover happens and not just when cluster rebuilds happen.
(This used to be ctdb commit 7f4c591388adae20e98984001385cba26598ec67)
Add a new "ctdb deltickle" command to delete tickles from the database.
This can ONLY be used for tickles created by "ctdb addtickle".
Push any "addtickle/deltickle" updates to other nodes every TickleUpdateInterval seconds'
(This used to be ctdb commit acded034e2f0dcae4c2c9e54e16a001caf23caec)
This means we can distinguish which child is logging, esp. via syslog where we have no pid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 68b3761a0874429b90731741f0531f76dcfbb081)
There are some reports of freeze timeouts, and it looks like vacuuming might
be the culprit. So we add code to tell them to abort when a freeze is
going on.
(This is based on the 1.0.112 branch version 517f05e42f, but far
simpler since tdb is now robust against processes being killed during
transaction commit)
CQ:S1018154 & S1018349
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit f5d7dc679501e607c2c83a248a89d3cada9df146)
In Samba this is now called "tevent", and while we use the backwards
compatibility wrappers they don't offer EVENT_FD_AUTOCLOSE: that is now
a separate tevent_fd_set_auto_close() function.
This is based on Samba version 7f29f817fa.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 85e5e760cc91eb3157d3a88996ce474491646726)
It turns out that we *do* want a separate private arg for the message
handler and the completion callback, so we change that.
We also fix the prototypes of the remove_message functions as we
implement them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 332375246eccd95da626f434f6d49dd9458a9787)
In include/ctdb.h, ctdb_callback_t and ctdb_rrl_callback_t were
defined with a void *private variable. The variable name was
changed to void *private_data to avoid issues encountered in
the Samba autoconf script.
Evan Kinney <evan.kinney@sas.com>
(This used to be ctdb commit 1f453aa4b5e749468c7788afac09c6f0900ea18f)
We've been seeing "Invalid packet of length 0" errors, but we don't know
what is sending them. Add a name for each queue, and print nread.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit e6cf0e8f14f4263fbd8b995418909199924827e9)
The extra recovery interval wait was introduced in 821333afb458 but no
explanation was provided in that message. Nonetheless, if starting
the entire cluster for the first time, it should be safe to skip this.
We use the commandline arg --sloppy-start which should discourage
people from using it outside testing.
Seconds between ctdbd first log message and node healthy:
BEFORE: 16.10
AFTER: 4.03
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 509e2e89ae233a0e91998d95267bf62f296a73cd)
Because this doesn't use a generic callback, it's not quite as trivial
as the other sync wrappers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 1f20b938d46d4fcd50d2b473c1ab8dc31d178d2d)
These are important for testing, since we can easily tell if we
leak memory if there are outstanding allocations after calling
these.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 18a212aa40d0ff9ff59775c6fcf9dc973e991460)
Ronnie and I tracked down a bug which seems to be caused by a node
running so slowly that we timed out the request and reused the request
id before it responded.
The result was that we unlocked the wrong record, leading to the
following:
ctdbd: tdb_unlock: count is 0
ctdbd: tdb_chainunlock failed
smbd[1630912]: [2010/06/08 15:32:28.251716, 0] lib/util_sock.c:1491(get_peer_addr_internal)
ctdbd: Could not find idr:43
ctdbd: server/ctdb_call.c:492 reqid 43 not found
This exact problem is now detected, but in general we want to delay
id reuse as long as possible to make our system more robust.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 9eb9c53ef29f4871ae2fe62fc5cb6145fca89eed)
I missed some int->bool conversions previously, particularly the
return of ctdb_writerecord().
By always handing functions ctdb_connection or ctdb_db, we keep it
consistent with the rest of the API and can do extra lock consistency
checks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 3f939956ddd693cba6ea5c655288f4f5ca95f768)
Now we have more messages, it seems to make sense to document their usage
and make them consistent.
In particular, LOG_CRIT for internal libctdb problems, LOG_ALERT for
API misuse.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit a6fed3f577c7ec51df38ed15ecb9db6ea2ae7c8f)
This allows us to keep the datastructure valid after the lock has been released by the application and we can trap and warn when the application is accessing the lock after it has been released. I.e. application bugs.
(This used to be ctdb commit 463a266205f145cd9c4c36b9c59d3747eeef0e2e)
Full documentation for all the functions.
This looks longer than it is, because it sorts them into async and
sync parts, and also renames some formal parameters.
Added TODO to libctdb directory to track our plans.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 108e9c2450876a9f8821aa7efd5be971eee5afd3)
We're best off including ctdb_protocol.h to get these, even if we
document the important ones in ctdb.h.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit cdc19dc73032470d57f38bf825d8113b3a0c8cd1)
Return bool instead of -1/0; that's what the young kids are doing
these days!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit e285b5d5a9d4fbc4f75dbb237d2fcdbd84f2d605)
This is based on Ronnie's work, merged with mine. That means
errors are all my fault.
Differences from Ronnie's:
1) use syslog's LOG_ levels directly.
2) typesafe arg to log function, and use it (eg stderr) in helper function.
3) store fn in ctdb context, and expose ctdb_log_level directly thru API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 86259aa395555aaf7b2fae7326caa2ea62961092)
This is going to help for logging, since we want it there.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 0786152472bc43efae4c896f7c6c07c6e080b9b2)
After discussion with Ronnie, we decided to revisit this interface. We use
the name ctdb_readrecordlock_async, as it is *not* always a send, and we
use a specific callback to avoid the "fake request" creation on the fast
path.
The request itself is never exposed: this means it can't be cancelled,
but we can revisit that later if need be.
This makes both use and implementation simpler.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 03b5546ae45a60ab41eb4f7159a45bfdbf959888)
PDUs are padded to 8 byte boundary. If padding is used, memset it to 0
to keep valgrind happy.
(This used to be ctdb commit 8818d5c483558c0faa6a3923ed5e675fdcfc13af)
and print the time startistics was taken and for how long the statistics have been collected to the "ctdb statistics" output.
(This used to be ctdb commit 1bdfe0cd3370a335b960ce1ef97eade93b0cd2fa)
Previously we could hang in poll with the callback pending (since we
fake it): explicitly call it immediately.
Note: I experienced corruption using DLIST_ADD_END (ctdb->pnn was blatted
when adding to the message_handler list). I switched them all to DLIST_ADD,
but maybe I'm using it wrong?
(This used to be ctdb commit 3727165f0d206999d2cfc2800ff8868640868c7c)
This is a bit tricky for those cases where we need to do multiple or
zero I/Os (eg. attachdb and readrecordlock), but works well for the
simple cases.
(This used to be ctdb commit ebe4dd724338c156423cfdcc10a75b68c2084cde)
These simplifications mostly came up due to the implementation.
o Rename ctdb_context to ctdb_connection.
We already have a ctdb_context internally in ctdbd; don't confuse them!
o Rename ctdb_handle to struct ctdb_request.
From the user POV it's a request, and it's also useful internally to
avoid implicit cast to/from void *.
o Rename ctdb_db_context to ctdb_db.
o Introduce ctdb_lock.
This provides an explicit "lock object" you get from readrecordlock
and have to hand to those functions which need you to hold a lock.
o status args are "int" not int32_t.
Should this be a bool?
o Remove last traces on generic callback.
Without semi-sync API, this doesn't help anything and loses type safety.
o Remove the semi-async API.
We can add this later, but I think a sync and async API is enough for
our poor users for the moment :)
o Registering a message handler also takes a callback.
This way you can tell if it failed. Not sure if this is overkill, but it's
consistent.
o ctdb_service() takes an revents arg
Strictly not necessary for a nonblocking fd, but nice to know if a
read or write is possible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 86e1f93df856f9627182ed0e18bfcff6866c0954)
This imports ctdb.h and tst.c from Ronnie's work: it's a separate commit
for now to make the changes obvious.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 09f05cbfc883e5aac33d3781b163cde178ece4cf)
ctdb_client.h is the existing internal client interface (which was mainly
in ctdb.h), and ctdb_protocol.h is the information needed for the wire
protocol only.
ctdb.h will be the new, shiny, libctdb API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 4bba6b8cd47b352f98d41f9f06258d5ac3c9adef)
This resolves a problem with huge numbers of requests which could overflow
16 bits. Fortunately, the IDR should scale reasonably well, so we can simply
hold all the requests.
Although noone checks for failure, I added a constant for that.
BZ: 60540
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 72efc4122e37798227c3420a65ed1f706ca9ebe7)
verify that all nodes agree on the most recent ip address assignments
broke "ctdb moveip ..." since that call would never trigger
a full takeover run and thus would immediately trigger an inconsistency.
Add a new message to the recovery daemon where we can tell the recovery daemon to update its assignments.
BZ62782
(This used to be ctdb commit e7069082e5f0380dcddee247db8754218ce18cab)
In the case of a timeout, we dump a log of what's happening to a file
in /tmp. We do it from the signal handler, which is an unreliable hack
(BZ58365).
Instead, create another (lower-priority) child to do the dump, then
kill the timedout script.
Note that this doesn't quite work as intended (the dump is often run
after the script has been killed), so the next patch resolves this.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 7ee5ecc8d53e78e2dec21197b74a74cc4ae1834c)
addresses and verify that the remote nodes have/keep a consistent view of
assigned addresses.
If a remote node has an inconsistent view of addresses visavi the recovery
master this will trigger a full ip reallocation.
(This used to be ctdb commit f3bf2ab61f8dbbc806ec23a68a87aaedd458e712)
