IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Simple bash test for readonly locks on tdbbackup:
1. Running tdbbackup on a database with and without readonly locks enabled.
2. Dump both backups and original.
3. Check all three dumps match.
A binary sample_tdb.tdb file is included for the test because the existing
sample tdbs in lib/tdb/test are either corrupt or empty.
Signed-off-by: Aaron Haslett <aaron.haslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
The netcmd 'domain backup offline' command will use the tdbbackup tool but
require readonly locking of tdb databases, otherwise all database access would
be blocked during a backup. This patch adds the option. A backup script
should use this tool with the readonly locks option after taking a transaction
lock on the target database.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* Fix build on AIX
* Python3 compatibility fixes
* Use tdb_wipe_all in "erase" command
* Harden allocating the tdb recovery area
* Make sure the hash size fits
* Harden tdb_check_used_record against overflow
* Harden tdb_rec_read
* Handle TDB_NEXT_LOCK_ERR in tdb_traverse_internal
* Fix build warnings
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493
Here is the build error on AIX 7.1.
../../lib/tdb/tools/tdbtool.c:39:12: error: 'disable_lock' redeclared as different kind of symbol
static int disable_lock;
^~~~~~~~~~~~
In file included from /usr/include/sys/gfs.h:24:0,
from /usr/include/sys/vfs.h:27,
from ../../lib/replace/system/filesys.h:48,
from ../../lib/tdb/tools/tdbtool.c:26:
/usr/include/sys/lock_def.h:314:5: note: previous declaration of 'disable_lock' was here
int disable_lock(int,simple_lock_t);
^~~~~~~~~~~~
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
In py3, iterxxx methods are removed.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This is a lot quicker on large, fragmented databases. tdb_delete can
leave the freelist in a fragmented mess.
Also, it's a lot more robust: I've got a 4GB tdb file that was affected
by the problem fixed with c7211882a7. These databases have large space
at the end that is not part of any record or freelist
entry. tdb_wipe_all converts this space into a freelist entry. One
downside is that with those broken databases (which should not happen
after c7211882a7) have unallocated blocks in their file range after
this operation.
I think the speed advantage outweighs this disadvantage.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Many of those warnings are difficult to fix, but this one was easy :-)
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Mar 22 07:21:44 CET 2018 on sn-devel-144
* Add protection against EINTR.
* Truncate the file after expand failure, ENOSPC
* Use posix_fallocate() to expand the file
* Fix GCC compiler warnings
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Aug 24 21:17:48 CEST 2017 on sn-devel-144
This should be significantly faster than pwriting.
openbsd doesn't have posix_fallocate, so we do need the fallback. Also, it
might have weird failure modes, so we keep the old code in place except for
posix_fallocate returning success or ENOSPC.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Aug 24 05:38:49 CEST 2017 on sn-devel-144
More README.Coding, but I need "ret" in the next commit as well :-)
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Without this it's very easy to create virtually huge files: ftruncate expands a
file, the pwrites fail with ENOSPC, thus the write fails. The next writer runs
into the same situation, and ftruncate-expands the file even further. tdb_check
will then spend ages reading the 4GB of zeros byte by byte.
Here we hold the freelist lock or are inside a transaction, so it is safe to
cut the file again. Nobody can have used the space that we have tried to
allocate, so we can't have any stray pointers corrupting the database.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Due to the non-fixable bug in the BUCKET macro tdbtool list printed some
other hash chainlist, not the freelist.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=12888
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The following C program demonstrates the issue:
#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h>
#include <stdbool.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <dirent.h>
#include <sys/types.h>
int main(int argc, char **argv)
{
int hash = -1;
int tsize_signed = 10;
unsigned int tsize_unsigned = 10;
int bucket;
#define BUCKET(hash, tsize) ((hash) % (tsize))
bucket = BUCKET(hash, tsize_unsigned);
printf("hash [%d] tsize [%d] bucket [%d]\n", hash, tsize_unsigned, bucket);
bucket = BUCKET(hash, tsize_signed);
printf("hash [%d] tsize [%d] bucket [%d]\n", hash, tsize_signed, bucket);
return 0;
}
Output:
$ ./tmp
hash [-1] tsize [10] bucket [5]
hash [-1] tsize [10] bucket [-1]
The first version is what the current BUCKET() macro does. As a result
we lock the hashtable chain in bucket 5, NOT the freelist.
-1 is sign converted to an unsigned int 4294967295 and
4294967295 % 10 = 5.
As all callers will lock the same wrong list consistently locking is
still consistent.
Stumpled across this when looking at the output of `tdbtool DB list`
which always printed some other hashchain and not the freelist.
The freelist bucket offset computation doesn't use the BUCKET macro in
freelist.c (directly or indirectly) when working on the freelist, it
just directly uses the FREELIST_TOP define, so this problem only affects
tdbtool list.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The variable stores the hashtable bucket, not the hash. No change in
behaviour.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This is another level of indentation, but it took me a while staring at the
if-condition to find that "locked" was assigned the result of "==0", not the
return value of tdb_nest_lock().
Best viewed with "git show -b".
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
This fixes a GCC warning.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Aug 10 02:26:09 CEST 2017 on sn-devel-144
* allow tdb_traverse_read before tdb_transaction[_prepare]_commit()
* Improve documentation for tdb_transaction_start()
* Add new function tdb_transaction_active()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This will allow callers to avoid their own reference counting of transactions.
Additionally, this will always line up with the acutal transaction state, even
in the error cases where tdb can cancel the transaction
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
It now references the TDB_ALLOW_NESTING and TDB_DISALLOW_NESTING flags
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This restores the original intent of tdb_traverse_read() in
7dd31288a7
This is needed to avoid a deadlock with tdb_lockall() and the
transaction start, as ldb_tdb should take the allrecord lock during a
search (which calls tdb_traverse), and can otherwise deadlock against
a transaction starting in another process
We add a test to show that a transaction can now start while a read
traverse is in progress
This allows more operations to happen in parallel. The blocking point
is moved to the prepare commit.
This in turn permits a roughly doubling of unindexed search
performance, because currently ldb_tdb omits to take the lock due to
an unrelated bug, but taking the allrecord lock triggers the
above-mentioned deadlock.
This behaviour was added in 251aaafe3a for
Solaris 10 in 2005. But the run-fcntl-deadlock test works also on Solaris 10,
see https://lists.samba.org/archive/samba-technical/2017-April/119876.html.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This verifies the F_RDLCK => F_WRLCK upgrade logic in the kernel
for conflicting locks.
This is a standalone test to check the traverse_read vs.
allrecord_lock/prepare_commit interaction.
This is based on the example from
https://lists.samba.org/archive/samba-technical/2017-April/119861.html
from Douglas Bagnall <douglas.bagnall@catalyst.net.nz> and Volker Lendecke <vl@samba.org>.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
The current runtime check for robust mutexes in
tdb_runtime_check_for_robust_mutexes() is not thread-safe.
When called in a multi-threaded program where any another thread doesn't
have SIGCHLD blocked, we may end up hung in sigsuspend() waiting for a
SIGCHLD of a child procecss and the signal was delivered to another
thread.
Revert to the previous behaviour of waiting for the child instead of
waiting for the SIGCHLD signal.
Ensure the pid we wait for is not reset to -1 in a toctou race with the
signal handler.
Check whether waitpid() returns ECHILD which can happen if the signal
handler is run by more then one thread in parallel (yes, this can
happen) or if tdb_robust_mutex_wait_for_child() and the signal handler
are racing.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=12593
Pair-programmed-with: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Apr 22 09:16:16 CEST 2017 on sn-devel-144
When a process holds a readlock and wants to upgrade, this needs to be
reflected in the underlying lock. Without this, it is possible to cheat:
One process holds a readlock, and another process wants to write this
record. All the writer has to do is take a readonly lock on the key and
then do the store.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Drop the configure option for --disable-python as it is now
global in wafsamba.
If samba is set to use a system copy of tdb, and tdb wasn't built
with python support, then the system pytevent will not be found. If
samba is being built without python support then pytdb is not needed,
so do not bother to try and find it.
Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
