1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-23 17:34:34 +03:00
Commit Graph

6592 Commits

Author SHA1 Message Date
Volker Lendecke
74c7e3d19a audit_logging: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-11-02 21:21:16 +01:00
Douglas Bagnall
78ead841bf ldb/tests/py/index: remove dup value in dictionary
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:11 +01:00
Douglas Bagnall
fc64f2586d ldb/tests/py/api: use proper name for ldb.LdbError
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:11 +01:00
Volker Lendecke
500a729a55 tdb: Make record deletion circular-chain safe
Before this patch we had 3 loops walking a hash chain to delete
records:

tdb_do_delete() to find the predecessor of the record that was to be
deleted. tdb_count_dead(), the name says it all and tdb_purge_dead()
to give back all dead records from a chain to the freelist.

This patch introduces tdb_trim_dead that walks a hash chain just
once. While it does so it counts the number of dead records, and all
records beyond tdb->max_dead_records are moved to the freelist.

Normal record deletion now works by always marking a record as dead in
step 1 and then calling tdb_trim_dead. This is made safe against
circular chains by doing the slow chain walk only in the case when we
did not delete a dead record during our walk.

It changes our dynamics a bit:

When deleting a record with non-zero max_dead_records, now we always
leave that number of records around when deleting, doing a blocking
lock on the freelist when we found too many dead records.

Previously when exceeding max_dead_records we wiped all dead records
to start accumulating them from scratch, assuming we could lock the
freelist in a nonblocking fashion.

The net effect for an uncontended freelist is the same: In
tdb_allocate() we still completely hand over all dead records to the
freelist when we could lock it, it just happens later than without
this patch.

This means for a lightly loaded system we will potentially leave more
dead records around in databases like locking.tdb. However, on a
heavily loaded system we become more predictable: If the freelist is
so heavily contended that across many deletes we can't get hold of it,
previously we accumulated more dead records than max_dead_records
would allow. This is a really lowlevel tradeoff that is likely hard to
measure, but to me becoming more deterministic without sacrificing too
much parallelism (we keep more dead records around) is worth trying.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Oct 30 02:48:38 CET 2018 on sn-devel-144
2018-10-30 02:48:38 +01:00
Volker Lendecke
05212658ba tdb: Do early RDONLY error check for tdb_delete
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-29 23:36:25 +01:00
Volker Lendecke
4ed2a67a59 tdb: Purge dead records whenever we block the freelist
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-29 23:36:25 +01:00
Volker Lendecke
96f6768e05 tdb: Don't delete dead records in traverse
The next commit will change the handling of dead records, removing the
"tdb_do_delete" function. As traverses should not happen in normal
operations, dead records from them should be rare, and relying on
traverses to remove them is a very bad idea IMHO.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-29 23:36:24 +01:00
Volker Lendecke
c18e2ed00f tdb: Align an integer type
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-29 23:36:24 +01:00
Volker Lendecke
14abead3ca tdb: Fix a typo
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-29 23:36:24 +01:00
David Mulder
5391e21dea lib:socket: If returning early, set ifaces
Prevents a segfault in load_interfaces() when total interfaces == 1.
Fixes regression caused by da68a1b2f4.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13665

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Ralph Böhme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sun Oct 28 00:35:35 CEST 2018 on sn-devel-144
2018-10-28 00:35:34 +02:00
Douglas Bagnall
d2fadfd6e4 ldb/tests/py/api: reveal shadowed casefold test
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
2018-10-25 21:45:59 +02:00
Douglas Bagnall
87272cf17e tdb/test/_tdbtext: remove unused imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
2018-10-25 21:45:55 +02:00
Douglas Bagnall
02ed259942 talloc/test/pytalloc: remove unused imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
2018-10-25 21:45:55 +02:00
Douglas Bagnall
eef11c4479 ldb/tests/_ldb_text: remove unused imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
2018-10-25 21:45:55 +02:00
Douglas Bagnall
6b88685f6b ldb-samba/tests/match-rules: remove unused imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
2018-10-25 21:45:55 +02:00
Volker Lendecke
e2d9abd421 tdbtorture: Align integer types
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2018-10-25 17:58:24 +02:00
Volker Lendecke
afc616b67b tdbtorture: No transaction with mutexes
Right now we don't do transactions with mutexed tdbs. tdbtorture -m
locks up. I haven't really investigated why that is the case. The lockup
confused me quite a bit until I figured out it works fine as long as it
does not do transactions, which is all we need right now.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2018-10-25 17:58:24 +02:00
Volker Lendecke
8df11518c0 tdb: Remove "USE_RIGHT_MERGES" code
This has not been activated by default for ages and can be very
inefficient. With check_merge_with_left_record() we have an
alternative that will merge freelist records while we walk it
anyway. This has reduced fragmentation significantly

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2018-10-25 17:58:23 +02:00
Volker Lendecke
a895cc2a59 tdb: Fix a typo
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2018-10-25 17:58:23 +02:00
Volker Lendecke
c37bf2f938 tdb: Use explicit initialization
Let the compiler figure out the optimal code

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2018-10-25 17:58:23 +02:00
Volker Lendecke
ec209d28a2 tdb: Avoid casts
We have %PRIu32 and %zu these days

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2018-10-25 17:58:22 +02:00
David Disseldorp
9e03fb60dc talloc: deprecate talloc_set_memlimit()
The memlimit functionality was never utilized by Samba. It adds unneeded
complexity, so flag it as deprecated.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-19 23:11:26 +02:00
Douglas Bagnall
a0bad1364a ldb_ldif: avoid strlen(NULL)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Oct 19 03:43:58 CEST 2018 on sn-devel-144
2018-10-19 03:43:58 +02:00
Douglas Bagnall
ce518f387f ldb_ldif: be less horribly efficient in debugging
perf said all the time was in strlen.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Oct 18 13:17:30 CEST 2018 on sn-devel-144
2018-10-18 13:17:30 +02:00
Andreas Schneider
84615c1982 replace: Add memset_s() if not available
See https://en.cppreference.com/w/c/string/byte/memset

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-10-16 08:42:18 +02:00
Volker Lendecke
37f6e2a351 lib: Remove lib/crypto/crc32.[ch]
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Oct 10 01:41:52 CEST 2018 on sn-devel-144
2018-10-10 01:41:52 +02:00
Volker Lendecke
e7d3678ddb lib: Avoid the use of open_memstream in tevent_req_profile_string
Solaris does not have it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13629
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-08 22:17:11 +02:00
Volker Lendecke
f36a538cbd tdb: Basic test for circular freelist fix
Try to store a record for which the (circular) freelist does not have
any entry.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-08 22:17:11 +02:00
Volker Lendecke
bdacc41f46 tdb: Basic test for circular hash chain fix
This just walks tdb_find by searching for a nonexistent record

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-08 22:17:11 +02:00
Volker Lendecke
7e1ad4c588 tdb: Make the freelist walk circular-safe
We can't really do the full check while the freelist is modified on the
fly. As long as we don't merge any freelist entries, we should be good
to apply this check.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-08 22:17:11 +02:00
Volker Lendecke
75e79ca548 tdb: Align integer types
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-08 22:17:10 +02:00
Volker Lendecke
ade339c8c3 tdb: Make get_hash_length circular-safe
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-08 22:17:10 +02:00
Volker Lendecke
e02c4a417e tdb: Make tdb_find_dead circular-safe
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-08 22:17:10 +02:00
Volker Lendecke
6502f7a3d2 tdb: Make tdb_dump_chain circular-list safe
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-08 22:17:10 +02:00
Volker Lendecke
e63f7bd3a9 tdb: Make tdb_find circular-safe
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-08 22:17:10 +02:00
Volker Lendecke
b83763d175 tdb: Add tdb_chainwalk_check
This captures the tdb_rescue protection against circular hash chains
with a slow pointer updated only on every other record traverse

If a hash chain has a loop, eventually the next_ptr
will cycle around and be identical to the 'slow' pointer.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-08 22:17:10 +02:00
Noel Power
cddd54e865 lib/ldb: Ensure ldb.Dn can accept utf8 encoded unicode
Additionally remove the associated known fail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13616
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-27 01:54:26 +02:00
Noel Power
d1492ab919 lib/ldb/tests: add test for ldb.Dn passed utf8 unicode
object dn format should be a utf8 encoded string
Note: Currently this fails in python2 as the c python binding for
      the dn string param uses PyArg_ParseTupleAndKeywords() with 's'
      format, this will accept str *or* unicode in the default encoding.
      The default encoding in python2 is... ascii.

Also adding here a knownfail to squash the error produced by the test.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-27 01:54:26 +02:00
Noel Power
d786e1fca9 lib/ldb: Test correct variable for no mem condition
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-27 01:54:26 +02:00
Philipp Gesang
c98f997192 turn --with-json-audit into global --with-json
Fold the build option --with-json-audit into the toplevel wscript
to reflect the fact that JSON support is no longer local to the
audit subsystem.

Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-26 20:41:07 +02:00
Philipp Gesang
d50aafe932 lib/audit_logging: make json_{is_invalid,to_string}() accept a const*
Allow for json_is_invalid() and json_to_string() to be used on a
const pointer. Neither function requires for the json object to
be mutable so constraining them to non-const* is unnecessary.

Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-22 06:05:06 +02:00
Amitay Isaacs
3b94d31d24 ldb: Fix dependency of ldb module
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13626
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-21 20:04:22 +02:00
Noel Power
6c62e22c1c PY3: detect talloc_utils (for version of python running the build)
It seems all combinations have potential to fail here :/ however for
the moment making the assumption
  *  if python2 driving the build you are building against libpython2 (with the option of having python3 as 'extra-python'
  *  if python3 driving the build you are building against libpython3 (with the option of having python3 as 'extra-python' NOTE: this isn't inforced it probably should be !!!! In a system with python2 and python3 installed it seems waf needs PYTHON env variable to decide which libpython to build against. It's also an option that configure should use that too (to figure out which talloc_util to use or build)

However.... right now I just want the thing to build

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-16 06:16:20 +02:00
Noel Power
693834f731 PY3: fix iteritems usage, not supported in python3
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-16 06:16:19 +02:00
Christof Schmitt
18ab3ea7bc torture: Remove unnecesssary #include <fcntl.h>
This is now provided through system/filesys.h.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-15 15:18:25 +02:00
Volker Lendecke
92b01adf09 tevent: Fix a docu typo
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-09-14 07:49:13 +02:00
Volker Lendecke
b6b7c15758 dbwrap_tdb: Avoid double-call to talloc_get_type_abort
We've already retrieved "ctx" in the callers of db_tdb_fetch_locked_internal().

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-09-14 07:49:13 +02:00
Ralph Boehme
382620d42d dbwrap_tdb: use struct initializer
This ensures all struct members are implicitly initialized.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-09-12 22:25:19 +02:00
Ralph Boehme
9a9620bb42 dbwrap_tdb: move a function call out of an if condition
At least for me this improves readability somewhat. No change in
behaviour.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-09-12 22:25:19 +02:00
Stefan Metzmacher
200ddf8e73 Revert "lib/replace: force to unset defines if check has failed"
This reverts commit 8ec7eb0c01.

There was already a better fix under discusion.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-10 19:28:14 +02:00