1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-25 06:04:04 +03:00

643 Commits

Author SHA1 Message Date
Stefan Metzmacher
27ceb1c3ad tdb: version 1.4.7
* Fix build problems

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15071

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Jun  8 17:57:53 UTC 2022 on sn-devel-184
2022-06-08 17:57:53 +00:00
Stefan Metzmacher
22c46d9f41 configure/Makefile: export PYTHONHASHSEED=1 in all 'configure/Makefile' scripts
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2022-03-29 22:32:32 +00:00
Andreas Schneider
7fab6d5373 lib:tdb: Reformat shell scripts
shfmt -f lib/tdb/ | xargs shfmt -w -p -i 0 -fn

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2022-02-24 09:15:34 +00:00
Stefan Metzmacher
1c776e54cf tdb: version 1.4.6
* Use atomic operations for tdb_[increment|get]_seqnum

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Jule Anger <janger@samba.org>
2022-01-24 11:21:32 +00:00
Volker Lendecke
5b526f4533 tdb: Raw performance torture to beat tdb_increment_seqnum
Running this on sn-devel-184 takes ~14 seconds with the atomic
ops. Without them I did not wait for it to finish. After reducing
NPROCS from 500 to 50 it still ran for more than a minute.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Dec 15 01:03:56 UTC 2021 on sn-devel-184
2021-12-15 01:03:56 +00:00
Volker Lendecke
b9f06ab347 tdb: Use atomic operations for tdb_[increment|get]_seqnum
With locking.tdb now based on g_lock.c code, we change locking.tdb a
lot more often. I have a customer case where LDX tortures smbd very
hard with 800+ concurrent connections, which now completely falls over
where 4.12 still worked fine. Some debugging showed a thundering herd
on fcntl locking.tdb index 48 (TDB_SEQNUM_OFS). We still use fcntl for
the seqnum, back when we converted the chainlocks to mutexes we did
not consider it to be a problem. Now it is, but all we need to do with
the SEQNUM is to increment it, so an __atomic_add_fetch() of one is
sufficient.

I've taken a look at the C11 standard atomics, but I could not figure
out how to use them properly, to me they seem more general to be
initialized first etc. All we need is a X86 "lock incl 48(%rax)" to be
emitted, and the gcc __atomic_add_fetch seems to do this.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2021-12-15 00:15:33 +00:00
Joseph Sutton
02b1873033 Fix Python docstrings
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Sep  4 00:55:32 UTC 2021 on sn-devel-184
2021-09-04 00:55:32 +00:00
Stefan Metzmacher
bedeeb0b59 tdb: version 1.4.5
* fix standalone usage of tdb.h

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Jul 20 11:48:38 UTC 2021 on sn-devel-184
2021-07-20 11:48:37 +00:00
Günther Deschner
aacd3ecb45 tdb: Fix invalid syntax in tdb.h
Defining _PUBLIC_ in the same way as in talloc.h resolves an issue with
a previous fix for Solaris Studio compiler 12.4 that prefixed all calls
in tdb.h with _PUBLIC_.  Thanks to Lukas Slebodnik
<lslebodn@redhat.com>.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14762

Guenther

Signed-off-by: Günther Deschner <gd@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2021-07-20 10:57:35 +00:00
Stefan Metzmacher
1ad5df9f74 tdb: version 1.4.4
- Fix a memory leak on error
- python: remove all 'from __future__ import print_function'
- Fix CID 1471761 String not null terminated
- Use hex_byte() in parse_hex()
- Use hex_byte() in read_data()
- fix studio compiler build
- Fix some signed/unsigned comparisons
- also use __has_attribute macro to check for attribute support
- Fix clang 9 missing-field-initializer warnings
- pytdb tests: add test for storev()
- pytdb: add python binding for storev()
- tdbtorture: Use ARRAY_DEL_ELEMENT()
- py3: Remove #define PyInt_FromLong PyLong_FromLong
- py3: Remove #define PyInt_AsLong PyLong_AsLong
- py3: Remove #define PyInt_Check PyLong_Check
- tdb: Align integer types

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2021-07-14 21:06:36 +00:00
Andreas Schneider
252275f3a6 lib:tdb: Fix a memory leak on error
Found by covscan.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2021-07-08 09:30:40 +00:00
Douglas Bagnall
aecb2b779b python: remove all 'from __future__ import print_function'
This made Python 2's print behave like Python 3's print().

In some cases, where we had:

   from __future__ import print_function
   """Intended module documentation..."""

this will have the side effect of making the intended module documentation
work as the actual module documentation (i.e. becoming __doc__), because
it is once again the first statement in the module.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2021-04-28 03:43:34 +00:00
Volker Lendecke
5ef25729bd tdb: Fix CID 1471761 String not null terminated
This is a false positive (in is length 3 initialized to 0), but this
patch does not hurt

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Samuel Cabrero <scabrero@samba.org>
2021-01-14 13:29:35 +00:00
Volker Lendecke
fd05612794 tdb: Use hex_byte() in parse_hex()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2021-01-08 20:31:33 +00:00
Volker Lendecke
ad081bf2dd tdb: Use hex_byte() in read_data()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2021-01-08 20:31:33 +00:00
Björn Jacke
3c1013caf4 tdb: fix studio compiler build
Solaris Studio compiler 12.4 is pedantic about prototypes in headers having
the external visibility declarations too. It throws errors like:

redeclaration must have the same or more restrictive linker scoping: ...

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-11-10 06:53:43 +00:00
Martin Schwenke
cfd34eef58 tdb: Fix some signed/unsigned comparisons
[207/389] Compiling lib/tdb/tools/tdbdump.c
../../../lib/tdb/tools/tdbrestore.c: In function ‘read_linehead’:
../../../lib/tdb/tools/tdbrestore.c:43:13: warning: comparison of integer expressions of different signedness: ‘int’ and ‘long unsigned int’ [-Wsign-compare]
   43 |  for (i=0; i<sizeof(prefix); i++) {
      |             ^
../../../lib/tdb/tools/tdbrestore.c: In function ‘read_data’:
../../../lib/tdb/tools/tdbrestore.c:95:13: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
   95 |  for (i=0; i<size; i++) {
      |             ^

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2020-07-16 01:29:45 +00:00
Björn Jacke
2287083015 tdb: also use __has_attribute macro to check for attribute support
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-05-24 23:55:36 +00:00
Gary Lockyer
13a2f70a4d Fix clang 9 missing-field-initializer warnings
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-05-08 09:31:31 +00:00
Jule Anger
0847f56f24 pytdb tests: add test for storev()
Signed-off-by: Jule Anger <ja@sernet.de>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue May  5 11:24:52 UTC 2020 on sn-devel-184
2020-05-05 11:24:52 +00:00
Jule Anger
4d42497441 pytdb: add python binding for storev()
Signed-off-by: Jule Anger <ja@sernet.de>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2020-05-05 09:40:26 +00:00
Volker Lendecke
f71364bc2d tdbtorture: Use ARRAY_DEL_ELEMENT()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Thu Mar 26 16:22:00 UTC 2020 on sn-devel-184
2020-03-26 16:22:00 +00:00
Andrew Bartlett
5c1867ba45 py3: Remove #define PyInt_FromLong PyLong_FromLong
This allows us to end the use of Python 2/3 compatability macros.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power
2020-03-23 19:12:43 +00:00
Andrew Bartlett
4764e8b4c7 py3: Remove #define PyInt_AsLong PyLong_AsLong
This allows us to end the use of Python 2/3 compatability macros.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2020-03-23 19:12:43 +00:00
Andrew Bartlett
3aea3b1538 py3: Remove #define PyInt_Check PyLong_Check
This will allow us to remove some unused code in the PIDL-generated
python bindings.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2020-03-23 19:12:43 +00:00
Volker Lendecke
4fca8d7aa7 tdb: Align integer types
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Jan 23 20:41:46 UTC 2020 on sn-devel-184
2020-01-23 20:41:46 +00:00
Uri Simchoni
28e4ac0e7a tdb: Release tdb 1.4.3
* Upgrade waf to version 2.0.18 to fix a cross-compilation issue.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13846

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-11-18 10:11:41 +00:00
Björn Jacke
96c7653ee7 lib/tdb/docs/tracing.txt: typo fixes
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-10-31 00:43:36 +00:00
Stefan Metzmacher
60cba7b3a1 tdb: Release tdb 1.4.2
* Build fixes
* Improve the performance by inlining the tdb_oob() checks

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Aug 20 14:45:41 UTC 2019 on sn-devel-184
2019-08-20 14:45:41 +00:00
Volker Lendecke
f5735e2c66 tdb: Inline the common part of tdb_oob
When you set

in tdbtorture.c to make it more similar to locking.tdb use,

bin/tdbtorture -m -n 1 -l 100000 -s

becomes twice as fast. This is a pretty extreme case, but all other
tests that I did improve significantly as well.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-06 21:49:27 +00:00
Volker Lendecke
897bffa816 tdb: Speed up tdb_oob()
This is common between both implementations of tdb_oob(). It's
faster if we don't have to dereference function pointers.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-06 21:49:27 +00:00
Volker Lendecke
5a388453e0 tdb: Introduce tdb_oob()
Initially just encapsulate the pointer dereferences

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-06 21:49:27 +00:00
Volker Lendecke
885ba572ef tdb: Rename tdb_oob() to tdb_notrans_oob()
tdb_oob() will become a public function encapsulating the pointer
dereferences.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-06 21:49:27 +00:00
Volker Lendecke
f4430086fa tdb: Adapt _tdb_transaction_cancel() to README.Coding
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2019-07-03 08:55:23 +00:00
Volker Lendecke
4ef5a42ca2 tdb: Adapt tdb_rescue() to README.Coding
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2019-07-03 08:55:23 +00:00
Gary Lockyer
a77fda0cd4 lib tdb: memcmp ubsan warning
Fix the ubsan warning

lib/tdb/common/tdb.c:184:9: runtime error: null pointer passed as
argument 2, which is declared to never be null"

memcmp call now guarded by a length check.

memcmp returns zero when called with a zero length parameter.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>

Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Mon Jul  1 14:50:54 UTC 2019 on sn-devel-184
2019-07-01 14:50:53 +00:00
Noel Power
9e78f7b53d lib/tdb/common: Fix Array access results in a null pointer dereference
Fixes;

lib/tdb/common/transaction.c:613:7: warning: Array access (via field 'blocks') results in a null pointer dereference <--[clang]
                if (tdb->transaction->blocks[i] != NULL) {
                    ^
1 warning generated.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-07-01 13:34:10 +00:00
Noel Power
848290d37f lib/tdb/common: Fix warning: Null pointer passed as argument to param
Fixes:

lib/tdb/common/rescue.c:299:2: warning: Null pointer passed as an argument to a 'nonnull' parameter <--[clang]
        qsort(found.arr, found.num, sizeof(found.arr[0]), cmp_key);
        ^     ~~~~~~~~~

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-07-01 13:34:10 +00:00
Noel Power
cf43f1d052 clang: Fix Null pointer passed as argument warning
Fixes:
lib/tdb/common/transaction.c:354:2: warning: Null pointer passed as an argument to a 'nonnull' parameter <--[clang]
        memcpy(tdb->transaction->blocks[blk] + off, buf, len);
&

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-06-26 10:30:23 +00:00
Noel Power
7987e4af96 lib/tdb: clang: Fix warning: Dereference of null pointer
Fixes:

lib/tdb/common/lock.c:933:6: warning: Dereference of null pointer <--[clang]
        if (tdb->allrecord_lock.count) {
            ^~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-06-26 10:30:23 +00:00
Andrew Bartlett
75d87b29f1 py3: Remove PyStr_FromFormat() compatability macro
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
2019-06-24 17:24:27 +00:00
Andrew Bartlett
34f9a089d8 py3: Remove PyStr_FromString() compatability macro
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
2019-06-24 17:24:27 +00:00
Noel Power
49e2d36918 lib/tdb/common: clang: Fix 'Value stored to 'last_ptr' is never read'
Fixes

lib/tdb/common/freelistcheck.c:96:3: warning: Value stored to 'last_ptr' is never read <--[clang]
                last_ptr = rec_ptr;
                ^          ~~~~~~~

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer gary@catalyst.net.nz

Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Tue Jun 11 13:31:01 UTC 2019 on sn-devel-184
2019-06-11 13:31:01 +00:00
Volker Lendecke
77e37850fd tdb: Fix typo in README
README is far from a complete, good and accurate document, but what's
in there should at least not have obvious errors.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-05-23 16:56:17 +00:00
Andrew Bartlett
75cad1d79f tdb: Release tdb 1.4.1
* Do not propogate any errors from tdb_repack() to to tdb_transaction_commit()

  The repack may fail due to lock ordering or memory limits, but the original data has
  been written so the call must succeed.  (bug 13952)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri May 17 08:21:52 UTC 2019 on sn-devel-184
2019-05-17 08:21:52 +00:00
Andrew Bartlett
444b594fd1 tdb: Do not return errors from tdb_repack() in the tail of tdb_transaction_commit()
The call to tdb_repack() inside tdb_transaction_commit()
is an optimization, not part of the transaction itself,
so failing due to lock or other errors isn't a fatal error
that should cause the caller to think the transaction was
a failure by returning -1.

The tdb transaction itself has finished and been committed
onto stable storage via fsync and all locks released at the
point tdb_repack() is called.

tdb_repack() is only called here as it's a convenient point
to attempt to reduce tdb fragmentation without having to add
a timer call to repack in all users of tdb.

This causes lock ordering issues in Samba, showing up as:

ldb: ltdb: tdb(../private/sam.ldb.d/DC=SAMBA2008R2,DC=EXAMPLE,DC=COM.ldb): tdb_transaction_prepare_commit: failed to upgrade hash locks: Locking error

This is because Samba has multiple tdb databases open, and the lock order between them
is important.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13952

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-05-17 06:48:10 +00:00
Noel Power
5720f20a19 lib/tdb: squash 'cast between incompatible function types' warning
squash 'cast between incompatible function types' warning

To avoid warning above produced by using
-Wcast-function-type we;

  + ensure PyCFunctions of type METH_NOARGS defined dummy arg
  + ensure PyCFunctions of type METH_KEYWORDS use PY_DISCARD_FUNC_SIG
    macro

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-05-16 17:55:17 +00:00
Douglas Bagnall
9fd3939a71 tdbtool: avoid theoretical NULL dereference (CID 1361462)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-09 22:39:27 +00:00
Andrew Bartlett
fe69d807ea tdb: Release tdb 1.4.0
* Remove Python 2.x support except to build just the bare C library
 * Remove --extra-python (a build time mode to produce Python2 and Python3
   bindings at the same time)
 * New minor version to allow Samba 4.10 to release a tdb if
   required from that branch

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-03-21 04:06:16 +00:00
Andrew Bartlett
a574e8f517 build: Standardise on calling conf.SAMBA_CHECK_PYTHON() in libraries
We do this by removing the confusing mandatory option to
conf.SAMBA_CHECK_PYTHON{,_HEADERS}(), instead just use the value of
--disable-python internally

This follows the default minimum of Python 3.4 and keeps things consistent
with the main Samba build where --disable-python is required to skip building
python bindings.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-03-21 04:06:16 +00:00