1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-24 21:34:56 +03:00
Commit Graph

6288 Commits

Author SHA1 Message Date
Volker Lendecke
cd5d726c89 util_tdb: Make a few functions static
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-11-13 23:54:46 +01:00
Stefan Metzmacher
2e573eead9 tevent: version 0.9.34
* Remove unused select backend
* Fix a race condition in tevent_threaded_schedule_immediate()
  (bug #13130)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Mon Nov 13 18:02:46 CET 2017 on sn-devel-144
2017-11-13 18:02:46 +01:00
Volker Lendecke
20cfcb7dbc tevent: Fix a race condition
We can't rely on tctx to exist after we unlocked the mutex. It took a
while, but this does lead to data corruption. If *tctx is replaced with
something where tctx->wakeup_fd points to a real, existing file
descriptor, we're screwed. And by screwed, this means file corruption
on disk.

Again. I am not tall enough for this business.

http://bholley.net/blog/2015/must-be-this-tall-to-write-multi-threaded-code.html

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13130

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Nov 11 03:20:09 CET 2017 on sn-devel-144
2017-11-11 03:20:09 +01:00
Volker Lendecke
7e4908d72b lib: Use all_zero where appropriate
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-10-27 20:33:25 +02:00
Andrew Bartlett
593dacd274 replace: Link to -lbsd when building replace.c by hand
This ensures that we correctly detect HAVE_IFACE_GETIFADDRS
et al, which are based on a "build the source" style test.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13087

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-10-20 04:05:20 +02:00
Douglas Bagnall
42e7671226 nwrap: Fix strotoul checks for NSS_WRAPPER_MAX_HOSTENTS
The env and endptr pointers need to be dereferenced, but that is not
enough: we don't really want to regard an empty string (*env == '\0')
as a valid number.

Found by GCC 8.0.0 20170705 (experimental).

[2095/4103] Compiling lib/nss_wrapper/nss_wrapper.c
../lib/nss_wrapper/nss_wrapper.c: In function "nwrap_init":
../lib/nss_wrapper/nss_wrapper.c:1571:13: warning: comparison between pointer and zero character constant [-Wpointer-compare]
   if (((env != '\0') && (endptr == '\0')) ||
                ^~
                ../lib/nss_wrapper/nss_wrapper.c:1571:9: note: did you mean to dereference the pointer?
   if (((env != '\0') && (endptr == '\0')) ||
            ^
            ../lib/nss_wrapper/nss_wrapper.c:1571:33: warning: comparison between pointer and zero character constant [-Wpointer-compare]
   if (((env != '\0') && (endptr == '\0')) ||
                                    ^~
                                    ../lib/nss_wrapper/nss_wrapper.c:1571:26: note: did you mean to dereference the pointer?
   if (((env != '\0') && (endptr == '\0')) ||

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Thu Oct 19 16:42:17 CEST 2017 on sn-devel-144
2017-10-19 16:42:17 +02:00
Gary Lockyer
123042c2e3 source4/smbd: add a prefork process model.
Add a pre fork process model to bound the number processes forked by
samba.  Currently workers are only pre-forked for the ldap server,  all
the other services have pre-fork support disabled.

When pre-fork support is disabled a new process is started for each
service, and requests are processed by that process.

This commit partially reverts commit
b5be45c453.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-19 05:33:10 +02:00
Christof Schmitt
267cd25290 Removed unused 'oplock contention limit' config parameter
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-10-19 00:55:24 +02:00
Lumir Balhar
01e89ada25 python: Fix Python 2.6 compatibility
PyErr_NewExceptionWithDoc() isn't available in Python 2.6 so it can
be used only in higher versions of Python.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-10-18 10:20:26 +02:00
Stefan Metzmacher
96e471eecc krb5_wrap: ADDRTYPE_INET6 is available in all supported MIT versions
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13079

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Björn Jacke <bjacke@samba.org>
2017-10-11 08:30:13 +02:00
Stefan Metzmacher
7014684127 krb5_wrap: KRB5_ADDRESS_INET6 is not a define in Heimdal
All supported versions of Heimal already have KRB5_ADDRESS_INET6,
so there's no need for an explicit check.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13079

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Björn Jacke <bjacke@samba.org>
2017-10-11 08:30:13 +02:00
Ralph Boehme
6539cc8a24 lib/util/run_cmd: use a cleanup function instead of a destructor
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13062

Pair-programmed-with: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Sat Sep 30 12:14:57 CEST 2017 on sn-devel-144
2017-09-30 12:14:56 +02:00
Ralph Boehme
94a8331e54 lib/util/run_cmd: remove a printf
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13062

Pair-programmed-with: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
2017-09-30 08:01:25 +02:00
Ralph Boehme
4aaf072d1f lib/util/run_cmd: ensure fd_stdin gets set to -1 in the destructor
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13062

Pair-programmed-with: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
2017-09-30 08:01:24 +02:00
Ralph Boehme
9a8eeabd95 lib/util/run_cmd: prevent zombies in samba_runcmd_send on timeout
Ensure the state desctructor calls tfork_destroy to reap the waiter and
worker processes. Otherwise we leave the waiter process as a zombie
behind us as we never call waitpid on it in case of a timeout
or talloc_free() from the caller.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13062

Pair-programmed-with: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
2017-09-30 08:01:24 +02:00
Volker Lendecke
ac521c2d7d lib: Fix a signed/unsigned hickup
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-09-30 08:01:24 +02:00
Christof Schmitt
575f1e2bf5 lib/util: Fix build for lttng libraries that does not support tracef
On a system that has older lttng-ust development headers installed that
do not yet provide the tracef api, configure fails with:

Checking for library lttng-ust : yes
ERROR: Target 'lttng-ust' in directory /samba/lib/util re-defined as EMPTY - was SYSLIB

Fix the initialization order in waf, to not redefine the lttng-ust
target

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-09-27 22:11:11 +02:00
Gary Lockyer
3982b774f4 tfork: set waiter process title
Display a meaningful process title for the waiter process, now that smbd is
calling setproctitle_init() and this value will be displayed in ps
output.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Sep 26 04:38:59 CEST 2017 on sn-devel-144
2017-09-26 04:38:59 +02:00
Gary Lockyer
4f84b6dfa8 s4:smbd: call setproctitle_init
Call setproctitle_init() in main which suppresses the
  "samba: setproctitle not initialized, please either call
   setproctitle_init() or link against libbsd-ctor."
messages, but more importantly it displays meaningful details in ps
output.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9816

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-26 00:41:16 +02:00
Andrew Bartlett
c40531d9b9 ldb: Release ldb 1.3.0
* GUID Index support.

  NOTE: When activated by setting @IDXGUID in the @INDEXLIST dn, all
  entries in the DB are re-keyed in a way that is NOT visible to
  ldb 1.2.2 and earlier.  To re-key back to the previous format, remove
  the @IDXGUID attribute from @INDEXLIST using ldb 1.2.2 or later.

  (ldb 1.2.2 can re-key, but not otherwise read, the new DB format).

* Give LDB_ERR_CONSTRAINT_VIOLATION, not LDB_ERR_ENTRY_ALREADY_EXISTS
  when a duplicate value is detected in a unique index

* Print status information during a > 10,000 entry re-index
  (as this can be slow)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Sep 23 01:24:19 CEST 2017 on sn-devel-144
2017-09-23 01:24:19 +02:00
Andrew Bartlett
303739e801 ldb_tdb: Treat distinguishedName and objectGUID (in Samba) as unique
This avoids loading any second index for these cases.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-22 21:20:25 +02:00
Andrew Bartlett
644b3ffb14 ldb: Ensure we do not run out of File descriptors in autobuild
The python TestCase API will keep a reference to the test object until the end
of the tests, long after we need the actual LDB or the fd.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-22 21:20:25 +02:00
Andrew Bartlett
a0b46ecdd2 ldb: Do not make an ldb file for API tests
An ldb context is valid without a backing file for tests of ldb.Message and ldb.MessageElement

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-22 21:20:25 +02:00
Andrew Bartlett
8565ca1c6d ldb: Add test showing a search against the index is not possible
This is not actually a great test, as the filter would
fail to match these anyway, but it at least checks the
codepath is safe.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:25 +02:00
Andrew Bartlett
f0624d3890 ldb_tdb: Also ban a (indexed) search against like @IDXDN=foo
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:25 +02:00
Andrew Bartlett
f4f9acf781 ldb_tdb: Update comments for the delete_index() pass of ltdb_reindex()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:25 +02:00
Andrew Bartlett
61b66b8d0a ldb_tdb: Print progress messages on re-index
A re-index of 10,000 entries is slow enough and rare enought that we can
justify the message being at LDB_DEBUG_WARNING as otherwise the administrator
will be sure the "lockup" was one.

The default for ldb is to print LDB_DEBUG_WARNING in comand-line tools
and the default for Samba is to log it at level 2.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
c71ddab974 ldb_tdb: Remove incorrect early return from re-index
The ltdb->cache->attribute_indexes test is not correct with the GUID index mode
so for consistency remove it.  This will make re-index on a large un-indexed
database slower, but that is better than making the wrong choice on a large
GUID-indexed database.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
0503f0984a ldb: Add more tests covering in-transaction GUID index and unique index behaviour
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
00410ba99f ldb_tdb: Add UNIQUE_INDEX as a possible flag
This allows easy testing of our unique index code and behaivour from python

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
561ef20634 ldb_tdb: Remove LTDB_FLAG_HIDDEN and ignore "HIDDEN" in @ATTRIBUTES
This was (unintentionally) disabled by
6ef6182554 in 2006.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
29c303f21b ldb_tdb: Clean up index records on ltdb_index_add_new() failure.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
83bc607463 ldb_tdb: Describe index format and control points
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
3113d871bd ldb_tdb: Give a good error message on add without an objectGUID
(or whatever the @IDX_GUID value is)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
b9d561b023 ldb_tdb: Avoid canonicalise and base64 work for DN values, these are already OK
This is important with the GUID index, as a DN lookup is much more common now.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
3e60f02753 ldb_tdb: Clean up list in after use in ltdb_key_dn_from_idx()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
d198a43b55 ldb_tdb: Avoid allocation of a DN between the GUID index and the DB lookup
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
4e9978506a ldb_tdb: Move constants into ldb_tdb.h
This helps ensure we keep these all in sync.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
b0a6b5ab51 ldb_tdb: Optimise ltdb_search_and_return_base() to re-use casefolding
The casefolding of a DN is one of the more expensive and pointless things in LDB
operation.  The ldb_dn abstraction works hard to avoid duplicating this work, but
we can work harder to save that information.

Here we copy in the DN, that has been casefolded already for the index,
and keep that as the returned DN, after stripping any extended components.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
f14370d471 ldb: Add tests for base DN of a different case
This ensures we cover the case where the DN does not match the DB exactly

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
dfe85ecaca ldb_tdb: Use the DN extracted from the DB to filter the message later
This should ensure that the upper or lower case the user chooses does not impact
on the filtering, at least for database that have checkBaseOnSearch set.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
23e0a553d4 ldb_tdb: Add tests for add/modify with the GUID index
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
24c71ed211 ldb: Also test the new GUID index mode
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
64b4fbc613 ldb: Add an unused objectGUID to each record in SearchTests
This will then be used by the GUID index tests.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
a2ada5844d ldb_tdb: Duplicate values are no longer permitted in the index
By removing the qsort() we avoid work.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
3c6977b17c ldb_tdb: Reduce memory consumption in list_intersect()
We will never have more results than is in either list or list2.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
97b026a73f ldb_tdb: Use the binary search more efficiently in list_intersect()
This change ensures we walk the short list and look up into the longer of the two lists.

ltdb_dn_list_find_val() will do a binary search for the GUID case.

Before GUID indexes this was O(n*m), now it is O(n*log(m)).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
b6bf7e7b0b ldb_tdb: Use a binary search to speed up ltdb_dn_list_find_val()
This only works if we have the GUID index format, as otherwise these are unsorted.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
fdff9a7087 ldb_tdb: Rework list_union to not return duplicates, and keep sort order
This allows the binary search to still operate on the list, even after
a or operator in the search expression

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
b86a46df1d ldb_tdb: Sort inputs to list_union()
This allows us to merge the lists finding common values.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
1390e55bb5 ldb_tdb: sort GUID index list at add time by inserting in sorted order
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
9c9d9ef69e binsearch.h: Re-licence under LGPLv3 per agreement of the copyright holders
Documented in mails to contributing@samba.org.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
cf8537068f ldb_tdb: Change error code on unique index violation
The LDB_ERR_ENTRY_ALREADY_EXISTS error code is detected in repl_meta_data as indicating
that the DN exists, and that a conflict record should be created.

This is really a constraint violation, not a duplicate record.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
e8fb45125e ldb_tdb: Re-add of both existing DN and GUID must gives LDB_ERR_ENTRY_ALREADY_EXISTS
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
e16c8aa5da ldb_tdb: Add unique index checking for @IDXDN
This will give us errors earlier if the index code becomes broken

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
b97d556731 ldb_tdb: Improve debugging in ltdb_modify_index_dn() on casefold failure
This is unlikely, but when it happens it will be really painful to debug.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
1f42ded463 ldb_tdb: Add improved error strings on ltdb_key_dn_from_idx() failure
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
50f36e4fb5 ldb_tdb: Read from @INDEXLIST or an override if we are using a GUID index
This allows all the previous patches to be enabled.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
2f8a8c765f ldb_tdb: Optionally use GUID index values a direct TDB keys
This connects the GUID based index records to GUID based TDB keys.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:23 +02:00
Andrew Bartlett
93b18984bd ldb_tdb: Trust the BASE and ONELEVEL index
This avoids re-checking the fetched DN against the scope

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
e4562e58f9 ldb_tdb: Add ltdb_idx_to_key() and use it in ltdb_index_filter()
This will allow a common point to parse index records into a TDB key,
allowing them to be a GUID or DN in the future

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
ba762fac81 ldb_tdb: Do not add an index for GUID_index_attribute
This would be pointless and we no longer query for it.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
c4f35462e9 ldb_tdb: Do not query an index on the GUID_index_attribute
The objectGUID (or similar) is already the record key, there is
no need to index it to itself.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
6008382191 ldb_tdb: Optionally use GUID index in ltdb_search_dn1()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
fa44c5262b ldb_tdb: Use the objectGUID (or similar) as the TDB key in ltdb_key_msg()
When we have the full ldb_message we can read the objectGUID as the TDB key

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
f253dcf8b9 ldb_tdb: Use ltdb_key_msg() in ltdb_delete_noindex()
This allows the optional use of GUID based TDB key.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
6ad4bdc9de ldb_tdb: Add mem_ctx to ltdb_key_dn() and ltdb_key_msg()
This follows modern Samba coding style where memory
returned is allocated on a supplied memory context.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
fb77ad4244 ldb_tdb: Check version number on index
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
e394b9a856 ldb_tdb: Add an index shortcut for a <GUID= DN
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
ccb9443664 ldb_tdb: Add a function to get the GUID key for a DN
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
f26d1a8c5d ldb_tdb: Add a function to take a GUID and make the TDB_DATA key
This allows us to format the TDB key as DN=GUID=f7c953ee-cf9c-433f-b423-21ce04d09591
and so be compatible with an un-indexed search and a re-index with an old ldb.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
aa68957f7f ldb_tdb: Load the syntax of the GUID index attr during ltdb_cache_load()
This allows us to use the ldif_write function later to create a string GUID for the TDB key.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
8555f6ccc2 ldb_tdb: add control points for the new GUID index mode
The @IDXGUID attribute in the @INDEXLIST will be objectGUID
in Samba.

The @IDX_DN_GUID attribute in the @INDEXLIST will be GUID
in Samba.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
3b0698e0b6 ldb_tdb: Do not directly assign DN into the index result list
With the GUID index option, the values in the index result list may
not be a DN but the objectGUID.  We look up the @IDXDN index with the
case-folded DN to get that if required.

We re-use the code from the SCOPE_BASE search do avoid duplication
and for reliablity.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
19ee0438ff ldb_tdb: Pass ltdb_private to ltdb_dn_list_load()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
2bac00a7b9 ldb_tdb: Give LDB_ERR_CONSTRAINT_VIOLATION when a duplicate GUID index is detected
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
06e67c5894 ldb_tdb: Give LDB_ERR_ENTRY_ALREADY_EXISTS when a duplicate DN index is detected
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
ec30439c43 ldb_tdb: Add/remove a GUID index of the DN during ltdb_index_add_all()/ltdb_index_delete()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
98e82113ed ldb_tdb: Split ltdb_index_onelevel() into a helper function
This will allow the code to be re-used for storing the DN->GUID index

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
b2aff12366 ldb_tdb: Prepare to handle rename with GUID index by using ltdb_search_base()
This will allow use of a GUID TDB key in the future.  While ltdb_search_base()
might be marginally slower than tdb_exists(), no allocation is done for the
attributes or DN, and renmaes are not a very common operation.

This allows a check if the target DN exists even when the direct DN -> key
link is broken.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
c24df8e431 ldb_tdb: Implement ltdb_search_base() for a GUID index
The GUID index case can not directly use ltdb_key_dn() and tdb_exists() to
show that a records exists.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
754329a9d9 ldb_tdb: Optionally store a GUID as the index record
This allows, when enabled, the index record to contain (say) the objectGUID, not the DN
of the record.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
b154acb0c7 ldb_tdb: Optionally use a GUID index key in ltdb_dn_list_find_msg()
This function is used to find an existing index value and this
change allows it to find the value by GUID rather than by DN once
the GUID index is enabled.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
9d9ae1c005 ldb_tdb: Pass the full ldb_message to ldb index funtions
This allows the objectGUID, rather than the DN, to be the index key

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
4c01ccb47e ldb_tdb: Delete a successful tdb_store on index add fail in ltdb_add_internal()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
e8cdacc509 ldb_tdb: modify ltdb_delete_noindex() to take a struct ldb_message
This will make it easier to delete records with the GUID TDB key

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
de641db8b7 ldb_tdb: Write GUID index values as version 3
Nothing reads these currently, but we should refuse to load a mixed up index
in the future

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
91e6028c08 ldb_tdb: Refuse to load a GUID index that is not a multiple of 16 bytes
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
ad26b4e292 ldb_tdb: Read GUID index values as one packed LDB attribute
This packing should be more efficient to read than the ldb_pack format.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
f6e953d316 ldb_tdb: Store GUID index values in one packed ldb attribute
This should make them more memory efficient

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
367b7ab748 ldb_tdb: Move constants into ldb_tdb.h
This helps ensure we keep these all in sync.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
c17404826e ldb_tdb: replace strange dn_list_cmp() in index code
This replaces dn_list_cmp() with functions that do not attempt to
to care about string termination.  All index values are case sensitive
and correctly length-bound already, even for a DN index
so just use a length check and memcmp()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
a44d3aedba ldb_tdb: Do not allow a modification of the GUID_index_attribute (objectGUID)
This would totally break our index scheme if this could be modified.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
26dd6f6365 ldb_tdb: Add GUID_index_attribute to ltdb_private
This will be used to determine if we are in GUID index mode

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
3d952157d7 ldb_tdb: Provide struct ltdb_private to index routines
This will make it easier to switch the GUID index mode on and off

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
dded01598e ldb_tdb: Use a more complete error mapping in ltdb_search_key()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
6db8095835 ldb_tdb: Add ltdb_search_key()
This allows us to slowly split out the tdb key in the DB from being the DN

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
d4c1a600b1 ldb_tdb: Use ltdb_key_msg() in re_index()
This will allow changing to a GUID tdb key in the future

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
1b310ad99c ldb_tdb: provide ldb_key_dn() and ldb_key_msg()
This will in time allow us to generate a TDB key from
the msg, eg from an objectGUID.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
cd8ecb3332 ldb_tdb: Add helper function ltdb_search_and_return_base()
This avoids an extra DB lookup for the base, when that is the only
record we will return, and avoids going into the index code for
a base search, as that won't work for special DNs once the GUID
index mode is enabled.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
d8a22884c1 ldb_tdb: Make ldb_match_message() available to ldb_tdb
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
6c28abc249 ldb: Release 1.2.3
* Bug #13033 LDB open with LDB_FLG_RDONLY can cause the database
   to fail to open

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13033

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-20 02:25:30 +02:00
Gary Lockyer
f5f3657c48 ldb: Add tests for read only behaviour
As the kernel is no longer enforcing the read-only DB
add some tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13033

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-20 02:25:30 +02:00
Andrew Bartlett
22854f9b8d ldb_tdb: Change ltdb_connect() NOT to request a kernel-level read only TDB
We support opening and LDB multiple times in a process, but do not support this in tdb.

As we can open the ldb with different flags, we must ensure a later read-write
open is possible.

Additionally, a read-only TDB will refuse the all-record lock, preventing
the ldb from even loading.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13033

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-20 02:25:30 +02:00
Andrew Bartlett
13777d35d0 ldb_tdb: Give a debug message as well as setting the error string if prepare_commit() fails
This is a serious condition, and should be logged.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13033

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-20 02:25:30 +02:00
Andrew Bartlett
75e88e40a2 ldb_tdb: Map TDB error codes into LDB error codes in ltdb_lock_read()
The ltdb_lock_read() routine did not return an LDB error code, but -1.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13033

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-20 02:25:30 +02:00
Gary Lockyer
563bbb9c24 util_runcmd: Free the fde in event handler.
Free the fde in the event handler to prevent the event triggering again
While not strictly necessary in this case, this code serves as an
example of the usage of tfork.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13037

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Sat Sep 16 23:50:27 CEST 2017 on sn-devel-144
2017-09-16 23:50:27 +02:00
Ralph Boehme
6c36ea0737 lib/util: only close the event_fd in tfork if the caller didn't call tfork_event_fd()
Make closing of the event_fd the global responsibility of the
parent process if it called tfork_event_fd().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13037

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-09-16 19:53:23 +02:00
Gary Lockyer
f6a40ff2a1 util/tfork: Write to the status pipe
The previous design relied on only calling close() of the status pipe.

We now write a single 0 byte to the status FD as well as closing it in the
parent process.  Both of these operations typically trigger a read
event on the other end of the FD, held in the waiter process (the child).

The child process blocks on the status FD, until it becomes readable.

However if there is a sibling process that was launched after the waiter
process they also will hold the status FD open and the status FD would,
until this change, never become readable to the waiter process (the child).

This caused the waiter process (child) not to exit and the parent process
to hang in tfork_status() while expecting the waitpid() to return.

That is, file descriptors are essentially global variables copied
to children in the process tree.  The last child that (unwittingly) holds
the file descriptor open is the one that needs to trigger the close() this
code previously depended on.

Without this change, there is no notification of process death until
all these unrelated children exit for their own reasons.

We can write up to 4K (PIPE_BUF) into this pipe before blocking,
but we only write one byte.  Additionally sys_write() refuses to block.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13037

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-09-16 19:53:23 +02:00
Gary Lockyer
28edf7012b tests util/tfork: Tests for status and event fd
Add tests to ensure that:
- The event_fd becomes readable once the worker process has terminated
- That the event_fd is not closed by the tfork code.
  - If this is done in tevent code and the event fde has not been
    freed, "Bad talloc magic value - " errors can result.
- That the status call does not block if the parent process launches
  more than one child process.
  - The status file descriptor for a child is passed to the
    subsequent children.  These processes hold the FD open, so that
    closing the fd does not make the read end go readable, and the
    process calling status blocks.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13037

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-09-16 19:53:22 +02:00
Volker Lendecke
d1d298bb17 lib: util_tdb.h needs tdb.h
It uses TDB_DATA

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-09-16 08:36:16 +02:00
Jeremy Allison
2a003b1a57 lib: tevent: Remove select backend.
select() is no longer useful on modern systems.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sat Sep 16 08:35:39 CEST 2017 on sn-devel-144
2017-09-16 08:35:39 +02:00
Stefan Metzmacher
3ed9c90367 charset: fix str[n]casecmp_m() by comparing lower case values
The commits c615ebed6e3d273a682806b952d543e834e5630d^..f19ab5d334e3fb15761fb009e5de876dfc6ea785
replaced Str[n]CaseCmp() by str[n]casecmp_m().

The logic we had in str[n]casecmp_w() used to compare
the upper cased as well as the lower cased versions of the
characters and returned the difference between the lower cased versions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13018

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Sep 15 02:23:29 CEST 2017 on sn-devel-144
2017-09-15 02:23:29 +02:00
Stefan Metzmacher
9d99b640b9 charset/tests: also tests the system str[n]casecmp()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13018

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-09-14 22:30:20 +02:00
Stefan Metzmacher
2a3d4fe0c9 charset/tests: add more str[n]casecmp_m() tests to demonstrate the bug
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13018

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-09-14 22:30:20 +02:00
Stefan Metzmacher
c18ecdecec charset/tests: assert the exact values of str[n]casecmp_m()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13018

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-09-14 22:30:20 +02:00
Justin Maggard via samba-technical
13971ba02e smbd: add missing newline to debug message in daemon_status()
Signed-off-by: Justin Maggard <jmaggard@netgear.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Sep  8 06:26:52 CEST 2017 on sn-devel-144
2017-09-08 06:26:52 +02:00
Andrew Bartlett
51289a6f9b debug: Add new debug class "drs_repl" for DRS replication processing
This is used in the client and in the server

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:27 +02:00
Andrew Bartlett
ba54816875 ldb: version 1.2.2
* Bug #13017: Add ldb_ldif_message_redacted_string() to allow debug
              of redacted log messages, avoiding showing secret values

* Bug #13015: Allow re-index of newer databases with binary GUID TDB keys
              (this officially removes support for re-index of the original
              pack format 0, rather than simply segfaulting).
* Avoid memory allocation and so make modify of records in ldb_tdb faster

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:27 +02:00
Andrew Bartlett
37bb62990b ldb: Add new ldb_ldif_message_redacted_string() with tests
This is designed to be a drop in replacement for
ldb_ldif_message_string() while better protecting privacy.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13017

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:27 +02:00
Andrew Bartlett
01c49b1a35 ldb_tdb: Refuse to re-index very old database with no DN in the record
These are not found on any AD DC, and would segfault previous LDB
versions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13015

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:27 +02:00
Andrew Bartlett
5463601122 ldb_tdb: Use braces in ltdb_dn_list_find_val()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13015

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:26 +02:00
Andrew Bartlett
6246c326a7 ldb_tdb: Check for talloc_strdup() failure in ltdb_index_add1()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13015

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:26 +02:00
Andrew Bartlett
ffc8023a0e ldb_tdb: Check for errors during tdb operations in ltdb_reindex()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13015

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:26 +02:00
Andrew Bartlett
3ce80cfb60 ldb_tdb: Use memcmp rather than strncmp() in ltdb_key_is_record(), re_key() and re_index()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:26 +02:00
Andrew Bartlett
fec666b334 ldb_tdb: Create a common ltdb_key_is_record() allowing multiple key forms
If backported, this allows old ldb versions to full-search and re-index newer databases
and in current code allows the transition to and from a GUID or incrementing ID based index

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13016

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:26 +02:00
Andrew Bartlett
2d0007ee5a ldb_tdb: Do not trigger the unique index check during a re-index, use another pass
We want to rename the objects, then scan looking for the index values.

This avoids a DB modify during the index scan traverse (the index values
are actually added to an in-memory TDB, written in prepare_commit()).

This allows us to remove the "this might already exist" case in the
index handling, we now know that the entry did not exist in the index
before we add it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13015

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:26 +02:00
Andrew Bartlett
da1e23418a ldb_tdb: Use memcmp() to compare TDB keys in re_index()
The keys may not always be a null terminated string, they could well
be a binary GUID in a future revision, for efficiency..

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13016

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:26 +02:00
Jeremy Allison
5f87a05aa3 lib: crypto: Add the ability to select Intel AESNI instruction set at configure time.
Add --accel-aes=[none|intelaesni] to select.
Default is none.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-09-07 02:01:08 +02:00
Jeremy Allison
53ac0f7c59 lib: crypto: Plumb in the Intel AES instructions.
Causes:

AES_set_encrypt_key()
AES_set_decrypt_key()
AES_encrypt()
AES_decrypt()

to probe for the Intel AES instructions at runtime (only once)
and then call the hardware implementations if so, otherwise
fall back to the software implementations.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008

Based on original work by Justin Maggard <jmaggard@netgear.com>

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-09-07 02:01:08 +02:00
Jeremy Allison
3324b55bde lib: crypt: Prepare the existing code to switch to Intel AES hardware instructions.
Rename the old struct aes_key as an intermediate struct aes_key_rj
and wrap it in a union so we can chose an alternate aes_key struct
when using Intel AES hardware.

Rename the original software implementations of:

 AES_set_encrypt_key()
 AES_set_decrypt_key()
 AES_encrypt()
 AES_decrypt()

by adding an _rj on the end, and call them via a wrapper
function.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008

Based on original work by Justin Maggard <jmaggard@netgear.com>

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-09-07 02:01:08 +02:00
Andreas Schneider
4c9608fb27 param: Add 'binddns dir' parameter
This allows to us to have restricted access to the directory by the group
'named' which bind is a member of.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12957

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
2017-09-05 23:58:20 +02:00
Volker Lendecke
5b53254ffc lib: Fix 1417431 Unchecked return value from library
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-09-05 19:13:12 +02:00
Volker Lendecke
981e674a74 pthreadpool: Test fork with an active thread
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13006
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Aug 31 21:34:57 CEST 2017 on sn-devel-144
2017-08-31 21:34:57 +02:00
Volker Lendecke
ff98e3fb66 pthreadpool: Fix fork behaviour
glibc's pthread_cond_wait(&c, &m) increments m.__data.__nusers, making
pthread_mutex_destroy return EBUSY. Thus we can't allow any thread waiting for
a job across a fork. Also, the state of the condvar itself is unclear across a
fork. Right now to me it looks like an initialized but unused condvar can be
used in the child. Busy worker threads don't cause any trouble here, they don't
hold mutexes or condvars. Also, they can't reach the condvar because _prepare
holds all mutexes.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13006
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-08-31 17:44:22 +02:00
Andrew Bartlett
a5a2243f07 ldb: Add tests for indexed and unindexed search expressions
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Aug 30 14:58:32 CEST 2017 on sn-devel-144
2017-08-30 14:58:32 +02:00
Andrew Bartlett
841e763a2b ldb: Fix tests to call the parent tearDown(), not setUp in tearDown
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-08-30 10:48:20 +02:00
Peter Somogyi
f0b85c1da2 s4/torture: make --unclist active with smb2 testcases
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12985

Signed-off-by: Peter Somogyi <psomogyi@hu.ibm.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Aug 30 00:16:51 CEST 2017 on sn-devel-144
2017-08-30 00:16:51 +02:00
Andrew Bartlett
3164c0ac54 ldb_tdb: Rework ltdb_modify_internal() to use ltdb_search_dn1() internally
This avoids duplicate code and allows us to use the allocation-avoiding
LDB_UNPACK_DATA_FLAG_NO_DATA_ALLOC flag.

We can not use LDB_UNPACK_DATA_FLAG_NO_VALUES_ALLOC as el2->values
is talloc_realloc()ed in the routine.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Aug 29 11:13:50 CEST 2017 on sn-devel-144
2017-08-29 11:13:50 +02:00
Andrew Bartlett
bff81a2c9c ldb: Add LDB_UNPACK_DATA_FLAG_NO_ATTRS
This will allow us to avoid a full unpack in situations where we just want to confirm
if the DN exists

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-08-29 07:23:29 +02:00
Stefan Metzmacher
c804f26085 tdb: version 1.3.15
* Add protection against EINTR.
* Truncate the file after expand failure, ENOSPC
* Use posix_fallocate() to expand the file
* Fix GCC compiler warnings

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Aug 24 21:17:48 CEST 2017 on sn-devel-144
2017-08-24 21:17:48 +02:00
Volker Lendecke
a475e1c4b0 tdb: Use posix_fallocate
This should be significantly faster than pwriting.

openbsd doesn't have posix_fallocate, so we do need the fallback. Also, it
might have weird failure modes, so we keep the old code in place except for
posix_fallocate returning success or ENOSPC.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Aug 24 05:38:49 CEST 2017 on sn-devel-144
2017-08-24 05:38:49 +02:00
Volker Lendecke
a05debc113 tdb: Add an intermediate variable
More README.Coding, but I need "ret" in the next commit as well :-)

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-08-24 01:46:08 +02:00
Volker Lendecke
c7211882a7 tdb: Truncate the file after expand failure
Without this it's very easy to create virtually huge files: ftruncate expands a
file, the pwrites fail with ENOSPC, thus the write fails. The next writer runs
into the same situation, and ftruncate-expands the file even further. tdb_check
will then spend ages reading the 4GB of zeros byte by byte.

Here we hold the freelist lock or are inside a transaction, so it is safe to
cut the file again. Nobody can have used the space that we have tried to
allocate, so we can't have any stray pointers corrupting the database.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-08-24 01:46:08 +02:00
Volker Lendecke
3e7efbfb36 tdb: Protect against EINTR
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-08-24 01:46:08 +02:00
Volker Lendecke
ab132ba7e5 configure: Centralize check for posix_fallocate
This checks for posix_fallocate unless we are sitting on an ancient glibc.
With this we don't need HAVE_BROKEN_POSIX_FALLOCATE anymore,
HAVE_POSIX_FALLOCATE will only be defined if we have a valid [g]libc.

./configure tested on Debian, FreeBSD (which does have posix_fallocate) and
OpenBSD (which does not have posix_fallocate). Also tested with changing the
not have an old-enough glibc around. All did the right thing.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-08-24 01:46:08 +02:00
Volker Lendecke
c66b214537 tdb: Fix a typo
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-08-24 01:46:08 +02:00
Ralph Boehme
a43affd4c5 tdb: fix tbdtool list freelist output
Due to the non-fixable bug in the BUCKET macro tdbtool list printed some
other hash chainlist, not the freelist.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12888

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-08-22 23:32:13 +02:00
Ralph Boehme
d2c78695c6 tdb: document hashtable bucket offset calculation madness
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-08-22 23:32:13 +02:00
Ralph Boehme
f2b7bc23df tbd: BUCKET(-1) returns wrong offset because tdb->hash_size is an unsigned int
The following C program demonstrates the issue:

  #include <stdio.h>
  #include <stdlib.h>
  #include <stdarg.h>
  #include <stdbool.h>
  #include <string.h>
  #include <unistd.h>
  #include <errno.h>
  #include <dirent.h>
  #include <sys/types.h>

  int main(int argc, char **argv)
  {
      int hash = -1;
      int tsize_signed = 10;
      unsigned int tsize_unsigned = 10;
      int bucket;

  #define BUCKET(hash, tsize) ((hash) % (tsize))

      bucket = BUCKET(hash, tsize_unsigned);
      printf("hash [%d] tsize [%d] bucket [%d]\n", hash, tsize_unsigned, bucket);

      bucket = BUCKET(hash, tsize_signed);
      printf("hash [%d] tsize [%d] bucket [%d]\n", hash, tsize_signed, bucket);

      return 0;
  }

Output:

$ ./tmp
hash [-1] tsize [10] bucket [5]
hash [-1] tsize [10] bucket [-1]

The first version is what the current BUCKET() macro does. As a result
we lock the hashtable chain in bucket 5, NOT the freelist.

-1 is sign converted to an unsigned int 4294967295 and

  4294967295 % 10 = 5.

As all callers will lock the same wrong list consistently locking is
still consistent.

Stumpled across this when looking at the output of `tdbtool DB list`
which always printed some other hashchain and not the freelist.

The freelist bucket offset computation doesn't use the BUCKET macro in
freelist.c (directly or indirectly) when working on the freelist, it
just directly uses the FREELIST_TOP define, so this problem only affects
tdbtool list.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-08-22 23:32:13 +02:00
Ralph Boehme
a5a19fa5ef tdb: rename struct tdb_traverse_lock hash member to list
The variable stores the hashtable bucket, not the hash. No change in
behaviour.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-08-22 23:32:13 +02:00
Martin Schwenke
c4771246b7 util: Add documentation for PID file handling
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Thu Aug 17 19:45:32 CEST 2017 on sn-devel-144
2017-08-17 19:45:32 +02:00
Volker Lendecke
a90702fc5d tdb: Clarify the CLEAR_IF_FIRST locked logic
This is another level of indentation, but it took me a while staring at the
if-condition to find that "locked" was assigned the result of "==0", not the
return value of tdb_nest_lock().

Best viewed with "git show -b".

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-08-17 11:54:10 +02:00
Martin Schwenke
9468d0f41e util: Add error handling to become_daemon()
Log failure and exit if fork() or setsid() fails.

Leave the logic in the non-setsid() code as it is.  This is probably
meant to fall through on failure of either opening /dev/tty or
ioctl().  Documentation for the ioctl() failure case is far from
clear.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Aug 17 11:48:32 CEST 2017 on sn-devel-144
2017-08-17 11:48:32 +02:00
Martin Schwenke
ed8c914b42 util: Move become_daemon.c to samba-util-core
So that CTDB can use it.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-08-17 08:00:23 +02:00
Martin Schwenke
b9804e9e3c util: Modernise logging
Switch to using DBG_ERR(), wrap logging/sd_notifyf() lines.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-08-17 08:00:23 +02:00
Martin Schwenke
a50092531a util: Make function definitions consistent with header file
no_process_group -> no_session, name -> daemon, drop _PUBLIC_.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-08-17 08:00:23 +02:00
Martin Schwenke
298af748ac util: Add become_daemon.h
Rename argument no_process_group to no_session to describe what it
actually does.  Consistently use "daemon" for name of daemon argument.
Add documentation.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-08-17 08:00:23 +02:00
Martin Schwenke
2b982fc309 util: Avoid use of includes.h
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-08-17 08:00:23 +02:00
Jeremy Allison
bee22f187c lib: tevent: Use system <tevent.h>, not internal header path (except in self-test).
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-08-16 20:58:13 +02:00
Volker Lendecke
ff0eed62d3 dbwrap: Simplify dbwrap_unmarshall_fn
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-08-16 00:27:10 +02:00
Volker Lendecke
ef2de247c1 dbwrap: Simplify dbwrap_trans_delete
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-08-16 00:27:10 +02:00
Volker Lendecke
2ddf8e0abc dbwrap: Simplify dbwrap_trans_store
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-08-16 00:27:10 +02:00
Volker Lendecke
b01b7c9ef9 dbwrap: Simplify dbwrap_store_uint32_bystring
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-08-16 00:27:10 +02:00
Volker Lendecke
ae28e4db44 dbwrap: Simplify dbwrap_store_int32_bystring
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-08-16 00:27:10 +02:00
Volker Lendecke
a11450d825 dbwrap: Convert dbwrap_delete to dbwrap_do_locked
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-08-16 00:27:10 +02:00
Volker Lendecke
5095733f92 dbwrap: Convert dbwrap_store to dbwrap_do_locked
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-08-16 00:27:10 +02:00
Andreas Schneider
d7e60bc17e tdb: Do not allow to pass NULL as the buffer to transaction_write()
This fixes a GCC warning.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Aug 10 02:26:09 CEST 2017 on sn-devel-144
2017-08-10 02:26:09 +02:00
Andreas Schneider
47bb27652e tdb: Write zero data using 8k buffer in transaction_expand_file()
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-08-09 22:34:17 +02:00
Volker Lendecke
f6a382fff8 tdb: Avoid NULL tdb_write
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-08-09 22:34:17 +02:00
Volker Lendecke
5c55c2563d tdb: Consistency check for tdb_storev
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-08-09 22:34:17 +02:00
Andreas Schneider
977cf75166 lib: Fix integer overflowed argument issue with strtoul()
This fixes CID 1415704

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Aug  3 15:06:34 CEST 2017 on sn-devel-144
2017-08-03 15:06:34 +02:00
Martin Schwenke
09b0398865 util: Reimplement pidfile_create() using pidfile_path_create()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-08-02 03:39:12 +02:00
Martin Schwenke
411b7c8753 util: New functions pidfile_path_create(), pidfile_fd_close()
Uses the core of CTDB's create_pidfile_context() for
pidfile_path_create(). pidfile_fd_close() is a subset of CTDB's
pidfile_context_destructor().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-08-02 03:39:11 +02:00
Martin Schwenke
4dbfc16bdd util: Add pidfile.* to samba-util-core
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-08-02 03:39:11 +02:00
Martin Schwenke
d665b74dbb util: Clean up includes
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-08-02 03:39:11 +02:00
Martin Schwenke
8731f1cfab util: pidfile_pid() should not unlink PID file
This causes a race.  If 2 callers to pidfile_create() both a find a
stale PID file using pidfile_pid().  The 1st may then return to
pidfile_create() and create a new PID file, which can then be unlinked
by the 2nd caller.

Consequently, PID file creation can not depend on creating the file,
so drop O_EXCL from the call to open().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-08-02 03:39:11 +02:00
Ian Stakenvicius
5b316a4c7a waf: disable-python - fix talloc wscript if bundling disabled
The pytalloc-util dependency logic in lib/talloc/wscript on a
standalone build checks for pytalloc-util in a manner that will
fail if bundling is disabled, this causes issues on
--disable-python builds of ldb, tevent, and samba.

This patch restructures the logic to skip checks if python
is disabled, instead just setting the temporary state variable
'using_system_pytalloc_util' to False

Successfully tested patch on ldb-1.1.31 and above, tevent-0.9.33,
and samba-4.7_rc3

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-08-01 03:59:24 +02:00
Volker Lendecke
9927e19965 lib: Fix 1415704 CID Integer overflowed argument
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-07-31 10:03:27 +02:00
Jeremy Allison
99e4bea89e lib: ldb: Python. Take care of freeing the passed in module description if ldb_register_module() fails.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12932

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-26 21:35:22 +02:00
Jeremy Allison
41b1f8a20c lib: ldb: Use NULL to allocate modules not talloc_autofree_context().
ldb modules are not (yet) unloaded and are only loaded once (there is a check
that makes sure of this). Allocate off the NULL context. We never want this
to be freed until process shutdown. If eventually we add the ability to
unload ldb modules we can add a deregister function that walks and frees the list.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12932

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-26 21:35:21 +02:00
Volker Lendecke
e3a0f0c100 dbwrap_tdb: Implement do_locked
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-07-25 17:43:16 +02:00
Volker Lendecke
cb2a28eb9a dbwrap: Add dbwrap_do_locked
With a proper implementation this enables modifications without
having to allocate a record. In really performance sensitive code
paths this matters.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-07-25 17:43:16 +02:00
Volker Lendecke
45f5a55ae6 dbwrap: Factor out the core dbwrap lock order logic
This will allow dbwrap_do_locked to check the order without talloc

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-07-25 17:43:16 +02:00
Volker Lendecke
10c59f5b20 dbwrap: Add dbwrap_record_storev
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-07-25 17:43:16 +02:00
Volker Lendecke
09915bf8e5 dbwrap: Convert backend store to storev
Convert all implementors of dbwrap_store to a storev-style call
by using the dbwrap_merge_dbufs call

For dbwrap_tdb, this matches tdb_storev.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-07-25 17:43:16 +02:00
Volker Lendecke
f0598e9853 dbwrap: Add dbwrap_merge_dbufs
Transitional code to implement dbwrap_record_storev

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-07-25 17:43:16 +02:00
Andreas Schneider
a841745166 uwrap: Update to version 1.2.4
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-25 09:14:19 +02:00
Stefan Metzmacher
19dfccea74 ldb: version 1.2.1
* Bug #12882: Do not install _ldb_text.py if we have system libldb
* Use libraries from build dir for testsuite
* Bug #12900: Fix index out of bound in ldb_msg_find_common_values

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Sat Jul 22 03:46:25 CEST 2017 on sn-devel-144
2017-07-22 03:46:25 +02:00
Stefan Metzmacher
7ad3a99d46 tevent: version 0.9.33
* make tevent_req_print() more robust against crashes

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-07-21 23:31:03 +02:00
Stefan Metzmacher
ca3e98488a tevent: handle passing req = NULL to tevent_req_print()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-07-21 23:31:03 +02:00
Stefan Metzmacher
21b56ffd98 tevent: avoid calling talloc_get_name(NULL) in tevent_req_default_print()
We have the same information available under req->internal.private_type.

This way it's possible to call tevent_req_print() after
tevent_req_received() was called.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-07-21 23:31:03 +02:00
Stefan Metzmacher
be7f6f4d5d talloc: version 2.1.10
* build, documentation and python3 improvements

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-07-21 23:31:03 +02:00
Douglas Bagnall
4b3de61185 ldb/tests: more thoroughly test empty ldb_msg elements
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Jul  7 20:10:37 CEST 2017 on sn-devel-144
2017-07-07 20:10:37 +02:00
Douglas Bagnall
282410fa24 ldb: avoid searching empty lists in ldb_msg_find_common_values
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-07-07 15:51:25 +02:00
Lukas Slebodnik
963d9f12f9 ldb: Fix index out of bound in ldb_msg_find_common_values
cmocka unit test failed on i386
[==========] Running 2 test(s).
[ RUN      ] test_ldb_msg_find_duplicate_val
[       OK ] test_ldb_msg_find_duplicate_val
[ RUN      ] test_ldb_msg_find_common_values
[  FAILED  ] test_ldb_msg_find_common_values
[==========] 2 test(s) run.
[  ERROR   ] --- 0x14 != 0
[   LINE   ] --- ../tests/ldb_msg.c:266: error: Failure!
[  PASSED  ] 1 test(s).
[  FAILED  ] 1 test(s), listed below:
[  FAILED  ] test_ldb_msg_find_common_values
 1 FAILED TEST(S)

But we were just lucky on other platforms because there is
index out of bound according to valgrind error.

==3298== Invalid read of size 4
==3298==    at 0x486FCF6: ldb_val_cmp (ldb_msg.c:95)
==3298==    by 0x486FCF6: ldb_msg_find_common_values (ldb_msg.c:266)
==3298==    by 0x109A3D: test_ldb_msg_find_common_values (ldb_msg.c:265)
==3298==    by 0x48E7490: ??? (in /usr/lib/libcmocka.so.0.4.1)
==3298==    by 0x48E7EB0: _cmocka_run_group_tests (in /usr/lib/libcmocka.so.0.4.1)
==3298==    by 0x1089B7: main (ldb_msg.c:352)
==3298==  Address 0x4b07734 is 4 bytes after a block of size 48 alloc'd
==3298==    at 0x483223E: malloc (vg_replace_malloc.c:299)
==3298==    by 0x4907AA7: _talloc_array (in /usr/lib/libtalloc.so.2.1.9)
==3298==    by 0x486FBF8: ldb_msg_find_common_values (ldb_msg.c:245)
==3298==    by 0x109A3D: test_ldb_msg_find_common_values (ldb_msg.c:265)
==3298==    by 0x48E7490: ??? (in /usr/lib/libcmocka.so.0.4.1)
==3298==    by 0x48E7EB0: _cmocka_run_group_tests (in /usr/lib/libcmocka.so.0.4.1)
==3298==    by 0x1089B7: main (ldb_msg.c:352)

Signed-off-by: Lukas Slebodnik <lslebodn@redhat.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-07-07 15:51:25 +02:00
Andreas Schneider
98e30cf491 waf: Do not install _ldb_text.py if we have system libldb
_ldb_text.py is installed as part of the ldb package and also if you
compile Samba with the system ldb version. This way we have have the
file twice in the same location and run into file confilcts.

This has already been fixed some time ago:
    60dc26bfe1

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12882

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
2017-07-07 09:14:26 +02:00
Lukas Slebodnik
d5d6d209de ldb: Use libraries from build dir for testsuite
There was a failure when tests were executed after after extracting
ldb tarball.

  sh$ make -j8 check
  WAF_MAKE=1 PATH=buildtools/bin:../../buildtools/bin:$PATH waf test
  ldbadd: error while loading shared libraries: libldb.so.1: cannot open shared object file: No such file or directory
  cat: write error: Broken pipe
  Traceback (most recent call last):
    File "tests/python/api.py", line 10, in <module>
      import ldb
  ImportError: libldb.so.1: cannot open shared object file: No such file or directory
  Traceback (most recent call last):
    File "tests/python/api.py", line 10, in <module>
      import ldb
  ImportError: libpyldb-util.so.1: cannot open shared object file: No such file or directory
  bin/ldb_tdb_mod_op_test: error while loading shared libraries: libldb.so.1: cannot open shared object file: No such file or directory
  testsuite returned 1

Signed-off-by: Lukas Slebodnik <lslebodn@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Alexander Bokovoy <ab@samba.org>
2017-07-04 21:39:22 +02:00
Lukas Slebodnik
9e1cbce273 talloc: Fix execution of test_magic_differs from tarball
make check failed in case of tarball because test_magic_differs.sh
is in top level directory and not in sub-directory lib/talloc

  sh: ./lib/talloc/test_magic_differs.sh: No such file or directory
  magic differs test returned 127

Signed-off-by: Lukas Slebodnik <lslebodn@redhat.com>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-07-04 21:39:21 +02:00
Lukas Slebodnik
4ad58497e5 talloc: Use libraries from build dir for testsuite
There was a failure when tests were executed after after extracting
talloc tarball.

  sh$ make -j8 check
  WAF_MAKE=1 PATH=buildtools/bin:../../buildtools/bin:$PATH waf test
  bin/talloc_testsuite: error while loading shared libraries: libtalloc.so.2: cannot open shared object file: No such file or directory
  sh: ./lib/talloc/test_magic_differs.sh: No such file or directory
  Traceback (most recent call last):
    File "test_pytalloc.py", line 11, in <module>
      import talloc
  ImportError: libtalloc.so.2: cannot open shared object file: No such file or directory

Signed-off-by: Lukas Slebodnik <lslebodn@redhat.com>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-07-04 21:39:21 +02:00
Andrew Bartlett
00db3aba6c param: Add new "disabled" value to "ntlm auth" to disable NTLM totally
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11923
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-07-04 06:57:20 +02:00
Andrew Bartlett
d0d266bbf7 param: Disable LanMan authentication unless NTLMv1 is also enabled
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11923
2017-07-04 06:57:20 +02:00
Andrew Bartlett
d139d77ae3 auth: Allow NTLMv1 if MSV1_0_ALLOW_MSVCHAPV2 is given and re-factor 'ntlm auth ='
The ntlm auth parameter is expanded to more clearly describe the
role of each option, and to allow the new mode that permits MSCHAPv2
(as declared by the client over the NETLOGON protocol) while
still banning NTLMv1.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12252
Signed-off-by: Andrew Bartlett <abartlet@samba.org>

Based on a patch by Mantas Mikulėnas <mantas@utenos-kolegija.lt>:

Commit 0b500d413c ("Added MSV1_0_ALLOW_MSVCHAPV2 flag to ntlm_auth")
added the --allow-mschapv2 option, but didn't implement checking for it
server-side. This implements such checking.

Additionally, Samba now disables NTLMv1 authentication by default for
security reasons. To avoid having to re-enable it globally, 'ntlm auth'
becomes an enum and a new setting is added to allow only MSCHAPv2.

Signed-off-by: Mantas Mikulėnas <mantas@utenos-kolegija.lt>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-07-04 06:57:20 +02:00
Ralph Boehme
d10ca1a005 lib/util: add more tfork tests
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-03 19:59:07 +02:00
Ralph Boehme
9b1c779447 lib/util: adjust loglevel in tfork test with samba_runcmd_send()
No change in behaviour, this just ensures stdout and stderror are
logged with log level 0.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-03 19:59:07 +02:00
Ralph Boehme
12a3eb7b5e lib/util: make use of tfork in samba_runcmd_send()
This makes it possible to use samba_runcmd_send() in processes like smbd
that install a SIGCHLD handler that reaps all terminated children.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-03 19:59:07 +02:00
Ralph Boehme
844db65498 lib/util: enhanced tfork()
This function is a solution to the problem of fork() requiring special
preperations in the caller to handle SIGCHLD signals and to reap the
child by wait()ing for it.

Instead, tfork provides a pollable file descriptor. The caller gets the
file descriptor by calling tfork_event_fd() on the handle returned from
tfork_create() and the caller can then get the status of the child
with a call to tfork_status().

tfork avoids raising SIGCHLD signals in the caller by installing a
temporary SIGCHLD handler from inside tfork_create() and tfork_status().

The termination signal of other child processes not created with tfork()
is forwarded to the existing signal handler if any.

There's one thing this thing can't protect us against and that is if a
process installs a SIGCHLD handler from one thread while another thread
is running inside tfork_create() or tfork_status() and the signal
handler doesn't forward signals for exitted childs it didn't fork, ie
our childs.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2017-07-03 19:59:07 +02:00
Stefan Metzmacher
3f0a2500ce ldb: version 1.2.0
* handle one more LDB_FLAG_INTERNAL_DISABLE_SINGLE_VALUE_CHECK
  case in ldb_tdb
* fix ldb_tdb locking (performance) problems
* fix ldb_tdb search inconsistencies by adding
  read_[un]lock() hooks to the module stack
  (bug #12858)
* add cmocka based tests for the locking issues
* ldb_version.h provides LDB_VERSION_{MAJOR,MINOR,RELEASE} defines
* protect ldb_modules.h from being used by Samba < 4.7
  Note: that this release (as well as 1.1.30 and 1.1.31)
  may cause problems for older applications, e.g. Samba
  See https://bugzilla.samba.org/show_bug.cgi?id=12859

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-02 17:35:20 +02:00
Stefan Metzmacher
0fac7e4d73 ldb:includes: protect ldb_modules.h from being used by Samba < 4.7
Samba versions before 4.7 are incompatible with the read_[un]lock()
behaviour introduced into ldb.

This makes sure older Samba versions fail to compile against
ldb >= 1.2.0.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12859

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-02 17:35:20 +02:00
Stefan Metzmacher
a1158d2237 ldb:wscript: define EXPECTED_SYSTEM_LDB_VERSION_{MAJOR,MINOR,RELEASE}
This indicates what feature set Samba assumes from the used
libldb from the system.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12859

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-02 17:35:19 +02:00
Stefan Metzmacher
774b2e4f5a ldb:wscript: provide LDB_VERSION_{MAJOR,MINOR,RELEASE} in ldb_version.h
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
b8b8439f1a ldb:tests: Extend api.py testsuite to show transaction_commit() blocks against the whole-db read lock
The new ldb whole-db lock behaviour now allows this test

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
6d7208c36f ldb:tests: Extend api.py testsuite to show transaction contents can not be seen outside the transaction
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
86e8425418 ldb:tests: Add test to show that locks are released on TALLOC_FREE(req)
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
4b7c274cc0 ldb:tests: Correct comment about version numbers
(ldb releases have been made while this patch set was in train)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
a8958515f0 ldb: Lock the whole backend database for the duration of a search
We must hold locks not just for the duration of each search, but for the whole search
as our module stack may make multiple search requests to build up the whole result.

This is explains a number of replication and read corruption issues in Samba

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
d3c0df35c9 ldb_tdb: Implement read_lock and read_unlock module operations
This allows Samba to provide a consistent view of the DB
despite the use of multiple databases via the partitions module
and over multiple callbacks via a module stack.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
6a3e77edc3 ldb: Add read_lock and read_unlock to ldb_module_ops
This will be used to implement read locking in ldb_tdb

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
31ef1d6211 ldb:tests: Add test encoding current locking behaviour during ldb_search()
Currently, a lock is not held against modifications once the final
record is returned via a callback, so modifications can be made
during the DONE callback.  This makes it hard to write modules
that interpert an ldb search result and do further processing
so will change in the future to allow the full search to be
atomic.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
0cd75c1fb1 ldb:tests: Show that writes do not appear during an ldb_search()
A modify or rename during a search must not cause a search to change
output, and attributes having an index should in particular not see
any change in behaviour in this respect

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
fd0beef179 ldb:tests: don't assert the results before doing the final search finished
This is required to pass the test in future, because
otherwise the clean up will fail because we hold locks.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Garming Sam
289d7d8fc5 ldb:tdb: Ensure we correctly decrement ltdb->read_lock_count
If we do not do this, then we never take the all record lock, and instead do a lock
for every record as we go, which is very slow during a large search

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
25e580f86e ldb_pack: use ldb_dn_from_ldb_val() and avoid a duplicate strlen() call
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Stefan Metzmacher
49c45fbb41 tdb: version 1.3.14
* allow tdb_traverse_read before tdb_transaction[_prepare]_commit()
* Improve documentation for tdb_transaction_start()
* Add new function tdb_transaction_active()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
32702a9745 tdb: Add new function tdb_transaction_active()
This will allow callers to avoid their own reference counting of transactions.

Additionally, this will always line up with the acutal transaction state, even
in the error cases where tdb can cancel the transaction

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
e08d02d027 tdb: Improve documentation for tdb_transaction_start()
It now references the TDB_ALLOW_NESTING and TDB_DISALLOW_NESTING flags

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
f80076fe43 tdb: Remove locking from tdb_traverse_read()
This restores the original intent of tdb_traverse_read() in
7dd31288a7

This is needed to avoid a deadlock with tdb_lockall() and the
transaction start, as ldb_tdb should take the allrecord lock during a
search (which calls tdb_traverse), and can otherwise deadlock against
a transaction starting in another process

We add a test to show that a transaction can now start while a read
traverse is in progress

This allows more operations to happen in parallel.  The blocking point
is moved to the prepare commit.

This in turn permits a roughly doubling of unindexed search
performance, because currently ldb_tdb omits to take the lock due to
an unrelated bug, but taking the allrecord lock triggers the
above-mentioned deadlock.

This behaviour was added in 251aaafe3a for
Solaris 10 in 2005. But the run-fcntl-deadlock test works also on Solaris 10,
see https://lists.samba.org/archive/samba-technical/2017-April/119876.html.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:18 +02:00
Jeremy Allison
bd9285b197 s3: VFS: Change SMB_VFS_GETWD to return struct smb_filename * instead of char *.
We need to migrate all pathname based VFS calls to use a struct
to finish modernising the VFS with extra timestamp and flags parameters.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Richard Sharpe <realrichardsharpe@gmail.com>
2017-07-01 03:07:11 +02:00
Andrew Bartlett
daeb74aed8 debug: new debug class for kerberos
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-06-30 02:12:22 +02:00
Stefan Metzmacher
1199907cbe param: change the effective default for "client max protocol" to the latest supported protocol
Currently it's SMB3_11.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-06-27 16:57:48 +02:00
Stefan Metzmacher
ec2da944d3 krb5_wrap: add smb_krb5_salt_principal2data()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12782

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-06-27 16:57:43 +02:00
Stefan Metzmacher
5df46700cf krb5_wrap: add smb_krb5_salt_principal()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12782

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-06-27 16:57:43 +02:00
Volker Lendecke
a433f1a757 lib: Give util_paths.c its own header
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-06-24 01:21:10 +02:00
Björn Baumbach
3a175830e5 waf:lib/replace: Fix building with older GCC versions
Using gcc 4.3.2:
cc1: error: unrecognized command line option "-Wno-format-truncation"

Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-06-23 21:01:17 +02:00
Stefan Metzmacher
e9b4978a76 tevent: version 0.9.32
* Fix mutex locking in tevent_threaded_context_destructor().
* Fix a memleak on FreeBSD.
* Re-init threading in tevent_re_initialise().
* Include the finish location in tevent_req_default_print().

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Jun 22 17:17:33 CEST 2017 on sn-devel-144
2017-06-22 17:17:33 +02:00
Stefan Metzmacher
d7f649b704 tevent: include the finish location in tevent_req_default_print()
It's verify useful when debugging code without a debugger to
be able to use tevent_req_print() in DEBUG statements.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-06-22 13:07:41 +02:00
Volker Lendecke
46a1ae5d71 dbwrap: Remove unused dbwrap_file
This has stopped working ages ago. The idea is clear, but if someone
wants to revive it, I think it needs a completely fresh start.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-06-21 03:15:08 +02:00
Volker Lendecke
fe1d17a48f dbwrap: Remove unused dbwrap_cache
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-06-21 03:15:08 +02:00
Volker Lendecke
a7504f555e tevent: Simplify create_immediate
Not much change, just 9 lines less of code.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-06-21 03:15:07 +02:00
Volker Lendecke
b67262b40a ldb: Fix CID 1412926 Unchecked return value
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-06-20 18:50:06 +02:00
Stefan Metzmacher
b4ae820648 krb5_wrap: handle KRB5_ERR_HOST_REALM_UNKNOWN in smb_krb5_get_realm_from_hostname()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-06-16 19:34:17 +02:00
Volker Lendecke
1fe7ec237a tevent_threads: Fix a rundown race introduced with 1828011317
The race is easily reproduced by adding a poll(NULL,0,10) in between the two
pthread_mutex_unlock calls in _tevent_threaded_schedule_immediate.

Before 1828011317, the main thread was signalled only after the helper
had already unlocked event_ctx_mutex.

Full explaination follows:
-----------------------------------------------------------------
Inside _tevent_threaded_schedule_immediate() we have:

476         ret = pthread_mutex_unlock(&ev->scheduled_mutex);
477         if (ret != 0) {
478                 abort();
479         }

HERE!!!!

481         ret = pthread_mutex_unlock(&tctx->event_ctx_mutex);
482         if (ret != 0) {
483                 abort();
484         }

At the HERE!!! point, what happens is tevent_common_threaded_activate_immediate(),
which is blocked on ev->scheduled_mutex, get released and does:

514         while (ev->scheduled_immediates != NULL) {
515                 struct tevent_immediate *im = ev->scheduled_immediates;
516                 DLIST_REMOVE(ev->scheduled_immediates, im);
517                 DLIST_ADD_END(ev->immediate_events, im);
518         }

- making an immediate event ready to be scheduled.

This then returns into epoll_event_loop_once(), which then calls:

910         if (ev->immediate_events &&
911             tevent_common_loop_immediate(ev)) {
912                 return 0;
913         }

which causes the immediate event to fire. This immediate
event is the pthread job terminate event, which was previously
set up in pthreadpool_tevent_job_signal() by:

198         if (state->tctx != NULL) {
199                 /* with HAVE_PTHREAD */
200                 tevent_threaded_schedule_immediate(state->tctx, state->im,
201                                                    pthreadpool_tevent_job_done,
202                                                    state);

So we now call pthreadpool_tevent_job_done() - which does:

225         TALLOC_FREE(state->tctx);

calling tevent_threaded_context_destructor():

384         ret = pthread_mutex_destroy(&tctx->event_ctx_mutex); <---------------- BOOM returns an error !
385         if (ret != 0) {
386                 abort();
387         }

as we haven't gotten to line 481 above (the line after
HERE!!!!) so the tctx->event_ctx_mutex is still
locked when we try to destroy it.

So doing an additional:

        ret = pthread_mutex_lock(&tctx->event_ctx_mutex);
        ret = pthread_mutex_unlock(&tctx->event_ctx_mutex);

(error checking elided) forces tevent_threaded_context_destructor()
to wait until tctx->event_ctx_mutex is unlocked before it locks/unlocks
and then is guaranteed safe to destroy.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-06-16 19:34:17 +02:00
Stefan Metzmacher
bcd558eb50 docs-xml: change the default for "map untrusted to domain" to "auto"
This makes the behaviour much more robust, particularly with forest child
domains over one-way forest trusts.

Sadly we don't support this kind of setup with our current ADDC, so
there's no way to have automated tests for this behaviour, but
at least we know it doesn't break any existing tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=8630

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-06-16 03:21:29 +02:00
Garming Sam
990b23d7b6 replmd: check single values in replmd_add_fix_la
repl_meta_data knows whether linked attributes are appropriately
[un-]duplicated, and this is how it tells ldb_tdb that.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-06-15 17:33:10 +02:00
Douglas Bagnall
6f956cce62 ldb: 1.1.31
* Add efficient function to find duplicate values in ldb messages
  (this makes large multi-valued attributes in ldb_tdb more efficient)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-06-15 17:33:10 +02:00
Douglas Bagnall
7d6f36aae3 ldb: relatively efficient functions for finding duplicate values
ldb backends need to make sure they are not adding duplicate values to
multi-valued attributes in ADD and MODIFY operations. Until now they
have done this inefficiently using nested loops. Here we add common
functions that deal with large numbers of values in O(n log n) time,
but continue to use the simple methods for small numbers of values.

These functions take a struct ldb_context pointer and an options flag
arguments, although the ldb is not used, and only one bit of the
options has meaning. This is to allow further patches to switch on
schema-aware comparisons.

This entails an ABI jump to add the two new functions.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-06-15 17:33:10 +02:00
Douglas Bagnall
aa61a2212b ldb.h whitespace
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-06-15 17:33:10 +02:00
Douglas Bagnall
16d208acc4 ldb tests/ldb_mod_op_test: don't double include cmocka.h
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-06-15 17:33:10 +02:00
Douglas Bagnall
433d600c56 ldb: fix a typo
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-06-15 17:33:10 +02:00
Douglas Bagnall
44764ee33d ldb: fix whitespace in ldb_msg.c
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-06-15 17:33:10 +02:00