1
0
mirror of https://github.com/samba-team/samba.git synced 2025-02-28 01:58:17 +03:00

108539 Commits

Author SHA1 Message Date
Ralph Boehme
47600c4977 s4/torture: pass destination tree to test_setup_copy_chunk
No change in behaviour, will be used in subsequent commits.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-03 19:59:07 +02:00
Ralph Boehme
d6a8a79fcb librpc/idl: make use storage_offload_token
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-03 19:59:07 +02:00
Ralph Boehme
e3cb6a936d librpc/idl: fix STORAGE_OFFLOAD_TOKEN_TYPE_ZERO_DATA definition
STORAGE_OFFLOAD_TOKEN_TYPE_ZERO_DATA is defined as 0xffff0001 in MS-FSCC
2.3.79.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-03 19:59:07 +02:00
Ralph Boehme
ebee4589fb librpc/idl: convert offload flags to a bitmap
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-03 19:59:07 +02:00
Ralph Boehme
d10ca1a005 lib/util: add more tfork tests
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-03 19:59:07 +02:00
Ralph Boehme
9b1c779447 lib/util: adjust loglevel in tfork test with samba_runcmd_send()
No change in behaviour, this just ensures stdout and stderror are
logged with log level 0.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-03 19:59:07 +02:00
Ralph Boehme
12a3eb7b5e lib/util: make use of tfork in samba_runcmd_send()
This makes it possible to use samba_runcmd_send() in processes like smbd
that install a SIGCHLD handler that reaps all terminated children.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-03 19:59:07 +02:00
Ralph Boehme
844db65498 lib/util: enhanced tfork()
This function is a solution to the problem of fork() requiring special
preperations in the caller to handle SIGCHLD signals and to reap the
child by wait()ing for it.

Instead, tfork provides a pollable file descriptor. The caller gets the
file descriptor by calling tfork_event_fd() on the handle returned from
tfork_create() and the caller can then get the status of the child
with a call to tfork_status().

tfork avoids raising SIGCHLD signals in the caller by installing a
temporary SIGCHLD handler from inside tfork_create() and tfork_status().

The termination signal of other child processes not created with tfork()
is forwarded to the existing signal handler if any.

There's one thing this thing can't protect us against and that is if a
process installs a SIGCHLD handler from one thread while another thread
is running inside tfork_create() or tfork_status() and the signal
handler doesn't forward signals for exitted childs it didn't fork, ie
our childs.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2017-07-03 19:59:07 +02:00
Stefan Metzmacher
1f5a297b51 s3:rpc_server: wrap make_auth4_context() into {become,unbecome}_root()
This need to create a temporary messaging context in order to do
the auth logging. This can only be done as root.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12850

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Mon Jul  3 08:15:29 CEST 2017 on sn-devel-144
2017-07-03 08:15:29 +02:00
Garming Sam
59bee844b8 WHATSNEW: Improved AD performance (particularly linked attributes)
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-03 03:59:17 +02:00
Garming Sam
f8e83f39e8 WHATSNEW: DNS at domain join improvements
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-03 03:59:17 +02:00
Garming Sam
63a56fe821 WHATSNEW: Additional hashes introduced with WDigest
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-03 03:59:17 +02:00
Garming Sam
5e6b4c4b13 WHATSNEW: Improved RODC support
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-03 03:59:17 +02:00
Andrew Bartlett
0cfef7f50e selftest: Prime the netlogon cache during test_idmap_rfc2307
This ensures that the group memberships just created are reflected in the test
comparison.  Otherwise we are trusting that no caches are primed, which is
simply not safe in a test.

(The login will put a list of groups, as obtained by the login over NETLOGON or
via the PAC, into the samlogon cache).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Sun Jul  2 21:59:18 CEST 2017 on sn-devel-144
2017-07-02 21:59:18 +02:00
Andrew Bartlett
eb26522a52 selftest: Bind rfc2307 tests to exactly one server
The tests make changes to the DC, do not wait for replication, then expect
those to be reflected in the client.  If they bind to another server this
will not hold true.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:20 +02:00
Andrew Bartlett
8b97a0af32 selftest: Use tree_delete control in idmap_rfc2307 test
This control removes an entire subtree, which was the intention of the previouse code
but much more effectively.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:20 +02:00
Bob Campbell
c0a7d2bacd selftest: Do not force run of kcc at start of selftest
This should help to avoid clashes between periodic and manual runs of
the KCC during autobuild.

Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>

Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:20 +02:00
Andrew Bartlett
a6bd91f4ec WHATSNEW: Add entry for Multi-process LDAP Server
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:20 +02:00
Andrew Bartlett
9063669a05 WHATSNEW: Add an entry for the LDB whole DB locking issue
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:20 +02:00
Andrew Bartlett
f4ce77857b ldap: Run the LDAP server with the default (typically standard) process model
This allows one LDAP socket to proceed if another fails, and reduces the
impact of a crash becoming a DoS bug, as it only impacts one socket.

This may mean we have a lot of idle tasks, but this should not be a big
issue

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:20 +02:00
Andrew Bartlett
5ec724687e dsdb: Improve debugging on start transacton failure
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:20 +02:00
Andrew Bartlett
d21d832195 dsdb: Teach the Samba partition module how to lock all the DB backends
The metadata partition (sam.ldb) lock is not
enough to block another process in prepare_commit(),
because prepare_commit() is a no-op, if nothing
was changed in the specific backend.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:20 +02:00
Andrew Bartlett
4894f47e2e dsdb: Add tests showing that the CN=CONFIGURATION partition is also locked
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:20 +02:00
Andrew Bartlett
b3db6558d3 dsdb: Add new test adding a record to the top level sam.ldb file
This shows that locks are made on this file as well

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:20 +02:00
Stefan Metzmacher
4b5ff4a309 dsdb: Add more locking more tests, confirming blocking locks in both directions
These extended tests allow us to show that a search (read) blocks a
transaction commit (write), and that a transaction commit blocks a
search.

Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
2017-07-02 17:35:20 +02:00
Andrew Bartlett
c5b4cbf34e dsdb: Add test showing a search can't start while a transaction is already repared in a backend partition
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:20 +02:00
Andrew Bartlett
c0e2909595 dsdb: Add test showing a search can't start while a transaction is already repared
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:20 +02:00
Stefan Metzmacher
3f0a2500ce ldb: version 1.2.0
* handle one more LDB_FLAG_INTERNAL_DISABLE_SINGLE_VALUE_CHECK
  case in ldb_tdb
* fix ldb_tdb locking (performance) problems
* fix ldb_tdb search inconsistencies by adding
  read_[un]lock() hooks to the module stack
  (bug #12858)
* add cmocka based tests for the locking issues
* ldb_version.h provides LDB_VERSION_{MAJOR,MINOR,RELEASE} defines
* protect ldb_modules.h from being used by Samba < 4.7
  Note: that this release (as well as 1.1.30 and 1.1.31)
  may cause problems for older applications, e.g. Samba
  See https://bugzilla.samba.org/show_bug.cgi?id=12859

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
ldb-1.2.0
2017-07-02 17:35:20 +02:00
Stefan Metzmacher
0fac7e4d73 ldb:includes: protect ldb_modules.h from being used by Samba < 4.7
Samba versions before 4.7 are incompatible with the read_[un]lock()
behaviour introduced into ldb.

This makes sure older Samba versions fail to compile against
ldb >= 1.2.0.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12859

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-02 17:35:20 +02:00
Stefan Metzmacher
a1158d2237 ldb:wscript: define EXPECTED_SYSTEM_LDB_VERSION_{MAJOR,MINOR,RELEASE}
This indicates what feature set Samba assumes from the used
libldb from the system.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12859

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-02 17:35:19 +02:00
Stefan Metzmacher
774b2e4f5a ldb:wscript: provide LDB_VERSION_{MAJOR,MINOR,RELEASE} in ldb_version.h
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
b8b8439f1a ldb:tests: Extend api.py testsuite to show transaction_commit() blocks against the whole-db read lock
The new ldb whole-db lock behaviour now allows this test

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
6d7208c36f ldb:tests: Extend api.py testsuite to show transaction contents can not be seen outside the transaction
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
86e8425418 ldb:tests: Add test to show that locks are released on TALLOC_FREE(req)
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
4b7c274cc0 ldb:tests: Correct comment about version numbers
(ldb releases have been made while this patch set was in train)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
a8958515f0 ldb: Lock the whole backend database for the duration of a search
We must hold locks not just for the duration of each search, but for the whole search
as our module stack may make multiple search requests to build up the whole result.

This is explains a number of replication and read corruption issues in Samba

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
d3c0df35c9 ldb_tdb: Implement read_lock and read_unlock module operations
This allows Samba to provide a consistent view of the DB
despite the use of multiple databases via the partitions module
and over multiple callbacks via a module stack.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
6a3e77edc3 ldb: Add read_lock and read_unlock to ldb_module_ops
This will be used to implement read locking in ldb_tdb

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
31ef1d6211 ldb:tests: Add test encoding current locking behaviour during ldb_search()
Currently, a lock is not held against modifications once the final
record is returned via a callback, so modifications can be made
during the DONE callback.  This makes it hard to write modules
that interpert an ldb search result and do further processing
so will change in the future to allow the full search to be
atomic.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
0cd75c1fb1 ldb:tests: Show that writes do not appear during an ldb_search()
A modify or rename during a search must not cause a search to change
output, and attributes having an index should in particular not see
any change in behaviour in this respect

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
fd0beef179 ldb:tests: don't assert the results before doing the final search finished
This is required to pass the test in future, because
otherwise the clean up will fail because we hold locks.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Garming Sam
289d7d8fc5 ldb:tdb: Ensure we correctly decrement ltdb->read_lock_count
If we do not do this, then we never take the all record lock, and instead do a lock
for every record as we go, which is very slow during a large search

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
25e580f86e ldb_pack: use ldb_dn_from_ldb_val() and avoid a duplicate strlen() call
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Stefan Metzmacher
49c45fbb41 tdb: version 1.3.14
* allow tdb_traverse_read before tdb_transaction[_prepare]_commit()
* Improve documentation for tdb_transaction_start()
* Add new function tdb_transaction_active()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
tdb-1.3.14
2017-07-02 17:35:19 +02:00
Andrew Bartlett
32702a9745 tdb: Add new function tdb_transaction_active()
This will allow callers to avoid their own reference counting of transactions.

Additionally, this will always line up with the acutal transaction state, even
in the error cases where tdb can cancel the transaction

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
e08d02d027 tdb: Improve documentation for tdb_transaction_start()
It now references the TDB_ALLOW_NESTING and TDB_DISALLOW_NESTING flags

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
f80076fe43 tdb: Remove locking from tdb_traverse_read()
This restores the original intent of tdb_traverse_read() in
7dd31288a701d772e45b1960ac4ce4cc1be782ed

This is needed to avoid a deadlock with tdb_lockall() and the
transaction start, as ldb_tdb should take the allrecord lock during a
search (which calls tdb_traverse), and can otherwise deadlock against
a transaction starting in another process

We add a test to show that a transaction can now start while a read
traverse is in progress

This allows more operations to happen in parallel.  The blocking point
is moved to the prepare commit.

This in turn permits a roughly doubling of unindexed search
performance, because currently ldb_tdb omits to take the lock due to
an unrelated bug, but taking the allrecord lock triggers the
above-mentioned deadlock.

This behaviour was added in 251aaafe3a9213118ac3a92def9ab2104c40d12a for
Solaris 10 in 2005. But the run-fcntl-deadlock test works also on Solaris 10,
see https://lists.samba.org/archive/samba-technical/2017-April/119876.html.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:18 +02:00
Andrew Bartlett
b87af322d5 drsuapi: Improve debugging in DsAddEntry()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:18 +02:00
Stefan Metzmacher
4ca48ee4d0 wafsamba: add maxversion and version_blacklist to CHECK_BUNDLED_SYSTEM[_PKG]()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12859

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-02 17:35:18 +02:00
Jeremy Allison
51c1a2a458 s3: VFS: Change SMB_VFS_CONNECTPATH to take const struct smb_filename * instead of const char *.
We need to migrate all pathname based VFS calls to use a struct
to finish modernising the VFS with extra timestamp and flags parameters.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Richard Sharpe <realrichardsharpe@gmail.com>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Jul  1 07:20:28 CEST 2017 on sn-devel-144
2017-07-01 07:20:28 +02:00