1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-24 02:04:21 +03:00

107057 Commits

Author SHA1 Message Date
Karolin Seeger
cb60d1c217 Revert "s4:kdc: use the strongest possible tgs session key"
This reverts commit 9fdf175905efde803941a5876ce7e060013fc9a0.
2018-02-21 11:23:24 +01:00
Karolin Seeger
0cd6906175 Revert "TODO s4:kdc: msDS-SupportedEncryptionTypes only on computers"
This reverts commit fe146338f304a52f861777ada5774887fe0776e3.
2018-02-21 11:23:23 +01:00
Karolin Seeger
89f27fab18 Revert "TODO s4:kdc: indicate support for new encryption types by adding empty keys"
This reverts commit bf07697273017014516010475f79be3e59a2ce07.
2018-02-21 11:23:23 +01:00
Karolin Seeger
3a54a04973 Revert "HEIMDAL:kdc: use the correct authtime from addtitional ticket for S4U2Proxy tickets"
This reverts commit 9ecdf21e174ba7525b77035664428fbdcbf53690.
2018-02-21 11:23:23 +01:00
Volker Lendecke
56a40ab005 samba: Only use async signal-safe functions in signal handler
Otherwise shutdown can hang

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13240

Signed-off-by: Björn Baumbach <bb@sernet.de>
(similar to commit 361ea743576cf125d7957a97ed78a0446dab1a19)

Autobuild-User(v4-6-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-6-test): Tue Feb 20 17:03:44 CET 2018 on sn-devel-144
2018-02-20 17:03:44 +01:00
Garming Sam
670af37291 subnet: Avoid a segfault when renaming subnet objects
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13031

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-02-20 12:52:17 +01:00
Stefan Metzmacher
f2e21e6926 HEIMDAL:kdc: use the correct authtime from addtitional ticket for S4U2Proxy tickets
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13137

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2018-02-20 12:52:17 +01:00
Stefan Metzmacher
ffda28e9b1 TODO s4:kdc: indicate support for new encryption types by adding empty keys
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13135
2018-02-20 12:52:17 +01:00
Stefan Metzmacher
075f061ca3 TODO s4:kdc: msDS-SupportedEncryptionTypes only on computers
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13135
2018-02-20 12:52:17 +01:00
Stefan Metzmacher
7d0559e0eb s4:kdc: use the strongest possible tgs session key
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13135

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2018-02-20 12:52:17 +01:00
Stefan Metzmacher
2a7392d3b2 HEIMDAL:hdb: export a hdb_enctype_supported() helper function
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13135

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2018-02-20 12:52:17 +01:00
Stefan Metzmacher
8ac00b066c HEIMDAL:kdc: let _kdc_encode_reply() use the encryption type based on the server key
Currently the value is the same anyway as the session key is always of the
same type as server key up to now, but that will change shortly.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13135

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2018-02-20 12:52:17 +01:00
Stefan Metzmacher
9f3571aa20 s4:kdc: fix the principal names in samba_kdc_update_delegation_info_blob
We need the target service without realm, but the proxy services with realm.

I have a domain with an w2008r2 server and a samba and now both generate
the same S4U_DELEGATION_INFO.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13133

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2018-02-20 12:52:17 +01:00
Stefan Metzmacher
312bf1c331 HEIMDAL:kdc: if we don't have an authenticator subkey for S4U2Proxy we need to use the additional tickets key
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13131

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2018-02-20 12:52:17 +01:00
Stefan Metzmacher
3dd52dd0df HEIMDAL:kdc: decrypt b->enc_authorization_data in tgs_build_reply()
We do this after checking for constraint delegation (S4U2Proxy).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13131

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2018-02-20 12:52:17 +01:00
Stefan Metzmacher
9ec1a523d2 HEIMDAL:kdc: fix memory leak when decryption AuthorizationData
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13131

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2018-02-20 12:52:17 +01:00
Karolin Seeger
2ed874196b VERSION: Bump version up to 4.6.14...
and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>
2018-02-14 10:39:29 +01:00
Karolin Seeger
fd09a022d0 VERSION: Disable GIT_SNAPSHOT for the 4.6.13 release.
Signed-off-by: Karolin Seeger <kseeger@samba.org>
samba-4.6.13
2018-02-14 10:39:29 +01:00
Karolin Seeger
38e71baaba WHATSNEW: Add release notes for Samba 4.6.13.
Signed-off-by: Karolin Seeger <kseeger@samba.org>
2018-02-14 10:39:29 +01:00
David Disseldorp
666c6802cb build: fix ceph_statx check when configured with libcephfs_dir
When configured with a custom libcephfs_dir, the ceph_statx check fails
to link. This is due to the location of the ceph-common dependency,
which is installed under a ceph subdirectory.

ceph/build > make DESTDIR=./inst install
...
ceph/build > find inst/|grep -e /libcephfs -e /libceph-common
inst/usr/local/lib64/ceph/libceph-common.so.0
inst/usr/local/lib64/ceph/libceph-common.so
inst/usr/local/lib64/libcephfs.so.2.0.0
inst/usr/local/lib64/libcephfs.so.2
inst/usr/local/lib64/libcephfs.so
inst/usr/local/include/cephfs/libcephfs.h

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13250

Signed-off-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Jeff Layton <jlayton@samba.org>

(cherry picked from commit ff7df3d3f5259362a6bb6780d6b532e57e89681d)

Autobuild-User(v4-6-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-6-test): Tue Feb 13 13:36:13 CET 2018 on sn-devel-144
2018-02-13 13:36:13 +01:00
Ralph Boehme
301555830d vfs_fruit: set delete-on-close for empty finderinfo
We previously removed the stream from the underlying filesystem stream
backing store when the client zeroes out FinderInfo in the AFP_AfpInfo
stream, but this causes certain operations to fail (eg stat) when trying
to access the stream over any file-handle open on that stream.

So instead of deleting, set delete-on-close on the stream. The previous
commit already implemented not to list list streams with delete-on-close
set which is necessary to implemenent correct macOS semantics for this
particular stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13181

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Jan  9 17:09:12 CET 2018 on sn-devel-144

(backported from commit e61e9e98e9ff461055daae2fe78f0202f7ed8663)

Autobuild-User(v4-6-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-6-test): Thu Jan 25 19:25:24 CET 2018 on sn-devel-144
2018-01-25 19:25:24 +01:00
Ralph Boehme
9e47e9e28c vfs_fruit: filter out AFP_AfpInfo streams with pending delete-on-close
This is in preperation of fixing the implementation of removing the
AFP_AfpInfo stream by zeroing the FinderInfo out.

We currently remove the stream blob from the underyling filesystem
backing store, but that results in certain operations to fail on any
still open file-handle.

The fix comes in the next commit which will convert to backing store
delete operation to a set delete-on-close on the stream.

This commit adds filtering on streams that have the delete-on-close
set. It is only needed for the fruit:metadata=stream case, as with
fruit:metadata=netatalk the filtering is already done in
fruit_streaminfo_meta_netatalk().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13181

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 70d8f7c5d25f35b58620c2db8f57c7c0758267b3)
2018-01-25 15:06:21 +01:00
Ralph Boehme
c1e0396bcb vfs_fruit: factor out delete_invalid_meta_stream() from fruit_streaminfo_meta_stream()
No change in behaviour, just some refactoring before adding more code to
fruit_streaminfo_meta_stream() in the next commit.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13181

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(backported from commit c41e1ea9247611473d30184efd953c61955ead15)
2018-01-25 15:06:21 +01:00
Ralph Boehme
d95b278b00 s4/torture/fruit: enhance zero AFP_AfpInfo stream test
This test more operations in the zeroed out FinderInfo test, ensuring
after zeroing out FinderInfo, operations on the filehandle still work
and that enumerating streams doesn't return the stream anymore.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13181

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(backported from commit df31e94eb6241f5e5594f6fd0ec1ad7896e02e27)
2018-01-25 15:06:21 +01:00
Ralph Boehme
26da45be00 s4/torture/fruit: ensure AFP_AfpInfo blobs are 0-initialized
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13181

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit a22833c2971dc7234b32741305f40ed62e232e0b)
2018-01-25 15:06:21 +01:00
David Disseldorp
21d0446c48 vfs_default: use VFS statvfs macro in fs_capabilities
Currently the vfs_default fs_capabilities handler calls statvfs
directly, rather than calling the vfs macro. This behaviour may cause
issues for VFS modules that delegate fs_capabilities handling to
vfs_default but offer their own statvfs hook.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13208

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 4b25c9f4a4d336a16894452862ea059701b025de)

Autobuild-User(v4-6-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-6-test): Mon Jan 22 13:14:03 CET 2018 on sn-devel-144
2018-01-22 13:14:03 +01:00
David Disseldorp
a6b780c52b vfs_ceph: add fs_capabilities hook to avoid local statvfs
Adding the fs_capabilities() hook to the CephFS VFS module avoids
fallback to the vfs_default code-path, which calls statvfs() against the
share path on the *local* filesystem.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13208

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 2724e0cac29cd1632ea28075a740fcc888affb36)
2018-01-22 08:50:09 +01:00
Jeremy Allison
579b6a4f60 s3: smbd: Use identical logic to test for kernel oplocks on a share.
Due to inconsistent use of lp_kernel_oplocks() we could miss kernel
oplocks being on/off in some of our oplock handling code, and thus
use the wrong logic.

Ensure all logic around koplocks and lp_kernel_oplocks() is consistent.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13193

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Jan  4 16:03:38 CET 2018 on sn-devel-144

(cherry picked from commit 114f5da2fab6f587de77e792274b396fb3d7ce71)

Autobuild-User(v4-6-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-6-test): Fri Jan  5 13:40:55 CET 2018 on sn-devel-144
2018-01-05 13:40:55 +01:00
Christof Schmitt
6ba6125274 smbd: Fix coredump on failing chdir during logoff
server_exit does an internal tree disconnect which requires a chdir to
the share directory. In case the file system encountered a problem and
the chdir call returns an error, this triggers a SERVER_EXIT_ABNORMAL
which in turn results in a panic and a coredump. As the log already
indicates the problem (chdir returned an error), avoid the
SERVER_EXIT_ABNORMAL in this case and not trigger a coredump.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13189

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Dec 16 01:56:06 CET 2017 on sn-devel-144

(cherry picked from commit 7fa91fc4791d076c609eaf119753e38dd3c50a1c)

Autobuild-User(v4-6-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-6-test): Tue Jan  2 14:01:29 CET 2018 on sn-devel-144
2018-01-02 14:01:29 +01:00
Christof Schmitt
60eb51d6e6 selftest: Add test for failing chdir call in smbd
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13189

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 0d3000be2af8f8c4a37892d95ae694ad834d7b3a)
2018-01-02 10:01:10 +01:00
Christof Schmitt
e6ec5ae882 selftest: Make location of log file available in tests
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13189

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit b0e1fc74fdacecb86f46b47e527b3fdf1906d27b)
2018-01-02 10:01:10 +01:00
Christof Schmitt
90d87d4b9c selftest: Add share for error injection testing
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13189

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 8b6402f3e5ff98c2701e626e47246b2400f76e5f)
2018-01-02 10:01:10 +01:00
Christof Schmitt
919d16e3f3 vfs_error_inject: Add new module
This module allow injecting errors in vfs calls. It only implements one
case (return ESTALE from chdir), but the idea is to extend this to more
vfs functions and more errors when needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13189

Signed-off-by: Christof Schmitt <cs@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 24623d53256c2424563709dedc19af1a106ccc73)
2018-01-02 10:01:10 +01:00
Amitay Isaacs
d932fcf6c0 ctdb-recovery-helper: Deregister message handler in error paths
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13188

If PULL_DB control times out but the remote node is still sending the
data, then the tevent_req for pull_database_send will be freed without
removing the message handler.  So when the data is received, srvid
handler will be called and it will try to access tevent_req which will
result in use-after-free and abort.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2018-01-02 10:01:10 +01:00
Uri Simchoni
a3dc640c9d sysacls: change datatypes to 32 bits
The SMB_ACL_PERMSET_T and SMB_ACL_PERM_T were defined as
mode_t, which is 16-bits on some (non-Linux) systems. However,
pidl *always* encodes mode_t as uint32_t. That created a bug on
big-endian systems as sys_acl_get_permset() returns a SMB_ACL_PERMSET_T
pointer to an internal a_perm structure member defined in IDL as a mode_t,
which pidl turns into a uin32_t in the emitted header file.

Changing to 32 bits fixes that.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13176

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(back-ported from commit 75e7da9741c529f96fa409655ac5b326a6c071c5)
2018-01-02 10:01:10 +01:00
Uri Simchoni
e64528a915 pysmbd: fix use of sysacl API
Fix pysmbd to use the sysacl (POSIX ACL support) as intended, and
not assume too much about the inner structure and implementation
of the permissions in the sysacl API.

This will allow the inner structure to change in a following commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13176

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit d6f5ee6707fa5404e2bef6fc81ae06b393ebd8ff)
2018-01-02 10:01:10 +01:00
Stefan Metzmacher
f502340923 HEIMDAL:kdc: fix dh->q allocation check in get_dh_param()
Thanks to Doug Nazar <nazard@nazar.ca> for spotting this!

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12986

Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from heimdal commit a79b59ba27070a015479e8d981b7e685dbe34310)
Reviewed-by: Andreas Schneider <asn@samba.org>

(cherry picked from commit 183e5d1e3dc306491c06f94c8c98e4882c64bc27)

Autobuild-User(v4-6-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-6-test): Sun Dec 24 01:15:07 CET 2017 on sn-devel-144
2017-12-24 01:15:07 +01:00
Love Hornquist Astrand
c6dfb4e1cc HEIMDAL: don't bother seeing q if not sent
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12986

Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from heimdal commit 19f9fdbcea11013cf13ac72c416f161ee55dee2b)

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Aug 28 15:10:54 CEST 2017 on sn-devel-144

(cherry picked from commit dd3e06f14ec1788a59e4c6ba4ea165fd77b9135e)
2017-12-23 21:16:25 +01:00
Love Hornquist Astrand
03c69a536d HEIMDAL: allow optional q in DH DomainParameters
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12986

Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from heimdal commit e8317b955f5a390c4f296871ba6987ad05478c95)

(cherry picked from commit 9f245aafdca8397df7dc050e25cfd858aeb1cc7f)
2017-12-23 21:16:25 +01:00
Stefan Metzmacher
f69814f035 g_lock: fix cleanup of stale entries in g_lock_trylock()
g_lock_trylock() always incremented the counter 'i', even after cleaning a stale
entry at position 'i', which means it skipped checking for a conflict against
the new entry at position 'i'.

As result a process could get a write lock, while there're still
some read lock holders. Once we get into that problem, also more than
one write lock are possible.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13195

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Dec 20 20:31:48 CET 2017 on sn-devel-144
(similar to commit 576fb4fb5dc506bf55e5cf87973999dca444149b)

Autobuild-User(v4-6-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-6-test): Fri Dec 22 22:11:00 CET 2017 on sn-devel-144
2017-12-22 22:11:00 +01:00
Stefan Metzmacher
e39dcec5e1 s4:kdc: only map SDB_ERR_NOT_FOUND_HERE to HDB_ERR_NOT_FOUND_HERE
HDB_ERR_NOT_FOUND_HERE indicated a very specific error on an RODC.

We should not map any error to HDB_ERR_NOT_FOUND_HERE,
we should just pass errors along unmapped.

Otherwise we'll hit the logic bug in:

    if (ret == KDC_PROXY_REQUEST) {
        uint16_t port;

        if (!sock->kdc_socket->kdc->am_rodc) {
            DEBUG(0,("kdc_udp_call_loop: proxying requested when not RODC"));
                    talloc_free(call);
            goto done;
        }

And just don't send an error message to the client.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13132

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Dec  6 23:16:54 CET 2017 on sn-devel-144

(cherry picked from commit aaa946bb9eb8088389b8ffdec460023f1961616c)
2017-12-22 18:11:19 +01:00
Karolin Seeger
51fb772b3b VERSION: Bump version up to 4.6.13...
and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>
2017-12-20 21:33:54 +01:00
Karolin Seeger
1377b56d38 VERSION: Disable GIT_SNAPSHOT for the 4.6.12 release.
Signed-off-by: Karolin Seeger <kseeger@samba.org>
samba-4.6.12
2017-12-20 21:30:24 +01:00
Karolin Seeger
d665971030 WHATSNEW: Add release notes for Samba 4.6.12.
Signed-off-by: Karolin Seeger <kseeger@samba.org>
2017-12-20 21:29:41 +01:00
Volker Lendecke
d9aaf8d3d6 messaging: Always register the unique id
The winbind child does not call serverid_register, so the unique id is not
registered. ctdbd_process_exists now calls CTDB_CONTROL_CHECK_PID_SRVID, which
then fails.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13180
Signed-off-by: Volker Lendecke <vl@samba.org>

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Fri Dec 15 15:35:25 CET 2017 on sn-devel-144

(cherry picked from commit 1eb08445d96a2c41593719925203f43f881b3567)

Autobuild-User(v4-6-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-6-test): Mon Dec 18 15:32:49 CET 2017 on sn-devel-144
2017-12-18 15:32:49 +01:00
Volker Lendecke
1a8c27f408 pthreadpool: Add a test for the race condition fixed in the last commit
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13179
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 53f7bbca0451e4f57cdbe8ab4f67f601fe8d40c1)

Autobuild-User(v4-6-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-6-test): Fri Dec 15 15:22:27 CET 2017 on sn-devel-144
2017-12-15 15:22:27 +01:00
Volker Lendecke
b181b26caa pthreadpool: Fix starvation after fork
After the race is before the race:

1) Create an idle thread
2) Add a job: This won't create a thread anymore
3) Immediately fork

The idle thread will be woken twice before it's actually woken up: Both
pthreadpool_add_job and pthreadpool_prepare_pool call cond_signal, for
different reasons. We must look at pool->prefork_cond first because otherwise
we will end up in a blocking job deep within a fork call, the helper thread
must take its fingers off the condvar as quickly as possible.  This means that
after the fork there's no idle thread around anymore that would pick up the job
submitted in 2). So we must keep the idle threads around across the fork.

The quick solution to re-create one helper thread in pthreadpool_parent has a
fatal flaw: What do we do if that pthread_create call fails? We're deep in an
application calling fork(), and doing fancy signalling from there is really
something we must avoid.

This has one potential performance issue: If we have hundreds of idle threads
(do we ever have that) during the fork, the call to pthread_mutex_lock on the
fork_mutex from pthreadpool_server (the helper thread) will probably cause a
thundering herd when the _parent call unlocks the fork_mutex. The solution for
this to just keep one idle thread around. But this adds code that is not
strictly required functionally for now.

More detailed explanation from Jeremy:

First, understanding the problem the test reproduces:

add a job (num_jobs = 1) -> creates thread to run it.
job finishes, thread sticks around (num_idle = 1).
num_jobs is now zero (initial job finished).

a) Idle thread is now waiting on pool->condvar inside
pthreadpool_server() in pthread_cond_timedwait().

Now, add another job ->

	pthreadpool_add_job()
		-> pthreadpool_put_job()
			This adds the job to the queue.
		Oh, there is an idle thread so don't
		create one, do:

		pthread_cond_signal(&pool->condvar);

		and return.

Now call fork *before* idle thread in (a) wakes from
the signaling of pool->condvar.

In the parent (child is irrelevent):

Go into: pthreadpool_prepare() ->
		pthreadpool_prepare_pool()

		Set the variable to tell idle threads to exit:

		pool->prefork_cond = &prefork_cond;

		then wake them up with:

		pthread_cond_signal(&pool->condvar);

		This does nothing as the idle thread
		is already awoken.

b) Idle thread wakes up and does:

		Reduce idle thread count (num_idle = 0)

		pool->num_idle -= 1;

		Check if we're in the middle of a fork.

		if (pool->prefork_cond != NULL) {

			Yes we are, tell pthreadpool_prepare()
			we are exiting.

			pthread_cond_signal(pool->prefork_cond);

			And exit.

			pthreadpool_server_exit(pool);
			return NULL;
		}

So we come back from the fork in the parent with num_jobs = 1,
a job on the queue but no idle threads - and the code that
creates a new thread on job submission was skipped because
an idle thread existed at point (a).

OK, assuming that the previous explaination is correct, the
fix is to create a new pthreadpool context mutex:

pool->fork_mutex

and in pthreadpool_server(), when an idle thread wakes up and
notices we're in the prepare fork state, it puts itself to
sleep by waiting on the new pool->fork_mutex.

And in pthreadpool_prepare_pool(), instead of waiting for
the idle threads to exit, hold the pool->fork_mutex and
signal each idle thread in turn, and wait for the pool->num_idle
to go to zero - which means they're all blocked waiting on
pool->fork_mutex.

When the parent continues, pthreadpool_parent()
unlocks the pool->fork_mutex and all the previously
'idle' threads wake up (and you mention the thundering
herd problem, which is as you say vanishingly small :-)
and pick up any remaining job.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13179
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit f6858505aec9f1004aeaffa83f21e58868749d65)
2017-12-15 11:18:07 +01:00
Ralph Boehme
7dcc119c14 winbindd: idmap_rid: error code for failing id-to-sid mapping request
NT_STATUS_NO_SUCH_DOMAIN triggers complete request failure in the parent
winbindd. By returning NT_STATUS_NONE_MAPPED winbindd lets the individual
mapping fail but keeps processing any remaining mapping requests.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13052

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Oct 10 19:57:37 CEST 2017 on sn-devel-144

(cherry picked from commit 490c35df35bad6c2f1c4acd2f056d6fdc480ec1f)

Autobuild-User(v4-6-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-6-test): Thu Dec 14 16:20:49 CET 2017 on sn-devel-144
2017-12-14 16:20:49 +01:00
Ralph Boehme
d85e691a0b winbindd: idmap_rid: don't rely on the static domain list
The domain list in the idmap child is inherited from the parent winbindd
process and may not contain all domains in case enumerating trusted
domains didn't finish before the first winbind request that triggers the
idmap child fork comes along.

The previous commits added the domain SID as an additional argument to
the wbint_UnixIDs2Sids request, storing the domain SID in struct
idmap_domain.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13052

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 108675c4cf4c3d5bd29468255743423a56bd1471)
2017-12-14 12:21:09 +01:00
Ralph Boehme
96cc7e0fe3 winbindd: pass domain SID to wbint_UnixIDs2Sids
This makes the domain SID available to the idmap child for
wbint_UnixIDs2Sids mapping request. It's not used yet anywhere, this
comes in the next commit.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13052

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 71f99cb132f4c26f9febac6cb7dcd79f4940216a)
2017-12-14 12:21:09 +01:00