1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-24 21:34:56 +03:00
Commit Graph

34442 Commits

Author SHA1 Message Date
Stefan Metzmacher
1eb8d8ec5a s4:dsdb: allocate DSDB_CONTROL_DBCHECK_FIX_DUPLICATE_LINKS oid
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-24 15:50:16 +01:00
Stefan Metzmacher
126d28d0b5 s4:schema_samba4: mark DSDB_CONTROL_INVALID_NOT_IMPLEMENTED 1.3.6.1.4.1.7165.4.3.32 as allocated
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-24 15:50:16 +01:00
Andrew Bartlett
a784cc3a7f selftest: Additional check for a backlink pointing at a deleted object
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-11-24 15:50:16 +01:00
Andrew Bartlett
7be38c6054 selftest: add more dbcheck tests
This validates some more combinations and ensures that the changes
in 962a1b3220 are tested.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-11-24 15:50:16 +01:00
Andrew Bartlett
3b111fbdbe dbcheck: Clarify error count bumping in deleted/gone DN handling
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-11-24 15:50:15 +01:00
Uri Simchoni
2bf01b286c s4-torture: fix type of enum in various places
Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-24 01:13:15 +01:00
Uri Simchoni
559367ed60 s4-torture: fix truncation warnings
Fix various places where there is potential truncation
while doing time / size calculations.

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-24 01:13:15 +01:00
Garming Sam
8eb95bc1d1 schema_set: Add comment about set schema from ldif in a transaction
This is normally called with a transaction or before access is shared.
The python code and some tests may also cause an issue, but as these are
fixed at runtime, this is only a temporary issue that resolves itself.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-24 01:13:14 +01:00
Garming Sam
17f1c6f9f4 schema: Make writing indices flag an enum for a new state
In schema_load_init, we find that the writing of indices is not locked
in any way. This leads to race conditions. To resolve this, we need to
have a new state (SCHEMA_COMPARE) which can report to the caller that we
need to open a transaction to write the indices.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-24 01:13:14 +01:00
Uri Simchoni
4171191dae s2-rpc-server: fix enum type in assignment
Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-22 10:20:21 +01:00
Uri Simchoni
1a2da5b0f8 s4-lib-policy: fix type of enum
Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-22 10:20:20 +01:00
Uri Simchoni
2947945dc3 s4-torture: get rid of extra parentheses
Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-22 10:20:20 +01:00
Uri Simchoni
a83953a094 s4-torture: fix file time checks
NTTIME is an unsigned quantity. When comparing two
of them, first calculate a signed difference, then
take absolute value.

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-22 10:20:20 +01:00
Uri Simchoni
5fd04020c2 dns server: fix warning about enum mismatch
Fix picky developer clang warning about assignment
of an enum value to a variable of a different enum type.

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-22 10:20:20 +01:00
Uri Simchoni
9ad9f7bc5e librpc-build: ignore unused functions in generated code
Some pidl-generated code includes static functions that are
to be optimized-away by the compiler if not used. When
running picky developer with clang that breaks the build. This
change ignores this warning for the pidl-generated python binding
files.

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-22 10:20:20 +01:00
Uri Simchoni
5aa8af0c4a torture: remove spurious semicolon
Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-22 10:20:20 +01:00
Garming Sam
6e7d037ace Fix formating of sources to be less than 80 lines
Signed-off-by: David Mulder <dmulder@suse.com>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Tue Nov 21 01:51:59 CET 2017 on sn-devel-144
2017-11-21 01:51:59 +01:00
Andrew Bartlett
3bc0c1f8ee gpoupdate: Move closer to 80 columns
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-11-20 21:41:15 +01:00
David Mulder
8be71f97b6 doc: Add samba_gpoupdate man page, update WHATSNEW
Signed-off-by: David Mulder <dmulder@suse.com>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-11-20 21:41:15 +01:00
David Mulder
e60f49783e gpo: Apply kerberos settings
Add kdc kerberos settings to gpo.tdb, then retrieve those settings in
lpcfg_default_kdc_policy.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-20 21:41:15 +01:00
David Mulder
4a7ccbeab7 gpo: Always enforce policy, even if unchanged
Policies should always be enforced, even if the gpo hasn't changed.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-20 21:41:15 +01:00
David Mulder
8d4c7229e9 gpo: Add GPO unapply
Keep a log of applied settings, and add an option to samba_gpoupdate to allow unapply. An unapply will revert settings to a state prior to any policy application.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-20 21:41:15 +01:00
David Mulder
e750e4a35f gpo: Add gpo tests
Lays down a sysvol gpttmpl.inf with password policies, then runs the samba_gpoupdate command. Verifies policies are applied to the samdb.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-20 21:41:15 +01:00
David Mulder
05235a56e3 gpo: Install the samba_gpoupdate script
The samba_gpoupdate script was not being installed by waf.
Added samba_gpoupdate to the wscripts so it gets installed as part of a make install.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-20 21:41:15 +01:00
David Mulder
de9cee2262 gpoupdate: Rewrite samba_gpoupdate
Use new python bindings and remove obsoleted code

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-20 21:41:15 +01:00
David Mulder
8eba3b5d38 gpo: Make the gpclass more easily extensible
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-20 21:41:15 +01:00
Luke Morrison
50a64b7ce9 gpo: enable gpo update with addition to build system
Split from "Initial commit for GPO work done by Luke Morrison" by Garming Sam

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Luke Morrison <luke@hubtrek.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-20 21:41:14 +01:00
Andrew Bartlett
377c0681e1 gpoupdate: Remove developer path from the comment
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-11-20 21:41:14 +01:00
Andrew Bartlett
a6ea6828f3 gpoupdate: Correct comment about hard-coded 5 second runing of the script
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-11-20 21:41:14 +01:00
Andrew Bartlett
2e432ef21e gpoupdate: Do not DEBUG(0) every scan interval
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-11-20 21:41:14 +01:00
Garming Sam
5662e49b49 gpo: Create the gpo update service
Split from "Initial commit for GPO work done by Luke Morrison" by David Mulder

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Luke Morrison <luke@hubtrek.com>
Signed-off-by: David Mulder <dmulder@suse.com>

Then adapted to current master

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
2017-11-20 21:41:14 +01:00
David Mulder
115615d836 gpo: Make the gpoupdate script much more reliable
Using a static file blanks the file when samba_gpoupdate crashes. Transformed
to a tdb file and added transactions. Add info logging to monitor gpo changes,
etc. Also handle parse errors and log an error message, then recover. Modified
the parsing code to use ConfigParser. Also, use the backslash in path names
when opening smb files, otherwise it fails against a windows server.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-20 21:41:14 +01:00
Luke Morrison
5194cd4e8d gpo: Initial commit for GPO work
Enclosed is my Summer of Code 2013 patch to have vital password GPO always applied to the Samba4 Domain Controller using a GPO update service.

To try it out "make -j" your samba with the patch, apply a security password GPO and see the difference in ~20 seconds. It also takes GPO hierarchy into account.

Split from "Initial commit for GPO work done by Luke Morrison" by David Mulder

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Luke Morrison <luke@hubtrek.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-20 21:41:14 +01:00
Andrew Bartlett
a80296b12e waf: Move script list to one-per-line
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-11-20 21:41:14 +01:00
Jeremy Allison
6c0d053ec0 s4: torture: Ensure kernel oplock test can't hang in pause().
Use an alarm to break out of waiting for a signal.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13121

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Nov 16 22:27:06 CET 2017 on sn-devel-144
2017-11-16 22:27:06 +01:00
Volker Lendecke
2e5ea35f0b smbtorture: Remove an unused variable
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Nov 14 03:55:37 CET 2017 on sn-devel-144
2017-11-14 03:55:37 +01:00
Volker Lendecke
a1b2daa06f dreplsrv: Use is_null_sid
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-11-13 23:54:47 +01:00
Jeremy Allison
ad82557e13 s4: torture: kernel oplocks. Add smb2.kernel-oplocks.kernel_oplocks8
Test if the server blocks whilst waiting on a kernel lease held by
a non-smbd process.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13121

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Sat Nov 11 20:12:26 CET 2017 on sn-devel-144
2017-11-11 20:12:26 +01:00
Jeremy Allison
15597a95ec s4: torture: kernel_oplocks. Create a regression test case for bug #13058.
It implements the following test case:

1. client of smbd-1 opens the file and sets the oplock.
2. client of smbd-2 tries to open the file. open() fails(EAGAIN) and open is deferred.
3. client of smbd-1 sends oplock break request to the client.
4. client of smbd-1 closes the file.
5. client of smbd-1 opens the file and sets the oplock.
6. client of smbd-2 calls defer_open_done(), sees that the file lease was not changed
			and does not reschedule open.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13058

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
2017-11-10 23:27:10 +01:00
Lumir Balhar
e00ba05d33 python: Port ntvfs posix bindings to Python 3 compatible form
Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-11-08 17:57:21 +01:00
Ralph Boehme
20d3ae6a45 librpc/idl: rename NFS4 ACL xattr name define
No change in behaviour.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-11-08 00:20:08 +01:00
Ralph Boehme
f3f119e456 selftest: split out failing owner related subtest from samba3.raw.acls.create_file|dir
All the other subtests in samba3.raw.acls.create_file|dir pass with
nfs4acl_xattr, it's just the subtest that tries to set the owner which
fails with everything else then acl_xattr.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-11-08 00:20:07 +01:00
Ralph Wuerthner
44c018bdcc s4: torture: Add smb2 FIND_and_set_DOC test case.
Regression tests doing an SMB2_find followed by
a set delete on close and then close on a directory.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13118

Signed-off-by: Ralph Wuerthner <ralph.wuerthner@de.ibm.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sun Nov  5 12:31:12 CET 2017 on sn-devel-144
2017-11-05 12:31:12 +01:00
Andrew Bartlett
dc3adc898e s4-smbtorture: Show that the KDC provides no protection from CVE-2017-11103
The server name in the AS-REQ is unprotected, sadly.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12894

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Nov  2 07:16:50 CET 2017 on sn-devel-144
2017-11-02 07:16:50 +01:00
Andrew Bartlett
4d056974dd s4-smbtorture: Add test krb5.kdc to prove fix for CVE-2017-11103
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12894

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-11-02 03:17:38 +01:00
Günther Deschner
910b0ce2a2 s4-torture: remove obsolete comment in libsmbclient torture suite.
Since smbc_setX calls now handle string allocation using malloc
themselves (since commit 2d41b1ab78) we
indeed no longer need to provide malloced strings (the extra malloc
already got removed earlier).

Guenther

Signed-off-by: Günther Deschner <gd@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Oct 30 21:09:14 CET 2017 on sn-devel-144
2017-10-30 21:09:14 +01:00
Andrew Bartlett
c3aa8809e1 repl_meta_data: Initialise parent_dn to NULL
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Mon Oct 30 04:16:42 CET 2017 on sn-devel-144
2017-10-30 04:16:42 +01:00
Andrew Bartlett
cdb42eae89 repl_meta_data: Explain that we do not truncate the DN at present
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-30 00:16:39 +01:00
Andrew Bartlett
68de8c66c7 repl_meta_data: Use replmd_make_prefix_child_dn() in replmd_conflict_dn()
Now both routines avoid the escape/unescape implicit in ldb_dn_add_child_fmt()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-30 00:16:39 +01:00
Andrew Bartlett
5364f8d3d5 repl_meta_data: Split replmd_make_deleted_child_dn() into a helper function
This will allow it to be used in common with replmd_conflict_dn()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-30 00:16:39 +01:00
Andrew Bartlett
5eff04e9b8 repl_meta_data: Move creation of deleted DN into helper: replmd_make_deleted_child_dn()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-30 00:16:39 +01:00
Andrew Bartlett
b838c14b1b repl_meta_data: Avoid printf() and use binary direct RDN creation for deleted objects
This makes it clearer that we are just replacing the RDN and ensures we do not
somehow create multiple components inside ldb_dn_add_child_fmt().

We also avoid an escape/un-escape round-trip.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-30 00:16:39 +01:00
Volker Lendecke
6475293bc8 samba: Fix CID 1420179 Code maintainability issues UNUSED_VALUE
I don't think pid is used at all here.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-10-27 20:33:25 +02:00
Volker Lendecke
e7a4c31047 samba: Fix CID 1420180 Null pointer dereferences
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-10-27 20:33:25 +02:00
Andreas Schneider
e56626e864 s4:pyparam: Fix resource leaks on error
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13101

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-10-27 20:33:25 +02:00
Andreas Schneider
b012f1589b s4:torture: Avoid useless strdup in libsmbclient test
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13101

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-10-27 20:33:25 +02:00
Andreas Schneider
ab44be7a27 s4:kdc: Pass down the task to get access to model_ops for kpasswd server
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-10-27 11:29:12 +02:00
Douglas Bagnall
30584a278a linked attribute tests: correct add_all_at_once test
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Oct 26 05:36:11 CEST 2017 on sn-devel-144
2017-10-26 05:36:11 +02:00
Douglas Bagnall
1e62bbaeac linked_attribute tests: helper assert function for expected LdbError
The logic involved in asserting that a function raises an LdbError with
a particular error value has shown itself to be too complicated for me
to repeat too often.

To test this function, you would want a put a test in a bit like this:

    def test_assertRaisesLdbError(self):
        for i in [1, 2, ldb.ERR_ENTRY_ALREADY_EXISTS, 999]:
            def f(*args, **kwargs):
                raise ldb.LdbError(i, 'msg %s' % i)
            self.assertRaisesLdbError(i, 'a message', f, 'la la', la='la')

            def f2(*args, **kwargs):
                raise ldb.LdbError(i + 1, 'msg %s' % i)
            def f3(*args, **kwargs):
                pass
            for f in (f2, f3):
                try:
                    self.assertRaisesLdbError(i, 'a message', f, 'la la', la='la')
                except AssertionError as e:
                    print i, e, f
                    pass
                else:
                    print i, f
                    self.fail('assertRaisesLdbError() failed to fail!')

..but a self-testing test-tester is getting a too meta to run in every
autobuild.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-26 01:32:14 +02:00
Douglas Bagnall
0f0acb00ea replmd: use check_parsed_dn_duplicates() more widely
replmd_add_fix_la() was already making the same check; here we move it
a bit earlier.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-26 01:32:14 +02:00
Douglas Bagnall
b6294c84a5 linked attribute tests: fix logic for add test
We were ensuring that when we got an LdbError it was the right type,
but we weren't ensuring we got one at all.

The new test doesn't fail.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-26 01:32:14 +02:00
Douglas Bagnall
7cf3bbcc5c linked attribute tests: ensure duplicate deletes fail
We can't remove the same thing twice in the same message.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-26 01:32:14 +02:00
Douglas Bagnall
625e65d9f3 replmd: check for duplicate values in MOD_REPLACE case
Because we already have a sorted parsed_dn list, this is a simple
linear scan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-26 01:32:14 +02:00
Douglas Bagnall
046fc1f7de linked attribute tests: test against duplicates in replace
We should not be able to introduce duplicate links using MOD_REPLACE.
It turns out we could and weren't testing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-26 01:32:14 +02:00
Andreas Schneider
2d260b28f5 s4:scripting: Fix ntstatus_gen.h generation on 32bit
The hex() function results in different output on 32bit systems. It adds
a L for long for some numbers. Thus we have a different header file.

This patch makes sure we have a consistent file generation on different
paltforms.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13099

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>

Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Wed Oct 25 22:28:39 CEST 2017 on sn-devel-144
2017-10-25 22:28:39 +02:00
Lumir Balhar
806c1bcacd python: Port samba.messaging module to Python 3 compatible form.
Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-10-23 11:42:19 +02:00
Volker Lendecke
c3a1348371 xattr.idl: Don't generate an interface table
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sun Oct 22 21:40:16 CEST 2017 on sn-devel-144
2017-10-22 21:40:16 +02:00
Volker Lendecke
0bd5d7bcbe nfs4acls: Don't generate an interface table for nfs4acls.idl
Nobody uses the function nfs4acl_test.

It took a while to figure out how to get this to build. The "uuid" line in the
idl file triggers pidl to generate the function table entry, which in turn then
triggers tables.pl to register this interface
./bin/default/source4/librpc/gen_ndr/tables.c. We could for example do the same
with xattr_parse_DOSATTRIB. Nobody uses this.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-10-22 17:46:15 +02:00
Tim Beale
da8af833cf selftest: Print link meta-data when developer debugging is used
For Windows, DRS is the only way to see the RMD_VERSION of a link, or to
tell what inactive links the DC. Add some debug to display this
information. By default, this debug is turned off.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Oct 20 08:01:35 CEST 2017 on sn-devel-144
2017-10-20 08:01:35 +02:00
Tim Beale
9b3b09ce41 replmd: Remove unnecessary replmd_build_la_val() param
replmd_build_la_val() is creating a new link attribute. In this case,
the RMD_ORIGINATING_USN and RMD_LOCAL_USN are always going to be the
same thing, so we don't need to pass them in as 2 separate parameters.

This isn't required for any bug fix, but is just a general code
tidy-up.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
cd936a725c replmd: Get rid of duplicated replmd_build_la_val() code
replmd_build_la_val() and replmd_set_la_val() are pretty much identical.
Keep the replmd_build_la_val() API (as it makes it clearer we're
creating a new linked attribute), but replace the code with a call to
replmd_set_la_val().

This isn't required for any bug fix, but is just a general tidy-up to
avoid code duplication.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
4cb260f8c0 replmd: Fix RMD_VERSION inital value to match Windows
The initial value for RMD_VERSION is one on Windows. The MS-DRSR spec
states the following in section 5.11 AttributeStamp:

  dwVersion: A 32-bit integer. Set to 1 when a value for the attribute is
  set for the first time. On each subsequent originating update, if the
  current value of dwVersion is less than 0xFFFFFFFF, then increment it
  by 1; otherwise set it to 0

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
cef17ce4f0 replmd: Remove static values passed to replmd_build_la_val()
replmd_build_la_val() is used to populate a new link attribute value
from scratch. The version parameter is always passed in as the initial
value (zero), and deleted is always passed in as false.

For cases (like replication) where we want to set version/deleted to
something other than the defaults, we can use replmd_set_la_val()
instead.

This patch changes these 2 parameters to variables instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
499fa6b4a6 selftest: Add test for initial link attribute RMD_VERSION value
While testing link conflicts I noticed that links on Windows start from
a different RMD_VERSION compared to Samba. This adds a simple test to
highlight the problem.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
8319536565 replmd: Small refactor to replmd_check_singleval_la_conflict()
Now that the code is all in one place we can refactor it to make it
slightly more readable.

- added more code comments
- tweaked the 'no conflict' return logic to try to make what it's checking
  for more obvious
- removed conflict_pdn (we can just use active_pdn instead)
- added a placeholder variable and tweaked a parameter name

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
c83dffc6f1 replmd: Change replmd_check_singleval_la_conflict() logic flow
Return immediately if there's no conflict, which reduces nesting.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
841e724e29 replmd: Move link conflict handling into separate function
Link conflict handling is a corner-case. The logic in
replmd_process_linked_attribute() is already reasonably busy/complex.
Split out the handling of link conflicts into a separate function so
that it doesn't detract from the core replmd_process_linked_attribute()
logic too much.

This refactor should not alter functionality.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
82b56e63b5 replmd: Handle single-valued conflicts for an existing link
Currently the code only handles the case where the received link
attribute is a new link (i.e. pdn == NULL). As well as this, we need to
handle the case where the conflicting link already exists, i.e. it's a
deleted link that has been re-added on another DC.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
f36b2bb126 replmd: Mark link conflicts as inactive correctly
The previous patch to handle link conflicts was simply overriding the
received information and marking the link as deleted. We should be doing
this as a separate operation to make it clear what has happened, and so
that the new (i.e. inactive) link details get replicated out.

This patch changes it so that when a conflict occurs, we immediately
overwrite the received information to mark it as deleted, and to update
the version/USN/timestamp/originating_invocation_id to make it clear
that this is a new change.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
7649652b63 replmd: Use replmd_set_la_val() when adding new links
replmd_set_la_val() and replmd_build_la_val() are almost identical. When
we were processing the replicated link attributes we were calling one
function if the link was new, and a different one if the link existed.
I think we should be able to get away with using replmd_set_la_val() in
both cases.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
f183dcfad5 replmd: Fix talloc inconsistency in replmd_set_la_val()
All the other talloc_asprintf()s in this function use the mem_ctx, but
for some reason the vstring was using the dsdb_dn->dn. This probably
isn't a big deal, but might have unintentional side-effects.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
a607a3e83e replmd: Make replmd_set_la_val() closer to replmd_build_la_val()
These two functions are almost identical. The main difference between
them is the RMD_ADDTIME. replmd_set_la_val() tries to use the
RMD_ADDTIME of the old_dsdb_dn. Whereas replmd_build_la_val() always
uses the time passed in.

Change replmd_set_la_val() so it can accept a NULL old_dsdb_dn (i.e. if
it's a new linked attribute that's being set). If so, it'll end up using
the nttime parameter passed in, same as replmd_build_la_val() does.

Also update replmd_process_linked_attribute (which used to use
replmd_build_la_val()) to now pass in a NULL old_dsdb_dn. There
shouldn't be a difference in behaviour either way, but this exercises
the code change.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
f196897bc8 replmd: Handle conflicts for single-valued link attributes better
If 2 DCs independently set a single-valued linked attribute to differing
values, Samba should be able to resolve this problem when replication
occurs.

If the received information is better, then we want to set the existing
link attribute in our DB as inactive.

If our own information is better, then we still want to add the received
link attribute, but mark it as inactive so that it doesn't clobber our
own link.

This still isn't a complete solution. When we add the received attribute
as inactive, we really should be incrementing the version, updating the
USN, etc. Also this only deals with the case where the received link is
completely new (i.e. a received link conflicting with an existing
inactive link isn't handled).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
70d532a5c7 replmd: Partial fix for single-valued link conflict
This is the first part of the fix for resolving a single-valued link
conflict.

When processing the replication data for a linked attribute, if we don't
find a match for the link target value, check if the link is a
single-valued attribute and it currently has an active link. If so, then
use the active link instead.

This change means we delete the existing active link (and backlink)
before adding the new link. This prevents the failure in the subsequent
dsdb_check_single_valued_link() check that was happening previously
(because the link would end up with 2 active values).

This is only a partial fix. It stops replication from failing completely
if we ever hit this situation (which means the test is no longer
hitting an assertion when replicating). However, ideally the existing
active link should be retained and just marked as deleted (with this
change, the existing link is overwritten completely).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:20 +02:00
Tim Beale
20c0f3e1e9 selftest: Add conflict test where the single-valued link already exists
As well as testing scenarios where both variants of the link are new, we
should also check the case where the received link already exists on the
DC as an inactive (i.e. previously deleted) link.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:20 +02:00
Tim Beale
77abba5880 selftest: Add test for deleted single-valued link conflict
Currently we're only testing the case where the links have been modified
independently on 2 different DCs and both the links are active. We also
want to test the case where one link is active and the other is deleted.

Technically, this isn't really a conflict - the links involve different
target DNs, and the end result is still only one active link.

It's still probably worth having these tests to prove that fixing bug
13055 doesn't break anything.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:20 +02:00
Tim Beale
9c54e7484f selftest: Make sure single-link conflict retains the deleted link
There should only ever be one active value for a single-valued link
attribute. When a conflict occurs the 'losing' value should still be
present, but should be marked as deleted.

This change is just making the test criteria stricter to make sure that
we fix the bug correctly.

Note that the only way to query the deleted link attributes present
is to send a DRS request.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:20 +02:00
Tim Beale
c9ea47ec6b replmd: Remove unused originating_usn variable
The previous refactor makes it obvious that we aren't actually using
this variable for anything.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:20 +02:00
Tim Beale
21179febe8 replmd: Refactor logic to check if replicated link is newer
This is precursor work for supporting single-link conflicts.

Split out the code to check if the link update is newer. It's now safe
to call this from the main codepath. This also means we can combine the 2
calls to get the seqnum into a single common call.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:20 +02:00
Tim Beale
91951d869f replmd: Refactor adding the backlink in replmd_process_linked_attribute()
The code to add the backlink is the same in both the 'if' and the 'else'
case, so move it outside the if-else block.

(We're going to rework this block of code quite a bit in order to
support single-value linked attribute conflicts, aka bug #13055).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:20 +02:00
Andreas Schneider
5bfe93b4eb s4:smbd: Add missing unistd.h include to fix build of process_prefork
error: implicit declaration of function ‘getpgrp’; did you mean ‘getpt’?
[-Werror=implicit-function-declaration]

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-10-19 23:41:11 +02:00
Joe Guo
8ed3cac9e5 python: add a failed test to show Popen deadlock
`Popen.wait()` will deadlock when using stdout=PIPE and/or stderr=PIPE and the
child process generates large output to a pipe such that it blocks waiting for
the OS pipe buffer to accept more data. Use communicate() to avoid that.

This patch is commited to show the issue, a fix patch will come later.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-19 05:33:10 +02:00
Gary Lockyer
ea0cd977a3 source4/smbd: replace DEBUG( with DBG_
Update the debug logging to use the currently preferred debug macros

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-19 05:33:10 +02:00
Gary Lockyer
123042c2e3 source4/smbd: add a prefork process model.
Add a pre fork process model to bound the number processes forked by
samba.  Currently workers are only pre-forked for the ldap server,  all
the other services have pre-fork support disabled.

When pre-fork support is disabled a new process is started for each
service, and requests are processed by that process.

This commit partially reverts commit
b5be45c453.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-19 05:33:10 +02:00
Gary Lockyer
778e9a810e source4/smbd: Fix code formatting after refactoring.
Fix code formatting from the refactoring in the previous commits.
Done as a separate patch to make the changes to functionality easier
to review.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-19 05:33:10 +02:00
Gary Lockyer
e027871b0d process_standard: Do not log at level 2 every time a child exits
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-19 05:33:10 +02:00
Gary Lockyer
0840252670 source4/smbd: Do not overstamp the process model with "single"
Instead, except in RPC which is a special SNOWFLAKE, we rely on the struct
service_details in the init function.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-19 05:33:10 +02:00
Gary Lockyer
62d7bf9e0a process_standard: Honour proc_ctx->inhibit_fork_on_accept
This allows the service to control if it should fork per accept() without needing
to replace the whole process model with process_single.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-19 05:33:10 +02:00
Gary Lockyer
193afc55e6 process_standard: Move child pipe setup further down standard_accept_connection()
This avoids cleaning up on error from accept() but more importantly
allows a future mode that acts like process_single and so has no child.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-19 05:33:10 +02:00
Gary Lockyer
17d3a9d431 process_standard: Use the new process_context
Use the new process_context to control the from_parent_fd
This avoids the use of global variables, and will in the next patch
allow process_standard to run as what was known as single without
over-stamping a different process model.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-19 05:33:10 +02:00
Gary Lockyer
b852ad044b source4/smbd: refactor the process model for prefork
Refactor the process model code to allow the addition of a prefork
    process model.

    - Add a process context to contain process model specific state
    - Add a service details structure to allow service to indicate which
      process model options they can support.

    In the new code the services advertise the features they support to the
    process model.  The process model context is plumbed through to allow the
    process model to keep track of the supported options, and any state
    the process model may require.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-19 05:33:09 +02:00
Gary Lockyer
6d7a8d80cd tests: Add a blackbox test for smbcontrol
Add tests to check that samba processes have started and that they can be
pinged.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-19 05:33:09 +02:00
Lumir Balhar
87154bcfa9 python: Port samba.registry module to Python 3 compatible form
Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-10-18 10:20:26 +02:00
Tim Beale
4c9b380527 selftest: Add sanity-check RODC can't use cache to reveal secrets
Bug 12977 highlighted that Samba only checks exop GetNcChanges requests
once, when they're first received. This makes sense because valid exop
requests should only ever involve a single request. For regular
(non-exop) GetNcChanges requests, the server stores a cache of the
object GUIDs to return.

What we don't want to happen is for a malicious/compromised RODC to use
this cache to circumvent privilege checks, and receive secrets that it's
normally not permitted to access (e.g. the administrator's password).

The specific scenario we're concerned about is:
- The RODC sends a regular GetNcChanges request for all objects (without
  secrets). (This causes the server to build its GUID array cache).
- The RODC then sends a follow-on request for the next chunk, but sets
  the REPL_SECRET exop this time.

The only thing inadvertently preventing Samba from leaking secrets in
this case is updating msDS-RevealedUsers for auditing. It's possible
that a future code change may alter the codepath and open up a
security-hole without realizing. This patch adds a test case so if that
ever did happen, the selftests would detect the problem.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12977

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-14 07:28:18 +02:00
Ralph Boehme
5f52a0fbe8 s4/torture: vfs_fruit: test xattr unpacking
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13076

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Oct 13 21:44:02 CEST 2017 on sn-devel-144
2017-10-13 21:44:01 +02:00
Ralph Boehme
c5b25d40e1 s4/torture: vfs_fruit: replace AppleDouble data blob with xattr data
The osx_adouble_w_xattr datablob is used to test conversion from sidecar
._ file metdata to Samba compatible ._ file.

The previous data blob didn't contain xattr data, the new one does.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13076

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-10-13 17:40:07 +02:00
Stefan Metzmacher
2717263c14 s4:heimdal_build: there's no need to define HAVE_KRB5_ADDRESSES twice
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Björn Jacke <bjacke@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Oct 11 12:33:42 CEST 2017 on sn-devel-144
2017-10-11 12:33:42 +02:00
Samuel Cabrero
015bdbd10b s3: spoolss: Extend publish_toggle test to check returned GUID string format
Extend the rpc.spoolss.printer.addprinter.publish_toggle test to
check the format of the returned GUID string in GetPrinter info
level 7 structure.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12993

Signed-off-by: Samuel Cabrero <scabrero@suse.de>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: David Disseldorp <ddiss@suse.de>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Oct 11 06:39:00 CEST 2017 on sn-devel-144
2017-10-11 06:39:00 +02:00
Gary Lockyer
099df25f56 source4 smbd: remove global control pipe from process_standard.
The standard model uses a pipe to signal the worker processes spawned on
accept that the controlling process has terminated and that they should
shut down.  This pipe is currently a static global variable in
process_standard.c.

This patch replaces that global pipe with a file descriptor passed into
the process model init functions, giving  a single mechanism across all process
models.  This paves the way for the addition of a pre-fork process model.

Ensuring that the correct file descriptors are closed, is difficult so
it is best do this only once rather than require the process models to
do this individually.

Notes on debugging pipe ownership:

Add code to log the process id and the file descriptor of the writeable
pipe.

run:
   lsof | grep FIFO | grep samba | grep <process id>
   this will produce lines like:

   samba 25624 him 4w FIFO 0,10 0t0 472206 pipe

   where: 4w is the file descriptor and mode and the number to the left
          of "pipe" is the pipe id.
then:
   lsof | grep FIFO | grep samba | grep <pipe id>

   This will display all the processes with the pipe open and the mode
   only the smbd master process should have it open in write mode.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Thu Sep 28 02:08:34 CEST 2017 on sn-devel-144
2017-09-28 02:08:34 +02:00
Tim Beale
0d4c3e5e3f replmd: RMD_VERSION incorrectly incremented for link conflicts
This problem was noticed when 2 DCs added the same linked attribute at
roughly the same time. One DC would have a later timestamp than the
other, so it would re-apply the same link information. However, when it
did this, replmd_update_la_val() would incorrectly increment the
RMD_VERSION for the attribute. We then end up with one DC having a
higher RMD_VERSION than the others (and it doesn't replicate the new
RMD_VERSION out).

During replication RMD_VERSION is used to determine whether a linked
attribute is old (and should be ignored), or whether the information is
new and should be applied to the DB. This RMD_VERSION discrepancy could
potentially cause a subsequent linked attribute update to be ignored.

Normally when a local DB operation is performed, we just pass in a
version of zero and get replmd_update_la_val() to increment what's
already in the DB. However, we *never* want this to happen during
replication - we should always use the version we receive from the peer
DC.

This patch fixes the problem by separating the API into two:
- replmd_update_la_val(): we're updating a linked attribute in the DB,
  and so as part of this operation we always want to increment the
  version number (the version no longer need to be passed in because
  we can work it out from the existing DB entry).
- replmd_set_la_val(): we want to set a linked attribute to use the
  exact values we're telling it, including the version. This is what
  replication needs to use.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13038
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Sep 26 09:36:48 CEST 2017 on sn-devel-144
2017-09-26 09:36:48 +02:00
Tim Beale
5f93ac1f6f selftest: Add test for a re-animated object conflict
Added a test to simulate a user accidentally being deleted and 2
different admins trying to resolve the problem simultaneously - one by
re-animating the object and one by just creating a new object with
the same name.

Currently this test fails on Samba because it chooses the higher
version
number as the winner instead of the latest change.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13039

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-26 05:33:17 +02:00
Tim Beale
d278f5ea99 selftest: Windows resolves object conflicts differently to Samba
While testing link conflicts I noticed that Windows resolves conflicts
differently to Samba. Samba considers the version number first when
resolving the conflict, whereas Windows always takes the latest change.

The existing object conflict test cases didn't detect this problem
because they were both modifying the object the same number of times (so
they had the same version number).

I've added new tests that highlight the problem. They are basically the
same as the existing rename tests, except that only one DC does the
rename. Samba will always pick the renamed object as the winner, whereas
Windows picks the most recent change.

I've marked this test as a known fail for now.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13039

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-26 05:33:17 +02:00
Tim Beale
1b395f488a selftest: replica_sync did not fully cleanup if test failed
Normally the replica_sync tests do the cleanup at the end of the test
case, rather than in the tearDown(). However, if the tests don't run to
completion (because they fail), then the objects may not get cleaned up
properly, which causes the tests to fail on the 2nd test-env.

The problem is the object deletion only occurs on DC2 and it relies on
replication to propagate the deletion to DC1. Presumably this
propagation could be missed because the tests are repeatedly turning off
inbound replication on both DCs.

This patch changes the tearDown() so it tries to delete the objects off
both DCs, which appears to fix the problem.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-26 05:33:17 +02:00
Gary Lockyer
4aa1c09dbc s4:smbd: set samba root process title
Set the process title in the samba root process to clearly identify it
in ps output.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-09-26 00:41:16 +02:00
Gary Lockyer
4f84b6dfa8 s4:smbd: call setproctitle_init
Call setproctitle_init() in main which suppresses the
  "samba: setproctitle not initialized, please either call
   setproctitle_init() or link against libbsd-ctor."
messages, but more importantly it displays meaningful details in ps
output.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9816

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-26 00:41:16 +02:00
Tim Beale
8c56aa2c91 selftest: Rename ntlmauth tests to ntlmdisabled
There are already some existing ntlm_auth tests, so the new tests I've
added make things a bit confusing. Also, ntlmdisabled probably better
reflects the specific case we're trying to test.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-26 00:41:16 +02:00
Tim Beale
1a1c4ad71c selftest: Add new AD DC testenv with NTLM disabled
This is so that we test the source4 case as well. Currently the only
testenv with NTLM disabled is ktest, and that only exercises the source3
code.

I've tried to support the new test environment with minimal changes to the
Samba4.pm setup code.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-26 00:41:16 +02:00
Andrew Bartlett
7ddf47951b scripting: Add script (backportable) to undo a GUID index
This script allows the DB to be read, and re-indexed, by an earlier Samba version,
most likely 4.7 with some backported patches.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Sep 23 09:16:31 CEST 2017 on sn-devel-144
2017-09-23 09:16:31 +02:00
Andrew Bartlett
5d404eaeab Do not re-use the attribute @IDXVERSION for SAMDB_INDEXING_VERSION
Confusing these two concepts is not a good idea, SAMDB_INDEXING_VERSION refers to
a change in a Samba rule to canonicalise one of our attributes, not the
in-DB index format.

As we already change @INDEXLIST in this version, this commit
is at no extra cost.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-23 05:26:15 +02:00
Andrew Bartlett
effac54893 dsdb: Set that Samba uses the GUID index in LDB
This is optional, but only to aid the downgrade script (and in case
there is some major issue found with it).  We don't support that mode,
as that would require us to test and maintain multiple code paths and
not optimise queries to be GUID centric.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-23 05:26:15 +02:00
Andrew Bartlett
a205369697 provision: Add a fixed GUID to the samba4top objectclass definition
This is only used in the OpenLDAP backend and will certainly be removed before this becomes production.

(a production backend will use the real AD top objectclass)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-23 05:26:15 +02:00
Christof Schmitt
508aebf40a torture: Add testcase for compound CREATE-WRITE-CLOSE request
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13047

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-09-22 05:45:21 +02:00
Richard Sharpe
2d97c8a4a5 Make sure smbtorture tests can run if someone has set their min protocol above NT1.
This code is SMB1 only, and already modifies
maxprotocol, so this change is appropriate.

Signed-off-by: Richard Sharpe <richard.sharpe@primarydata.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-09-20 22:48:15 +02:00
Andrew Bartlett
ee4418e73f dsdb: Only trigger a re-index once per @INDEXLIST modification
A modify of both @INDEXLIST and @ATTRIBUTES will still trigger two re-index passes
but that is a task for later.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9527

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Sep 20 12:29:49 CEST 2017 on sn-devel-144
2017-09-20 12:29:49 +02:00
Andrew Bartlett
9e9a8d8f88 s4-dnsserver: Check for too many DNS results
If we had this check in when the wildcard DNS tests were written, we would have
noticed that the name needed to be escaped (see previous commit).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12994
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-20 08:15:21 +02:00
Andrew Bartlett
c174702107 s4-dnsserver: Always encode user-supplied names when looking up DNS records
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12994

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-20 08:15:21 +02:00
Garming Sam
3e1870c26c kcc: Remove unused, untested KCC code
This code tries to implement the full KCC algorithm, but never
actually worked correctly.

Removing this doesn't affect the full-mesh KCC. This code only
attempted to calculate a graph using the "proper" algorithm, though it
neglected to write its results back into the database. The full-mesh
calculation occurs elsewhere.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Sep 20 06:28:07 CEST 2017 on sn-devel-144
2017-09-20 06:28:07 +02:00
Andrew Bartlett
dd53be2756 ldap_server: Plumb ldb error string from a failed connect to ldapsrv_terminate_connection()
However, do not plumb it to the client-seen error string, as it could contain server paths.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-20 02:25:30 +02:00
Andrew Bartlett
c1e41d489d samdb: Rework samdb_connect_url() to return LDB error code and an error string
This allows debugging of why the LDB failed to start up.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-20 02:25:30 +02:00
Günther Deschner
2c745cfac5 s4-torture: move lease break handler outside the lease testsuite.
Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>
Pair-Programmed-With: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Sep 19 09:36:40 CEST 2017 on sn-devel-144
2017-09-19 09:36:40 +02:00
Tim Beale
1541c50b37 selftest: Add some tests for linked attribute conflicts
Currently we have tests that check we can resolve object conflicts, but
these don't test anything related to conflicting linked attributes.
This patch adds some basic tests that checks that Samba can resolve
conflicting linked attributes.

This highlights some problems with Samba, as the following tests
currently fail:
- test_conflict_single_valued_link: Samba currently can't resolve a
  conflicting targets for a single-valued linked attribute - the
  replication exits with an error.
- test_link_deletion_conflict: If 2 DCs add the same linked attribute,
  currently when they resolve this conflict the RMD_VERSION for the
  linked attribute incorrectly gets incremented. This means the version
  numbers get out of step and subsequent changes to the linked attribute
  can be dropped/ignored.
- test_full_sync_link_conflict: fails for the same reason as above.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Mon Sep 18 09:56:41 CEST 2017 on sn-devel-144
2017-09-18 09:56:41 +02:00
Tim Beale
46c1f7bdee getncchanges.c: max_links calculation didn't work well in some cases
The max_links calculation didn't work particularly well if max_links was
set to a value lower than max_objects.

As soon as repl_chunk->object_count exceeded repl_chunk->max_links, the
chunk would be deemed full, even if there was only one link to send (or
even worse, no links to send). For example, if max_objects=100 and
max_links=10, then it would send back chunks of 10 objects (or 9 objects
and 1 link).

I believe the historic reason this logic exists is to avoid overfilling
the response message. It's hard to tell what the appropriate limit would
be because the total message size would depend on how many attributes
each object has.

I couldn't think of logic that would be suitable for all cases. I toyed
with the idea of working out a percentage of how full the message is.
However, adjusting the max_links doesn't really make sense when the
settings are small enough, e.g. max_objects=100 and max_links=100 is
never going to overfill the message, so there's no reason to alter the
values.

In the end I went with:
- If the user is using non-default values, just use those.
- In the default value case, just use the historic calculation

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:25 +02:00
Tim Beale
3c0d80d0c3 replmd: Avoid duplicated debug/warnings
We display warnings if a target object is missing but it's still OK to
continue the replication. Currently we need to check the target twice -
once to verify it when we first receive it, and once when we actually
commit it (we can't skip the 2nd check altogether because in the join
case, they could occur quite far apart).

One annoying side-effect is we get the same warning message coming out
twice in these special cases.

In the cases where we're checking the dsdb_repl_flags, we can actually
just bypass the verification checks for the target object (if it doesn't
exist we still continue anyway). This may save us a tiny bit of
unnecessary work.

For cross-partition links, we can limit logging these warnings to when
the objects are actually being committed. This avoids spurious warnings
in the join case (i.e. we receive the link before we receive the target
object's partition, but we have received all partitions by the time we
actually commit the objects).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:25 +02:00
Tim Beale
44ca84166e replmd: Allow missing targets if GET_TGT has already been set
While running the selftests, I noticed a case where DC replication
unexpectedly sends a linked attribute for a deleted object (created in
the drs.ridalloc_exop tests). The problem is due to the
msDS-NC-Replica-Locations attribute, which is a (known) one-way link.
Because it is a one-way link, when the test demotes the DC and deletes
the link target, there is no backlink to delete the link from the source
object.

After much debate and head-scratching, we decided that there wasn't an
ideal way to resolve this problem. Any automated intervention could
potentially do the wrong thing, especially if the link spans partitions.
Running dbcheck will find this problem and is able to fix it (providing
the deleted object is still a tombstone). So the recommendation is to
run dbcheck on your DCs every 6 months (or more frequently if using a
lower tombstone lifetime setting).

However, it does highlight a problem with the current GET_TGT
implementation. If the tombstone object had been expunged and you
upgraded to 4.8, then you would be stuck - replication would fail
because the target object can't be resolved, even with GET_TGT, and
dbcheck would not be able to fix the hanging link. The solution is to
not fail the replication for an unknown target if GET_TGT has already
been set (i.e. the dsdb_repl_flags contains
DSDB_REPL_FLAG_TARGETS_UPTODATE).

It's debatable whether we should add a hanging link in this case or
ignore/drop the link. Some cases to consider:
- If you're talking to a DC that still sends all the links last, you
  could still get object deletion between processing the source object's
  links and sending the target (GET_TGT just restarts the replication
  cycle from scratch). Adding a hanging link in this case would be
  incorrect and would add spurious information to the DB.
- Suppose there's a bug in Samba that incorrectly results in an object
  disappearing. If other DCs then remove any links that pointed to that
  object, it makes recovering from the problem harder. However, simply
  ignoring the link shouldn't result in data loss, i.e. replication won't
  remove the existing link information from other DCs. Data loss in this
  case would only occur if a new DC were brought online, or if it were a
  new link that was affected.
Based on this, I think ignoring the link does the least harm.

This problem also highlights that we should really be using the same
logic in both the unknown target and the deleted target cases.
Combining the logic and moving it into a common
replmd_allow_missing_target() function fixes the problem. (This also has
the side-effect of fixing another logic flaw - in the deleted object
case we would unnecessarily retry with GET_TGT if the target object was
in another partition. This is pointless work, because GET_TGT won't
resolve the target).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:25 +02:00
Tim Beale
278039ff78 getncchanges.c: Support GET_TGT better with large numbers of links
A source object can potentially link to thousands of target objects.
We have to be careful not to overfill the GetNCChanges response message
with more data than it's possible to send. We also don't want the client
to timeout while we're busy checking the linked attributes. The GET_TGT
support added so far is fairly dumb - this patch extends it to better
handle larger numbers of links.

To do so, this extends the repl_chunk usage so that it also works out if
the current chunk is full of links. Now as soon as the chunk is full of
either links or objects, we stop and send it back.

These changes now mean that we need to also check:
- that all the links for the last source object in the previous chunk
  have been sent, before we move on and send the next object. This only
  takes effect when immediate_link_sync is configured. It also means
  that a chunk in the middle of the replication cycle can now contain
  only links, and no objects.
- when GET_TGT is used, we only send back the links that we've verified
  the target object for. i.e. if we stop checking links because we timed
  out, we only send back the links whose targets were checked.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:25 +02:00
Garming Sam
7ba10844d1 getncchanges.c: Refactor to track more state using repl_chunk
To prepare GET_TGT to deal with a large number of links better, there
is now a 'repl_chunk' struct to help keep track of all the factors
relating to the current chunk of replication data (i.e. how many
objects/links we can send and how many we've already processed). This
means we can have a consistent way of working out whether the current
chunk is full (whether that be due to objects, links, or just too much
time taken).

This patch should not alter functionality. This is just a refactor to
add the basic framework, which will be used in the next patch.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-09-18 05:51:25 +02:00
Tim Beale
10df9f6bfd getncchanges.py: Add a multi-valued linked attribute test
Add a test where a source object links to multiple different targets.
First we do the replication without GET_TGT and check that the server
can handle sending a chunk containing only links (in the middle of the
replication). Then we repeat the replication forcing GET_TGT to be used.

To avoid having to create 1500 objects/links, I've lowered the 'max
link sync' setting on the vampire_dc testenv to 250.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:25 +02:00
Tim Beale
693e3adc1a getncchanges.py: Add a test for dropped cross-partition links
Samba would drop linked attributes that span partitions if it didn't
know about the target object. This patch adds a test that exposes the
problem.

I've re-used the code from the previous re-animation test to do this.
I've also added a very basic DcConnection helper class that basically
stores the connection state information the drs_base.py uses for
replication. This allows us to switch the DC we want to replicate from
easily. This approach could potentially be retro-fitted to some of the
existing test cases, as it allows us to test both the DRS client code
and server code at the same time.

Note this test case relates to the code change for commit
fae5df891c.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:24 +02:00
Tim Beale
9697190600 getncchanges.py: Add test for replicating reanimated objects
Reading between the lines, this scenario seems to be the main reason
that Microsoft added the GET_TGT flag. MS AD can handle getting links
for unknown targets OK, but if it receives links for a deleted/recycled
target then it would tend to drop the received links. Samba client also
used to drop the links if talking to a Microsoft DC (or a Samba server
with GET_TGT support).

The specific scenario is the client side already knows about a deleted
object. That object is then re-animated and used as the target for a
linked attribute. *Then* the target object gets updated again so it gets
sent in a later replication chunk to the linked attribute, i.e. the
client receives the link before it learns that the target object has
been re-animated.

In this test we're interested in particular at how the client behaves
when it receives a linked attribute for a deleted object. (It *should*
retry with GET_TGT to make sure the target is up-to-date. However, it
was just dropping the linked attribute).

To exercise the client-side, we disable replication, setup the
links/objects on one DC the way we want them, then force a replication
to the second DC. We then check that when we query each DC, they both
tell us about the links/objects we're expecting (i.e. no links got
lost).

Note that this wasn't a problem with older versions of Samba-to-Samba
because sending the links last guaranteed that the target objects were
always up-to-date.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:24 +02:00
Tim Beale
ed2fc52243 drs: Add basic GET_TGT support
This adds basic DRS_GET_TGT support. If the GET_TGT flag is specified
then the server will use the object cache to store the objects it sends
back. If the target object for a linked attribute is not in the cache
(i.e. it has not been sent already), then it is added to the response
message.

Note that large numbers of linked attributes will not be handled well
yet - the server could potentially try to send more than will fit in a
single repsonse message.

Also note that the client can sometimes set the GET_TGT flag even if the
server is still sending the links last. In this case, we know the client
supports GET_TGT so it's safe to send the links interleaved with the
source objects (the alternative of fetching the target objects but not
sending the links until last doesn't really make any sense).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:24 +02:00
Tim Beale
821094d50b getncchanges.py: Add tests for object deletion during replication
Add tests that delete the source and target objects for linked
attributes in the middle of a replication cycle.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:24 +02:00
Tim Beale
469aed088f getnc_exop.py: Extend EXOP_REPL_OBJ test case to use GET_TGT
We already check that when we use GET_ANC that we still only receive a
single object when EXOP_REPL_OBJ is used. This extends the test to also
check that only a single object is returned when GET_TGT is used.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:24 +02:00
Tim Beale
00b20c825c getncchanges.py: Add test for GET_ANC and GET_TGT combined
The code has to handle needing GET_ANC and GET_TGT in combination, i.e.
where we fetch the target object for the linked attribute and the target
object's parent is unknown as well. This patch adds a test case to
exercise this code path.

The second part of this test exercises GET_ANC/GET_TGT for an
incremental replication, where the objects are getting filtered by an
uptodateness-vector/HWM.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:24 +02:00
Tim Beale
6ec9ef2beb getncchanges.py: Add test for adding links during replication
We have identified a case where the Samba server can send linked
attributes but not the target object. In this case, the Samba DRS client
would hit the "Failed to re-resolve GUID" case in replmd and silently
discard the linked attribute.

However, Samba will resend the linked attribute in the next cycle
(because its USN is still higher than the committed HWM), so it should
recover OK. On older releases, this may have caused problems if the
first error resulting in a hanging link (which might mean the second
time it's processed it still fails to be added).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:24 +02:00
Tim Beale
af82bdefcc getncchanges.py: Add some GET_TGT test cases
test_repl_get_tgt:
- Adds 2 sets of objects
- Links one set to the other
- Changes the order so the target object comes last in the
  replication (which means the client has to use GET_TGT)
- Checks that when GET_TGT is used that we have received all target
  objects we need to resolve the linked attibutes
- Checks that we expect to receive the linked attributes *before*
  the last chunk is sent (by default, Samba sends all the links at
  the end, so this fails)
- Checks that we eventually receive all expected objects, and all
  links we receive match what is expected

test_repl_get_tgt_chain:
  This adds the linked attributes in a more complicated chain. We add
  300 objects, but the links for 100 objects will point to a linked
  chain of 200 objects.
  This was mainly to determine whether or not Windows follows the
  target object (i.e. whether it sends all the links for the target
  object as well). It turns out Windows maintains its own linked
  attribute DB, so it sends the links based on USN.

Note that the 2 testenvs fail for different reasons. promoted_dc fails
because it is sending all the linked attributes last. vampire_dc fails
because it doesn't support GET_TGT yet, so it sends the link before the
peer knows about the target object.

Note that to test against vampire_dc (rather than the ad_dc_ntvfs DC),
we need to send the GetNCChanges requests to DC2 instead of DC1.
I've left the DC numbering scheme as is, but I've addeed a test_ldb_dc
handle to drs_base.py - it defaults to DC1, but tests can override it
easily and still have everything work.

While running the new tests through autobuild, I noticed an intermittent
LDAP_ENTRY_ALREADY_EXISTS failure in the test setup(). This appears to
be due to a timing issue in the background replication between the
multiple testenvs. Adding some randomness so that the test base OU is
unique seems to avoid the problem.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:24 +02:00
Tim Beale
172eedc076 getnc_exop.py: Fix GET_TGT behaviour in DRS tests
The existing code never passed the more_flags parameter into the
actual getNCChanges request, i.e. _getnc_req10(). This meant the
existing GET_TGT tests effectively did nothing.

Passing the flag through properly means we have to now change the tests
as the DNs returned by Windows now include any target objects in the
linked attributes. These tests now fail against Samba (because it
doesn't support GET_TGT yet).

Also added comments to the tests to help explain what they are actually
doing.

Note that Samba and Windows can return the objects in different orders,
due to significant differences in their underlying DB implementations
(Windows stores links in a separate DB, so sends links ordered strictly
by USN, whereas Samba sends links based on the USN of the source
object). To make the test a fair comparison between Windows and Samba,
we need to use dn_ordered=False.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:24 +02:00
Gary Lockyer
af38d73b06 s4/smbd: set the process group.
Set the process group in the samba daemon, the --no-process-group option
allows this to be disabled.  The no-process-group option needs to be
disabled in self test.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Sep 18 04:39:50 CEST 2017 on sn-devel-144
2017-09-18 04:39:50 +02:00
Andreas Schneider
05169a6047 samba_upgradedns: When we setup the internal dns cleanup bind-dns dir
Make sure to remove everything from the bind-dns directory to avoid
possible security issues with the named group having write access to all
AD partions

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12957

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-09-16 00:57:25 +02:00
Andreas Schneider
8cf5c5f0fa samba_upgradedns: Print better hints after we migrated the config
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12957

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-09-16 00:57:25 +02:00
Andreas Schneider
aef2b915a2 samba_upgradedns: Change the group of the 'binddns dir' too
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12957

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-09-16 00:57:24 +02:00