1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-14 19:24:43 +03:00

4218 Commits

Author SHA1 Message Date
Andrew Bartlett
c3aa8809e1 repl_meta_data: Initialise parent_dn to NULL
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Mon Oct 30 04:16:42 CET 2017 on sn-devel-144
2017-10-30 04:16:42 +01:00
Andrew Bartlett
cdb42eae89 repl_meta_data: Explain that we do not truncate the DN at present
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-30 00:16:39 +01:00
Andrew Bartlett
68de8c66c7 repl_meta_data: Use replmd_make_prefix_child_dn() in replmd_conflict_dn()
Now both routines avoid the escape/unescape implicit in ldb_dn_add_child_fmt()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-30 00:16:39 +01:00
Andrew Bartlett
5364f8d3d5 repl_meta_data: Split replmd_make_deleted_child_dn() into a helper function
This will allow it to be used in common with replmd_conflict_dn()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-30 00:16:39 +01:00
Andrew Bartlett
5eff04e9b8 repl_meta_data: Move creation of deleted DN into helper: replmd_make_deleted_child_dn()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-30 00:16:39 +01:00
Andrew Bartlett
b838c14b1b repl_meta_data: Avoid printf() and use binary direct RDN creation for deleted objects
This makes it clearer that we are just replacing the RDN and ensures we do not
somehow create multiple components inside ldb_dn_add_child_fmt().

We also avoid an escape/un-escape round-trip.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-30 00:16:39 +01:00
Douglas Bagnall
30584a278a linked attribute tests: correct add_all_at_once test
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Oct 26 05:36:11 CEST 2017 on sn-devel-144
2017-10-26 05:36:11 +02:00
Douglas Bagnall
1e62bbaeac linked_attribute tests: helper assert function for expected LdbError
The logic involved in asserting that a function raises an LdbError with
a particular error value has shown itself to be too complicated for me
to repeat too often.

To test this function, you would want a put a test in a bit like this:

    def test_assertRaisesLdbError(self):
        for i in [1, 2, ldb.ERR_ENTRY_ALREADY_EXISTS, 999]:
            def f(*args, **kwargs):
                raise ldb.LdbError(i, 'msg %s' % i)
            self.assertRaisesLdbError(i, 'a message', f, 'la la', la='la')

            def f2(*args, **kwargs):
                raise ldb.LdbError(i + 1, 'msg %s' % i)
            def f3(*args, **kwargs):
                pass
            for f in (f2, f3):
                try:
                    self.assertRaisesLdbError(i, 'a message', f, 'la la', la='la')
                except AssertionError as e:
                    print i, e, f
                    pass
                else:
                    print i, f
                    self.fail('assertRaisesLdbError() failed to fail!')

..but a self-testing test-tester is getting a too meta to run in every
autobuild.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-26 01:32:14 +02:00
Douglas Bagnall
0f0acb00ea replmd: use check_parsed_dn_duplicates() more widely
replmd_add_fix_la() was already making the same check; here we move it
a bit earlier.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-26 01:32:14 +02:00
Douglas Bagnall
b6294c84a5 linked attribute tests: fix logic for add test
We were ensuring that when we got an LdbError it was the right type,
but we weren't ensuring we got one at all.

The new test doesn't fail.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-26 01:32:14 +02:00
Douglas Bagnall
7cf3bbcc5c linked attribute tests: ensure duplicate deletes fail
We can't remove the same thing twice in the same message.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-26 01:32:14 +02:00
Douglas Bagnall
625e65d9f3 replmd: check for duplicate values in MOD_REPLACE case
Because we already have a sorted parsed_dn list, this is a simple
linear scan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-26 01:32:14 +02:00
Douglas Bagnall
046fc1f7de linked attribute tests: test against duplicates in replace
We should not be able to introduce duplicate links using MOD_REPLACE.
It turns out we could and weren't testing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-26 01:32:14 +02:00
Tim Beale
9b3b09ce41 replmd: Remove unnecessary replmd_build_la_val() param
replmd_build_la_val() is creating a new link attribute. In this case,
the RMD_ORIGINATING_USN and RMD_LOCAL_USN are always going to be the
same thing, so we don't need to pass them in as 2 separate parameters.

This isn't required for any bug fix, but is just a general code
tidy-up.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
cd936a725c replmd: Get rid of duplicated replmd_build_la_val() code
replmd_build_la_val() and replmd_set_la_val() are pretty much identical.
Keep the replmd_build_la_val() API (as it makes it clearer we're
creating a new linked attribute), but replace the code with a call to
replmd_set_la_val().

This isn't required for any bug fix, but is just a general tidy-up to
avoid code duplication.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
4cb260f8c0 replmd: Fix RMD_VERSION inital value to match Windows
The initial value for RMD_VERSION is one on Windows. The MS-DRSR spec
states the following in section 5.11 AttributeStamp:

  dwVersion: A 32-bit integer. Set to 1 when a value for the attribute is
  set for the first time. On each subsequent originating update, if the
  current value of dwVersion is less than 0xFFFFFFFF, then increment it
  by 1; otherwise set it to 0

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
cef17ce4f0 replmd: Remove static values passed to replmd_build_la_val()
replmd_build_la_val() is used to populate a new link attribute value
from scratch. The version parameter is always passed in as the initial
value (zero), and deleted is always passed in as false.

For cases (like replication) where we want to set version/deleted to
something other than the defaults, we can use replmd_set_la_val()
instead.

This patch changes these 2 parameters to variables instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
8319536565 replmd: Small refactor to replmd_check_singleval_la_conflict()
Now that the code is all in one place we can refactor it to make it
slightly more readable.

- added more code comments
- tweaked the 'no conflict' return logic to try to make what it's checking
  for more obvious
- removed conflict_pdn (we can just use active_pdn instead)
- added a placeholder variable and tweaked a parameter name

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
c83dffc6f1 replmd: Change replmd_check_singleval_la_conflict() logic flow
Return immediately if there's no conflict, which reduces nesting.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
841e724e29 replmd: Move link conflict handling into separate function
Link conflict handling is a corner-case. The logic in
replmd_process_linked_attribute() is already reasonably busy/complex.
Split out the handling of link conflicts into a separate function so
that it doesn't detract from the core replmd_process_linked_attribute()
logic too much.

This refactor should not alter functionality.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
82b56e63b5 replmd: Handle single-valued conflicts for an existing link
Currently the code only handles the case where the received link
attribute is a new link (i.e. pdn == NULL). As well as this, we need to
handle the case where the conflicting link already exists, i.e. it's a
deleted link that has been re-added on another DC.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
f36b2bb126 replmd: Mark link conflicts as inactive correctly
The previous patch to handle link conflicts was simply overriding the
received information and marking the link as deleted. We should be doing
this as a separate operation to make it clear what has happened, and so
that the new (i.e. inactive) link details get replicated out.

This patch changes it so that when a conflict occurs, we immediately
overwrite the received information to mark it as deleted, and to update
the version/USN/timestamp/originating_invocation_id to make it clear
that this is a new change.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
7649652b63 replmd: Use replmd_set_la_val() when adding new links
replmd_set_la_val() and replmd_build_la_val() are almost identical. When
we were processing the replicated link attributes we were calling one
function if the link was new, and a different one if the link existed.
I think we should be able to get away with using replmd_set_la_val() in
both cases.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
f183dcfad5 replmd: Fix talloc inconsistency in replmd_set_la_val()
All the other talloc_asprintf()s in this function use the mem_ctx, but
for some reason the vstring was using the dsdb_dn->dn. This probably
isn't a big deal, but might have unintentional side-effects.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
a607a3e83e replmd: Make replmd_set_la_val() closer to replmd_build_la_val()
These two functions are almost identical. The main difference between
them is the RMD_ADDTIME. replmd_set_la_val() tries to use the
RMD_ADDTIME of the old_dsdb_dn. Whereas replmd_build_la_val() always
uses the time passed in.

Change replmd_set_la_val() so it can accept a NULL old_dsdb_dn (i.e. if
it's a new linked attribute that's being set). If so, it'll end up using
the nttime parameter passed in, same as replmd_build_la_val() does.

Also update replmd_process_linked_attribute (which used to use
replmd_build_la_val()) to now pass in a NULL old_dsdb_dn. There
shouldn't be a difference in behaviour either way, but this exercises
the code change.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
f196897bc8 replmd: Handle conflicts for single-valued link attributes better
If 2 DCs independently set a single-valued linked attribute to differing
values, Samba should be able to resolve this problem when replication
occurs.

If the received information is better, then we want to set the existing
link attribute in our DB as inactive.

If our own information is better, then we still want to add the received
link attribute, but mark it as inactive so that it doesn't clobber our
own link.

This still isn't a complete solution. When we add the received attribute
as inactive, we really should be incrementing the version, updating the
USN, etc. Also this only deals with the case where the received link is
completely new (i.e. a received link conflicting with an existing
inactive link isn't handled).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:21 +02:00
Tim Beale
70d532a5c7 replmd: Partial fix for single-valued link conflict
This is the first part of the fix for resolving a single-valued link
conflict.

When processing the replication data for a linked attribute, if we don't
find a match for the link target value, check if the link is a
single-valued attribute and it currently has an active link. If so, then
use the active link instead.

This change means we delete the existing active link (and backlink)
before adding the new link. This prevents the failure in the subsequent
dsdb_check_single_valued_link() check that was happening previously
(because the link would end up with 2 active values).

This is only a partial fix. It stops replication from failing completely
if we ever hit this situation (which means the test is no longer
hitting an assertion when replicating). However, ideally the existing
active link should be retained and just marked as deleted (with this
change, the existing link is overwritten completely).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:20 +02:00
Tim Beale
c9ea47ec6b replmd: Remove unused originating_usn variable
The previous refactor makes it obvious that we aren't actually using
this variable for anything.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:20 +02:00
Tim Beale
21179febe8 replmd: Refactor logic to check if replicated link is newer
This is precursor work for supporting single-link conflicts.

Split out the code to check if the link update is newer. It's now safe
to call this from the main codepath. This also means we can combine the 2
calls to get the seqnum into a single common call.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:20 +02:00
Tim Beale
91951d869f replmd: Refactor adding the backlink in replmd_process_linked_attribute()
The code to add the backlink is the same in both the 'if' and the 'else'
case, so move it outside the if-else block.

(We're going to rework this block of code quite a bit in order to
support single-value linked attribute conflicts, aka bug #13055).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-10-20 04:05:20 +02:00
Gary Lockyer
b852ad044b source4/smbd: refactor the process model for prefork
Refactor the process model code to allow the addition of a prefork
    process model.

    - Add a process context to contain process model specific state
    - Add a service details structure to allow service to indicate which
      process model options they can support.

    In the new code the services advertise the features they support to the
    process model.  The process model context is plumbed through to allow the
    process model to keep track of the supported options, and any state
    the process model may require.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-10-19 05:33:09 +02:00
Tim Beale
0d4c3e5e3f replmd: RMD_VERSION incorrectly incremented for link conflicts
This problem was noticed when 2 DCs added the same linked attribute at
roughly the same time. One DC would have a later timestamp than the
other, so it would re-apply the same link information. However, when it
did this, replmd_update_la_val() would incorrectly increment the
RMD_VERSION for the attribute. We then end up with one DC having a
higher RMD_VERSION than the others (and it doesn't replicate the new
RMD_VERSION out).

During replication RMD_VERSION is used to determine whether a linked
attribute is old (and should be ignored), or whether the information is
new and should be applied to the DB. This RMD_VERSION discrepancy could
potentially cause a subsequent linked attribute update to be ignored.

Normally when a local DB operation is performed, we just pass in a
version of zero and get replmd_update_la_val() to increment what's
already in the DB. However, we *never* want this to happen during
replication - we should always use the version we receive from the peer
DC.

This patch fixes the problem by separating the API into two:
- replmd_update_la_val(): we're updating a linked attribute in the DB,
  and so as part of this operation we always want to increment the
  version number (the version no longer need to be passed in because
  we can work it out from the existing DB entry).
- replmd_set_la_val(): we want to set a linked attribute to use the
  exact values we're telling it, including the version. This is what
  replication needs to use.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13038
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Sep 26 09:36:48 CEST 2017 on sn-devel-144
2017-09-26 09:36:48 +02:00
Andrew Bartlett
5d404eaeab Do not re-use the attribute @IDXVERSION for SAMDB_INDEXING_VERSION
Confusing these two concepts is not a good idea, SAMDB_INDEXING_VERSION refers to
a change in a Samba rule to canonicalise one of our attributes, not the
in-DB index format.

As we already change @INDEXLIST in this version, this commit
is at no extra cost.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-23 05:26:15 +02:00
Andrew Bartlett
effac54893 dsdb: Set that Samba uses the GUID index in LDB
This is optional, but only to aid the downgrade script (and in case
there is some major issue found with it).  We don't support that mode,
as that would require us to test and maintain multiple code paths and
not optimise queries to be GUID centric.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-23 05:26:15 +02:00
Andrew Bartlett
ee4418e73f dsdb: Only trigger a re-index once per @INDEXLIST modification
A modify of both @INDEXLIST and @ATTRIBUTES will still trigger two re-index passes
but that is a task for later.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9527

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Sep 20 12:29:49 CEST 2017 on sn-devel-144
2017-09-20 12:29:49 +02:00
Garming Sam
3e1870c26c kcc: Remove unused, untested KCC code
This code tries to implement the full KCC algorithm, but never
actually worked correctly.

Removing this doesn't affect the full-mesh KCC. This code only
attempted to calculate a graph using the "proper" algorithm, though it
neglected to write its results back into the database. The full-mesh
calculation occurs elsewhere.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Sep 20 06:28:07 CEST 2017 on sn-devel-144
2017-09-20 06:28:07 +02:00
Andrew Bartlett
c1e41d489d samdb: Rework samdb_connect_url() to return LDB error code and an error string
This allows debugging of why the LDB failed to start up.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-20 02:25:30 +02:00
Tim Beale
3c0d80d0c3 replmd: Avoid duplicated debug/warnings
We display warnings if a target object is missing but it's still OK to
continue the replication. Currently we need to check the target twice -
once to verify it when we first receive it, and once when we actually
commit it (we can't skip the 2nd check altogether because in the join
case, they could occur quite far apart).

One annoying side-effect is we get the same warning message coming out
twice in these special cases.

In the cases where we're checking the dsdb_repl_flags, we can actually
just bypass the verification checks for the target object (if it doesn't
exist we still continue anyway). This may save us a tiny bit of
unnecessary work.

For cross-partition links, we can limit logging these warnings to when
the objects are actually being committed. This avoids spurious warnings
in the join case (i.e. we receive the link before we receive the target
object's partition, but we have received all partitions by the time we
actually commit the objects).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:25 +02:00
Tim Beale
44ca84166e replmd: Allow missing targets if GET_TGT has already been set
While running the selftests, I noticed a case where DC replication
unexpectedly sends a linked attribute for a deleted object (created in
the drs.ridalloc_exop tests). The problem is due to the
msDS-NC-Replica-Locations attribute, which is a (known) one-way link.
Because it is a one-way link, when the test demotes the DC and deletes
the link target, there is no backlink to delete the link from the source
object.

After much debate and head-scratching, we decided that there wasn't an
ideal way to resolve this problem. Any automated intervention could
potentially do the wrong thing, especially if the link spans partitions.
Running dbcheck will find this problem and is able to fix it (providing
the deleted object is still a tombstone). So the recommendation is to
run dbcheck on your DCs every 6 months (or more frequently if using a
lower tombstone lifetime setting).

However, it does highlight a problem with the current GET_TGT
implementation. If the tombstone object had been expunged and you
upgraded to 4.8, then you would be stuck - replication would fail
because the target object can't be resolved, even with GET_TGT, and
dbcheck would not be able to fix the hanging link. The solution is to
not fail the replication for an unknown target if GET_TGT has already
been set (i.e. the dsdb_repl_flags contains
DSDB_REPL_FLAG_TARGETS_UPTODATE).

It's debatable whether we should add a hanging link in this case or
ignore/drop the link. Some cases to consider:
- If you're talking to a DC that still sends all the links last, you
  could still get object deletion between processing the source object's
  links and sending the target (GET_TGT just restarts the replication
  cycle from scratch). Adding a hanging link in this case would be
  incorrect and would add spurious information to the DB.
- Suppose there's a bug in Samba that incorrectly results in an object
  disappearing. If other DCs then remove any links that pointed to that
  object, it makes recovering from the problem harder. However, simply
  ignoring the link shouldn't result in data loss, i.e. replication won't
  remove the existing link information from other DCs. Data loss in this
  case would only occur if a new DC were brought online, or if it were a
  new link that was affected.
Based on this, I think ignoring the link does the least harm.

This problem also highlights that we should really be using the same
logic in both the unknown target and the deleted target cases.
Combining the logic and moving it into a common
replmd_allow_missing_target() function fixes the problem. (This also has
the side-effect of fixing another logic flaw - in the deleted object
case we would unnecessarily retry with GET_TGT if the target object was
in another partition. This is pointless work, because GET_TGT won't
resolve the target).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:25 +02:00
Tim Beale
ed2fc52243 drs: Add basic GET_TGT support
This adds basic DRS_GET_TGT support. If the GET_TGT flag is specified
then the server will use the object cache to store the objects it sends
back. If the target object for a linked attribute is not in the cache
(i.e. it has not been sent already), then it is added to the response
message.

Note that large numbers of linked attributes will not be handled well
yet - the server could potentially try to send more than will fit in a
single repsonse message.

Also note that the client can sometimes set the GET_TGT flag even if the
server is still sending the links last. In this case, we know the client
supports GET_TGT so it's safe to send the links interleaved with the
source objects (the alternative of fetching the target objects but not
sending the links until last doesn't really make any sense).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-18 05:51:24 +02:00
Andrew Bartlett
24600e8e91 repl_meta_data: Show failing replicated entry in error code
This re-work of our LDIF printing avoids some of the privacy issue from
printing the full LDIF at level 4, while showing the entry that actually fails.

Instead, with e3988f8f74f4a11e8f26a548e0a33d20f4e863f7 we now print the DN
only at level 4, then the full message at 8.

With this patch on failure, we print the redacted failing message at 5.

While all of the DRS replication data is potentially sensitive
the passwords are most sensitive, and are now not printed unencrypted.

This discourages users from sending the full failing trace, as the
last entry is much more likely the issue.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-14 18:43:16 +02:00
Andrew Bartlett
5d9bb80a02 schema: Rework dsdb_schema_set_indices_and_attributes() db operations
Commit ec9b1e881c3eef503d6b4b311594113acf7d47d8 did not fully fix this.

There is no value in using dsdb_replace(), we are under the read lock
and replace just confuses things further.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13025

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-14 18:43:16 +02:00
Andrew Bartlett
8d8d31eb2b dsdb: Add missing \n to debug
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:27 +02:00
Andrew Bartlett
51289a6f9b debug: Add new debug class "drs_repl" for DRS replication processing
This is used in the client and in the server

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:27 +02:00
Andrew Bartlett
e3988f8f74 repl_meta_data: Re-work printing of replicated entries
This re-work of our LDIF printing avoids some of the privacy issue from
printing the full LDIF at level 4, while showing the entry that actually fails.

Instead, we print the DN only at level 4, then the full message at 8.

While all of the DRS replication data is potentially sensitive
the passwords are most sensitive, and are now not printed unencrypted.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:27 +02:00
Andrew Bartlett
7cfaf70694 linked_attributes: Use ldb_ldif_message_redacted_string() for consistency
This avoids printing un-encrypted secret values in logs, and while links are not likely
secret, this avoids a future copy and paste using ldb_ldif_message_string() again.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:27 +02:00
Andrew Bartlett
cc78de5581 repl_meta_data: Use ldb_ldif_message_redacted_string() to avoid printing secrets in logs
This avoids printing un-encrypted secret values in logs

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:27 +02:00
Andreas Schneider
3fa7c43ef7 s4:bind_dlz: Use the 'binddns dir' if possible
The code makes sure we are backwards compatible. It will first check if
we still have files in the private directory, if yes it will use those.

If the the file is not in the private directory it will try the binddns
dir.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
2017-09-05 23:58:20 +02:00
Andrew Bartlett
a5dbcbeeed password_hash: Make a common failure with "password hash gpg key ids" clearer
This drove me to strace before I understood what it really meant.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-08-30 10:48:20 +02:00
Andrew Bartlett
7fdeea0f30 dsdb: Add comment showing where the normal password rules are applied
This looks like a footnote, but is actually where the default password rules are applied.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-08-30 10:48:19 +02:00