1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-28 17:47:29 +03:00

106411 Commits

Author SHA1 Message Date
Douglas Bagnall
33a5b6a934 replmd: treat a zero GUID as not present in get_parsed_dns
This roughly follows the pattern in the 2009 commit
0d5d7f58473c989bff4 by the Andrews Tridgell and Bartlett, which dealt
with zero GUIDs in replmd_add_fix_la(). That function is about to use
get_parsed_dns() [see next commit], and the other users of
get_parsed_dns don't really want to see zero guids, so it is simpler
to test here.

This makes hitting the GUID_all_zero branch of parsed_dn_find() even
more unlikely.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:16 +01:00
Douglas Bagnall
5fa1b5cc79 replmd: keep links sorted in replmd_process_linked_attribute
This is where linked attributes get added during a replication.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:16 +01:00
Douglas Bagnall
762bde22ed replmd linked_attributes: maintain sorted links in replace
We use a merge-like algorithm, which gives us a slight algorithmic
improvement (O(m + n) vs O(m log(n) + n log(m))) and keeps the results
sorted.

Here's an example. There are existing links to {A C D* F*} where D*
and F* represent deleted links, and we want to replace them with {B C
E F}.

existing:       A     C  D* E  F*
                      |     |  |
replacements:      B  C     E  F

result:         A* B  C  D* E  F

This is what happens to each link:

A  gets deleted to A*.
B  gets added.
C  is retained, with possible extended DN changes.
D* stays in the list as a deleted link
E  is retained like C
F  is undeleted.

Backlinks are created in the case of B and F
The backlink for A is deleted
The backlinks are not changed for C and E or D* (D* has none)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:16 +01:00
Douglas Bagnall
edcfededd2 replmd linked attributes: use really_parse_trusted_dn everywhere
This function fills out the DN and GUID fields of an unparsed
parsed_dn struct, which was happening in a few other places already.

In some places the GUID was not being filled out, which would probably
cause problems if the sorted_links switch was turned on.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:16 +01:00
Douglas Bagnall
ecf6e13749 replmd: simplify and optimise replmd_modify_la_delete
With the old binary search, we didn't get a pointer to the found
value, just a yes or no answer as to its existence. That meant we
ended up searching in both directions to find the links to be deleted.
As a consequence we needed to parse out the GUID of every existing
link, even if it wasn't being deleted.

Here we do it in one pass.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:16 +01:00
Douglas Bagnall
95bc1e0640 replmd: rearrange nothing-to-delete logic
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:16 +01:00
Andrew Bartlett
976a213b18 repl_meta_data: linked attributes use DRS sort order
Links come over the wire as if sorted by memcmp() on the binary blobs,
not as sorted by GUID_compare(). Until a few patches ago, a newly
joined DC would have its linked attributes in the memcmp order. This
restores that behaviour.

This comparison could be made more efficient by storing the GUID in
the original state, but it does not seem to be a bottleneck.

Signed-off-by:    Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-02-09 03:17:16 +01:00
Douglas Bagnall
bd6b417179 replmd: rework replmd_modify_la_add to merge efficiently
Because both the list of added links and the list of existing links
are sorted, it is possible to interlace the two and obtain a merged
sorted list.

We avoid a great amount of talloc_realloc()ing by observing that the
merged list can't be longer than the sum of the two lists.

In the (common) case where there are many existing links but few being
added, we avoid parsing most of the existing link DNs and GUIDs if the
sorted_links feature flag is set.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:15 +01:00
Andrew Bartlett
06cca52d78 replmd linked attrs: fully parse dn for upgrade check
Elsewhere we use the dsdb_dn pointer as a flag indicating parsed-ness,
so we have to be consistent.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:15 +01:00
Andrew Bartlett
47efee074a replmd linked attributes: lazy parsing for trusted DNs
If we know that links from the database are in sorted order (via the
replmd_private->sorted_links flag), we can avoid actually parsing them
until it is absolutely necessary.

In many cases we are adding a single link to a long list. The location
of the single link is found via a binary search, so we end up parsing
log(N) DNs instead of N.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:15 +01:00
Andrew Bartlett
ffff8c847e replmd: Add placeholder sorted_links to struct replmd_private
This will be initialised to false (zero) by default and will later come
from the compatibleFeatures in @SAMBA_DSDB

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-02-09 03:17:15 +01:00
Douglas Bagnall
c1bceb2528 replmd: replmd_check_upgrade_links() needs to first parses DNs
Because we now load the dns with get_parsed_dns_trusted we have
to manually explode them in the upgrade tests.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:15 +01:00
Douglas Bagnall
33b2758c55 replmd: parsed_dn_find() finds insertion point as well as exact hit
This will allow us to maintain the list of links in sorted order.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:15 +01:00
Douglas Bagnall
83c4ad778e binsearch: make BINARY_ARRAY_SEARCH_GTE compare against a pointer
This is in preparation for improvements in our handling of linked
attributes where we make changes to the pointer in the process of
comparing it (for caching purposes).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:15 +01:00
Douglas Bagnall
8bdec7034e binsearch: clarify variable name in greater-than-or-equal search
The exact match variable was called "result" following the other
macros, which confused me for a moment.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:15 +01:00
Douglas Bagnall
3c9483f7f9 replmd: fix variable names in replmd_check_upgrade_links
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:15 +01:00
Douglas Bagnall
5f52a8fbe7 replmd: replmd_check_upgrade_links() only checks the first DN
This assumes the links (on an object in the database) are either all in
the old format or all in the new.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:15 +01:00
Andrew Bartlett
225fa430d1 replmd: pass replmd_private down to replmd_add_backlink()
This is not much saving, but we are soon going to need replmd_private
in the intermediate layers (e.g. replmd_modify_la_add).

Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-02-09 03:17:15 +01:00
Douglas Bagnall
10e8bc103a replmd: Fix some whitespace in repl_meta_data.c
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:15 +01:00
Douglas Bagnall
996aafdbff replmd: check whether list is already sorted in get_parsed_dns()
If they are we can avoid the sort.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:15 +01:00
Andrew Bartlett
a9e0e7a9ef selftest: Do not test for link ordering in tombstones_expunge test
By testing only for the DNs that are returned we do not change the strictness of
the test, because it is a test of the match rule which applies to the whole
object, not the returned values.

However, when this code asserted the returned order of the links, it prevents
us from changing this order.  This order was not deterministic across DCs
but as this test ran against an offline DB, it was able to assume a
particular order.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-02-09 03:17:14 +01:00
Douglas Bagnall
2fab7585e7 s4/linked_attribute tests: remove helper function unused parameter
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:14 +01:00
Douglas Bagnall
d3dec7a232 s4/linked_attribute tests: try adding linked attributes directly
Previously we have only added linked attributes using a modify.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:14 +01:00
Douglas Bagnall
b69f48e19b s4/linked_attribute tests: test with the relax control
We had a theory this caused problems. It didn't, but the tests are
still worthwhile.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:14 +01:00
Douglas Bagnall
6bee3faadd s4/linked_attribute tests: compare link lists in sorted order
This isn't functionally different[1] from the previous use of set(),
but it makes the error output easier to read.

[1] OK, it will also show duplicates, which we really don't expect and
would definitely want to see.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:14 +01:00
Douglas Bagnall
069c4adbc4 s4/linked_attribute tests: remove unused code
We don't test for sort order because we don't depend on it. So this
test was never used.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:14 +01:00
Douglas Bagnall
1647f0cedc s4/linked_attribute tests: add multiple links and replace tests
Also a "delete all" test.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:14 +01:00
Douglas Bagnall
35029286f0 s4/linked_attributes test: pep8 tidy-up, remove unused imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-09 03:17:14 +01:00
Bob Campbell
5c918e2942 torture/drs: expand test for DRSUAPI_DRS_GET_ANC
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Feb  9 03:16:09 CET 2017 on sn-devel-144
2017-02-09 03:16:09 +01:00
Stefan Metzmacher
5109e777f7 getncchanges: implement DRSUAPI_DRS_GET_ANC more correctly
The most important case is the combination of
DRSUAPI_DRS_CRITICAL_ONLY and DRSUAPI_DRS_GET_ANC.

With DRSUAPI_DRS_GET_ANC we need to make sure all ancestors
included even if they're not marked with
isCriticalSystemObject=TRUE.

I guess we still don't behave exactly as Windows, but it's much
better than before and fixes the initial replication if
someone moved the administrator account to an OU.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Pair-Programmed-With: Bob Campbell <bobcampbell@catalyst.net.nz>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:19 +01:00
Stefan Metzmacher
c61d0c8957 getncchanges: calculate getnc_state->min_usn calculation based on the uptodateness vector
This should improve initial replication of a fresh destination dsa with
a zero highwatermark.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:19 +01:00
Stefan Metzmacher
02f11b925c getncchanges: improve get_nc_changes_add_links() by checking uSNChanged
This will make a difference once we handle DRSUAPI_DRS_GET_ANC correctly.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:19 +01:00
Stefan Metzmacher
c31777a701 getncchanges: improve get_nc_changes_build_object() by checking uSNChanged
This will make a difference once we handle DRSUAPI_DRS_GET_ANC correctly.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Stefan Metzmacher
51386342d5 getncchanges: fix highest_usn off by one calculation in get_nc_changes_add_links()
highest_usn is the the highest usn the destination dsa already knows about.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Stefan Metzmacher
7d8c409792 getncchanges: remove unused c++ comments/code in getncchanges_collect_objects()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Garming Sam
1a328bf404 getncchanges: do not replicate links for non critical objects if DRSUAPI_DRS_CRITICAL_ONLY is set
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Pair-programmed-with: Bob Campbell <bobcampbell@catalyst.net.nz>

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Stefan Metzmacher
1e15cdaa01 getncchanges: don't process DRSUAPI_DRS_CRITICAL_ONLY for EXOPs
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Stefan Metzmacher
488eed6977 getncchanges: remember the ncRoot_guid on the getncchanges state
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Stefan Metzmacher
23e45b4938 getncchanges: pass struct ldb_message as const
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Stefan Metzmacher
e935a04afb getncchanges: only set nc_{object,linked_attributes}_count with DRSUAPI_DRS_GET_NC_SIZE
The main change is that we return 0 values if DRSUAPI_DRS_GET_NC_SIZE is not
present in order to get the same result as a Windows server in that case.

If DRSUAPI_DRS_GET_NC_SIZE is return the number of links we found so far
during the cycle in addition the number of objects returned in this cycle.
Both values doesn't match what Windows returns, but doing that
correctly and efficient is a task for another day.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Stefan Metzmacher
41bc007d49 torture/drs: remove pointless nc_object_count replication checks in test_link_utdv_hwm()
nc_object_count and nc_linked_attributes_count are only filled if
DRSUAPI_DRS_GET_NC_SIZE is requested. And they should contain
the total number. This is only useful for the initial replication.

Samba ignores DRSUAPI_DRS_GET_NC_SIZE currently but that will change in
the following commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Stefan Metzmacher
dd6d119c80 python/join: use DRSUAPI_DRS_GET_NC_SIZE for the initial replication
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Stefan Metzmacher
330ef9e572 python/join: set common replica_flags in dc_join.__init__()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Stefan Metzmacher
f5d3b863c7 drsuapi.idl: make drsuapi_DsGetNCChangesRequest10 [public]
This allows ndr_print to work.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Stefan Metzmacher
0c77567a4e drsuapi.idl: add drsuapi_DrsMoreOptions with DRSUAPI_DRS_GET_TGT
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:18 +01:00
Stefan Metzmacher
2ef7594eca s4:libnet: s/highestCommitedUSN/highestCommittedUSN
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:17 +01:00
Stefan Metzmacher
7888c250da s4:dsdb/repl: s/highestCommitedUsn/highestCommittedUSN
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-08 23:20:17 +01:00
Amitay Isaacs
ce9b72c17a ctdb-build: Install CTDB tests correctly from toplevel
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12547

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Feb  2 08:25:57 CET 2017 on sn-devel-144
2017-02-02 08:25:57 +01:00
Jeremy Allison
a24ba3e408 s3: VFS: vfs_streams_xattr.c: Make streams_xattr_open() store the same path as streams_xattr_recheck().
If the open is changing directories, fsp->fsp_name->base_name
will be the full path from the share root, whilst
smb_fname will be relative to the $cwd.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12546

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Feb  2 01:55:42 CET 2017 on sn-devel-144
2017-02-02 01:55:42 +01:00
Volker Lendecke
129bc58eee smbd: Fix "map acl inherit" = yes
Brown-Paper-Bag bug in f85c2a6852a. The assignment contains a self-reference
in get_pai_flags which I missed.

Fix an uninitialized read.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12551
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Feb  1 22:06:50 CET 2017 on sn-devel-144
2017-02-01 22:06:50 +01:00