1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-21 18:04:06 +03:00

35446 Commits

Author SHA1 Message Date
Tim Beale
d54956fd51 replmd: Move talloc context one level up
Eventually we want to combine multiple link attributes, that apply to the
same source object, into a single DB 'modify' operation. This will mean
the memory context needs to hang around until we have performed the DB
operation (instead of allocating a temporary context for each link).

This patch moves the talloc context one level up, so a temp context gets
allocated for each link *group*, instead of for each link *attribute*.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 20:38:14 +01:00
Tim Beale
0ba028681d replmd: Split apart source/target checks for links
We've grouped the linked attributes by source-object. Next, we want to
avoid duplicated processing for the source object, i.e. we only need to
check the source object exists once, not once per link.

Before we can do this, we need to tease apart
replmd_extract_la_entry_details(), which is doing both source and target
object processing. Split out extracting the target DSDB-DN so that it's
done separately.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 20:38:14 +01:00
Tim Beale
628b9f71c7 replmd: Group together link attribute processing by source object
Instead of processing each link attribute one at a time, we want to
group them together by source object. This will mean we only have to
look-up the source object once, and only perform one DB 'modify'
operation. With groups with 1000s of members, this will help improve
performance.

This patch takes the first step of group together the links by
source-object. A new 'la_group' struct is added to help track what links
belong to the same source object. The la_list essentially becomes a
'list of lists' now.

Note that only related links *in the same chunk* are only grouped together.
While it is trivial to groups together links that span different
replication chunks, this would be a fairly insignificant efficiency gain,
but seems to have a fairly detrimental memory overhead, once you get
into groups with 10,000+ members.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 20:38:14 +01:00
Tim Beale
5528a4cce4 replmd: Add more debug for replicating links
During a join of a large DB, processing the linked attributes can take a
long time. The join hangs in 'Committing SAM database' for many minutes
with no indication of whether it's making progress or not.

This patch adds some extra debug to show how far through processing the
linked attributes we are, when there are many thousands of links.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 20:38:14 +01:00
Volker Lendecke
9b28d47b0d torture: Fix the clang build
It's used uninitialized if an early torture_assert fails

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Nov  1 17:34:31 CET 2018 on sn-devel-144
2018-11-01 17:34:31 +01:00
Tim Beale
018ff496e4 dnsupdate: Skip kerberos step if use-file specified
If there's a problem in get_credentials() (getting the machine account
Kerberos credentials), then we fallback to use_samba_tool (essentially
ignoring use-file). However, there's no need to do this, as use-file
shouldn't require Kerberos credentials.

This was making bootstrapping issues starting a testenv harder to debug.
Obviously, Kerberos is dependent on DNS functioning correctly, but
running dnsupdate was also dependent on having a working Kerberos KDC.
In my case, the testenv had a bad krb5.conf file, but the problem
appeared as resolv-wrapper errors (due to a missing RESOLV_WRAPPER_HOSTS
file, which should've been generated by dnsupdate).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-Programmed-With: Garming Sam <garming@catalyst.net.nz>
2018-11-01 09:41:20 +01:00
Tim Beale
c35fb3a88b dnsupdate: Pass smb.conf through to samba-tool commands
If you call samba_dnsupdate with a --configfile option, this wasn't
passed through to the samba-tool commands the script tries to run.
Normally, samba_dnsupdate would only be run on the DC itself, so it
shouldn't be a big deal, however, this may be a problem if you install
the samba database into a non-default location (i.e. not
/usr/local/samba).

This patch passes through the smb.conf file, if one was specified.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 09:41:19 +01:00
Douglas Bagnall
1a66eeb124 tests/rodc_rwdc: fix message format
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:13 +01:00
Douglas Bagnall
9148ec7d65 s4/dsdb/pytest/sites: do not use variables out of scope
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:11 +01:00
Douglas Bagnall
b1ba193dd0 s4/scripting/pfm_verify: PY3: use compat.text_type
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:10 +01:00
Douglas Bagnall
b4fff0ac7d s4/scripting/samba_upgradedns: avoid .has_key()
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:10 +01:00
Douglas Bagnall
2e6fccbb93 s4/scripting/minschema: whitespace reformat and py3 compatible print
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:10 +01:00
Douglas Bagnall
f109fde362 s4/scripting/autoidl: py3 compatible except
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:10 +01:00
Douglas Bagnall
8532308883 s4/script/depfilter.py: use py3 compatible regex import
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:10 +01:00
Douglas Bagnall
6c1d605348 tests/python/sec_descriptor: safer use of super()
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:09 +01:00
Douglas Bagnall
4e694079e5 tests/python/notification: safer use of super()
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:09 +01:00
Douglas Bagnall
a6368ae6d4 s4/dsdb/pytest/ldap_schema: fix typo in docs
Commit 311727947799e896e05d644103c9db80a665de88 removed a duplicate
test but it removed the wrong one, leaving this dreadful typo.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:09 +01:00
Ralph Boehme
9d8751db64 s4:torture/vfs/fruit: add test "empty_stream"
One to rule them all: consistently test critical operations on all
streams relevant to macOS clients: the FinderInfo stream, the Resource
Fork stream and an arbitrary stream that macOS maps to xattrs when
written to on a macOS SMB server.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:21 +01:00
Ralph Boehme
aba502d440 s4:torture/vfs/fruit: add check_stream_list_handle()
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:21 +01:00
Ralph Boehme
64b28e4a94 s4:torture/util: add torture_smb2_open()
This seems to be missing: a simple wrapper to just open a file without
fancy options.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:20 +01:00
Ralph Boehme
7485007bb0 s4:torture/vfs/fruit: enable AAPL extensions in a bunch of tests
These tests check for macOS SMB server specific behaviour. They work
currently against Samba without enabling AAPL because in vfs_fruit we're
currently don't check whether AAPL has been negotiated in one place. A
subsequent commit will change that and this commit prepares for that
change.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:20 +01:00
Ralph Boehme
3195ff9547 s4:torture/vfs/fruit: write some data to a just created teststream
Doesn't currently make a difference, but this prepares for a later
change in vfs_fruit that will filter out empty streams (which is the
macOS behaviour).

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:20 +01:00
Ralph Boehme
b35d9e7402 s4:torture/vfs/fruit: expand test "setinfo eof stream"
o Adds checks verifying that after setting eof to 0 on a stream, a
  subsequent open gets ENOENT, before and after closing the handle that
  had been used to set eof to 0.

o Verify that a write to a handle succeeds after that handle has been
  used to set eof to 0 on a stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:20 +01:00
Ralph Boehme
c3cf09a079 s4:torture/vfs/fruit: update test "creating rsrc with read-only access" for newer macOS versions
While this operation failed against older macOS versions, it passes
against versions 10.12 and newer. Update the test accordingly, a
subsequent commit will then update our implementation.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:19 +01:00
Ralph Boehme
6f428607e3 s4:torture/vfs/fruit: expand existing vfs_test "null afpinfo"
This adds a check that a read on a seperate handle also sees the
previously created AFP_AfpInfo stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:19 +01:00
Ralph Boehme
0d9a80b6b2 s4:torture/vfs/fruit: expand existing test "setinfo delete-on-close AFP_AfpInfo" a little bit
Add a check that verifies a create on a stream gets
NT_STATUS_DELETE_PENDING after delete-on-close has been set.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:19 +01:00
Ralph Boehme
9cd9859dc1 s4:torture/vfs/fruit: update test "read open rsrc after rename" to work with macOS
macOS SMB server seems to return NT_STATUS_SHARING_VIOLATION in this
case while Windows 2016 returns NT_STATUS_ACCESS_DENIED.

Lets stick with the Windows error code for now in the Samba fileserver,
but let the test pass against macOS.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:19 +01:00
Ralph Boehme
f7551d8fb3 s4:torture/vfs/fruit: ensure a directory handle is closed in all code paths
Otherwise we get a sharing violation when running against Samba and
opening the directory a second time.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:19 +01:00
Ralph Boehme
125498861a s4:torture/vfs/fruit: update test "stream names" to work with macOS
o create the basefile before trying to create a stream on it, otherwise
  this fails on macOS

o write something to the stream, otherwise the stream is not listed as
  macOS hides 0-byte sized streams

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:19 +01:00
Ralph Boehme
dba9a4743f s4:torture/vfs/fruit: update test "SMB2/CREATE context AAPL" to work against macOS
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:18 +01:00
Ralph Boehme
1d4e50649b s4:torture/vfs/fruit: set share_access to NTCREATEX_SHARE_ACCESS_MASK in check_stream_list
Avoid sharing conflicts with other opens on the basefile.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:18 +01:00
Ralph Boehme
394d1fc9ac s4:torture/vfs/fruit: fix a few error checks in "delete AFP_AfpInfo by writing all 0"
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:18 +01:00
Ralph Boehme
242f42ef79 s4:torture/vfs/fruit: skip a few tests when running against a macOS SMB server
These tests are designed to test specific vfs_fruit functionality.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:18 +01:00
Ralph Boehme
6f022e6159 s4:torture: add test for AppleDouble ResourceFork conversion
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:17 +01:00
Gary Lockyer
aeef8b41fa dsdb group audit tests: log_membership_changes extra tests
Add extra tests to ensure better test coverage of log_membership_changes

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Oct 30 20:20:26 CET 2018 on sn-devel-144
2018-10-30 20:20:26 +01:00
Gary Lockyer
c952fc1273 dsdb group audit tests: check_version improve diagnostics
Change check_version to display the expected, actual along with the
line and name of the failing test, rather than the line in check_version

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-10-30 16:40:13 +01:00
Gary Lockyer
e297088714 dsdb group audit tests: check_timestamp improve diagnostics
Change check_timestamp to display the expected, actual along with the
line and name of the failing test, rather than the line in
check_timestamp.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-10-30 16:40:13 +01:00
Gary Lockyer
8420a4d0fd dsdb group audit: align dn_compare with memcmp
Rename the parameter names and adjust the  return codes from dn_compare
so that:
dn_compare(a, b) =>

LESS_THAN means a is less than b.
GREATER_THAN means a is greater than b.

Thanks to metze for suggesting the correct semantics for dn_compare

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13664

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-10-30 16:40:13 +01:00
Gary Lockyer
eeb4089daf dsdb group_audit: Test to replicate BUG 13664
The group audit code incorrectly logs member additions and deletions.

Thanks to metze for the debugging that isolated the issue, and for
suggesting the fix to dn_compare.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13664

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-10-30 16:40:13 +01:00
Andrew Bartlett
852e1db12b dsdb: Add comments explaining the limitations of our current backlink behaviour
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Oct 30 10:32:51 CET 2018 on sn-devel-144
2018-10-30 10:32:51 +01:00
Stefan Metzmacher
7a36cb30b7 s4:samldb: internally use extended dns while changing the primaryGroupID field
This is important, otherwise we'll loose the <SID=> component of the
linked attribute.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-30 07:30:21 +01:00
Stefan Metzmacher
0386307e34 s4:repl_meta_data: add support for DSDB_CONTROL_DBCHECK_FIX_LINK_DN_SID
This will be used by dbcheck in the next commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-30 07:30:21 +01:00
Stefan Metzmacher
1ef145d9d7 s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_la_replace()
This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-30 07:30:21 +01:00
Stefan Metzmacher
738b52eb08 s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_la_delete()
This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-30 07:30:21 +01:00
Stefan Metzmacher
70a306d0bd s4:repl_meta_data: add missing \n to a DEBUG message in replmd_modify_la_add()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-30 07:30:21 +01:00
Stefan Metzmacher
42e69a86ca s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_la_add()
This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-30 07:30:21 +01:00
Stefan Metzmacher
966c7febaf s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_handle_linked_attribs()
This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-30 07:30:20 +01:00
Stefan Metzmacher
f81771c859 blackbox/dbcheck-links: Test broken links with missing <SID=...> on linked attributes
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-30 07:30:20 +01:00
Stefan Metzmacher
bb9c9e49a5 s4:dsdb: add DSDB_CONTROL_DBCHECK_FIX_LINK_DN_SID oid
This will be used to fix missing <SID=> components in future.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-30 07:30:20 +01:00
Stefan Metzmacher
364ed537e0 testprogs/blackbox: add samba4.blackbox.test_primary_group test
This demonstrates the bug, that happens when the primaryGroupID
of a user is changed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-30 07:30:20 +01:00