1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-23 17:34:34 +03:00
Commit Graph

114493 Commits

Author SHA1 Message Date
Volker Lendecke
831ee63f54 lib: Add error checks in dom_sid_string_buf
Also, avoid casts by using PRIxxx macros

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-11-02 17:03:26 +01:00
Noel Power
537a26d6d1 tests/py/rodc_rwdc: Fix py2/py3 .next compat issues
Python 3 does not have .next(), which we rely on, change the
code slightly so it works in py2/py3 (using builtin next function)

Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Fri Nov  2 06:56:24 CET 2018 on sn-devel-144
2018-11-02 06:56:24 +01:00
Douglas Bagnall
78f5b6e399 s4/scripting/*: py3 compatible print
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-02 03:48:52 +01:00
Douglas Bagnall
1cf142c30a selftesthelpers: fix py3 tests with extra_path
If a test was supplied with extra_path, a PYTHONPATH= env variable was
prepended to the args list, but the py3_compatible clause assumed the
first args element was /usr/bin/python.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-02 03:48:52 +01:00
Douglas Bagnall
f17a77af46 python dbcheck: don't use mutable default args
In this code

def f(a, b=[]):
    b.append(a)
    return b

all single argument calls to f() will affect the same copy of b.

In the controls case, controls=None has the same effect as
controls=[].

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-02 03:48:52 +01:00
Gary Lockyer
28826ec49c WHATSNEW: KDC prefork support
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Fri Nov  2 03:47:57 CET 2018 on sn-devel-144
2018-11-02 03:47:57 +01:00
Gary Lockyer
325e928db9 kdc: Update debug calls
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
2018-11-01 23:49:25 +01:00
Gary Lockyer
7366de3310 source4 smbd kdc: allow the kdc to run in prefork
Modify the kdc to allow it to run in the prefork process model. The
task_init function has been split up and code moved into the post_fork
function.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
2018-11-01 23:49:25 +01:00
Gary Lockyer
99aea42520 source4 smdb: Add a post fork hook to the service API
Add a post fork hook to the service API this will be called:

 - standard process model
   immediately after the task_init.

- single process model
  immediately after the task_init

- prefork process model, inhibit_pre_fork = true
  immediately after the task_init

- prefork process model, inhibit_pre_fork = false
  after each service worker has forked. It is not run on the service
  master process.

The post fork hook is not called in the standard model if a new process
is forked on a new connection. It is instead called immediately after
the task_init.

The task_init hook has been changed to return an error code. This ensures
the post_fork code is only run if the task_init code completed successfully.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
2018-11-01 23:49:24 +01:00
Gary Lockyer
d6777a66c0 source4 smbd: Make the service_details structure constant.
Make the service_details structure a static const.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
2018-11-01 23:49:24 +01:00
Gary Lockyer
4731c33821 windbindd: reword error message
Reword the asprintf() out of memory message to make it clear where the
issue is.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
2018-11-01 23:49:24 +01:00
Tim Beale
775054afbe replmd: Single DB operation per source object during link processing
Move the source object checks and DB modify operation up a level, so we
only do them once per source object rather than once per link.

This allows LMDB joins to succeed with ~15,000 members in a group.
Previously LMDB would fail with the error:

 Failed to apply linked attribute change '(-30792) - MDB_MAP_FULL:
  Environment mapsize limit reached at ../lib/ldb/ldb_mdb/ldb_mdb.c:203'

Rewriting the same object ~15000 times seemed to completely fill up
the LMDB 8Gb buffer. Presumably this was because LMDB is 'copy on
write', so it was storing ~15,000 copies of the same object. Strangely,
we don't see this problem writing the backlinks (which this patch won't
have helped with at all, because that's modifying the target object).

Note uSNChanged was only being added to the msg once, so the code has
been modified to replace the usnChanged each time (i.e. remove it and
re-add it).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Nov  1 23:48:21 CET 2018 on sn-devel-144
2018-11-01 23:48:21 +01:00
Tim Beale
d54956fd51 replmd: Move talloc context one level up
Eventually we want to combine multiple link attributes, that apply to the
same source object, into a single DB 'modify' operation. This will mean
the memory context needs to hang around until we have performed the DB
operation (instead of allocating a temporary context for each link).

This patch moves the talloc context one level up, so a temp context gets
allocated for each link *group*, instead of for each link *attribute*.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 20:38:14 +01:00
Tim Beale
0ba028681d replmd: Split apart source/target checks for links
We've grouped the linked attributes by source-object. Next, we want to
avoid duplicated processing for the source object, i.e. we only need to
check the source object exists once, not once per link.

Before we can do this, we need to tease apart
replmd_extract_la_entry_details(), which is doing both source and target
object processing. Split out extracting the target DSDB-DN so that it's
done separately.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 20:38:14 +01:00
Tim Beale
628b9f71c7 replmd: Group together link attribute processing by source object
Instead of processing each link attribute one at a time, we want to
group them together by source object. This will mean we only have to
look-up the source object once, and only perform one DB 'modify'
operation. With groups with 1000s of members, this will help improve
performance.

This patch takes the first step of group together the links by
source-object. A new 'la_group' struct is added to help track what links
belong to the same source object. The la_list essentially becomes a
'list of lists' now.

Note that only related links *in the same chunk* are only grouped together.
While it is trivial to groups together links that span different
replication chunks, this would be a fairly insignificant efficiency gain,
but seems to have a fairly detrimental memory overhead, once you get
into groups with 10,000+ members.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 20:38:14 +01:00
Tim Beale
5528a4cce4 replmd: Add more debug for replicating links
During a join of a large DB, processing the linked attributes can take a
long time. The join hangs in 'Committing SAM database' for many minutes
with no indication of whether it's making progress or not.

This patch adds some extra debug to show how far through processing the
linked attributes we are, when there are many thousands of links.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 20:38:14 +01:00
Volker Lendecke
9b28d47b0d torture: Fix the clang build
It's used uninitialized if an early torture_assert fails

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Nov  1 17:34:31 CET 2018 on sn-devel-144
2018-11-01 17:34:31 +01:00
Tim Beale
e2ee595202 selftest: Remove unnecessary code for backup testenvs
setup_namespaces() already gets done for the backupfromdc's domain, so
this step is unnecessary for the restoredc and offlinebackupdc testenvs
(which are based off the backupfromdc's database).

The setup_namespaces() step is still necessary for the renamedc/labdc,
as these don't have the UPN/SPN suffixes for the new realm yet.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Nov  1 13:06:01 CET 2018 on sn-devel-144
2018-11-01 13:06:01 +01:00
Tim Beale
018ff496e4 dnsupdate: Skip kerberos step if use-file specified
If there's a problem in get_credentials() (getting the machine account
Kerberos credentials), then we fallback to use_samba_tool (essentially
ignoring use-file). However, there's no need to do this, as use-file
shouldn't require Kerberos credentials.

This was making bootstrapping issues starting a testenv harder to debug.
Obviously, Kerberos is dependent on DNS functioning correctly, but
running dnsupdate was also dependent on having a working Kerberos KDC.
In my case, the testenv had a bad krb5.conf file, but the problem
appeared as resolv-wrapper errors (due to a missing RESOLV_WRAPPER_HOSTS
file, which should've been generated by dnsupdate).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-Programmed-With: Garming Sam <garming@catalyst.net.nz>
2018-11-01 09:41:20 +01:00
Tim Beale
c35fb3a88b dnsupdate: Pass smb.conf through to samba-tool commands
If you call samba_dnsupdate with a --configfile option, this wasn't
passed through to the samba-tool commands the script tries to run.
Normally, samba_dnsupdate would only be run on the DC itself, so it
shouldn't be a big deal, however, this may be a problem if you install
the samba database into a non-default location (i.e. not
/usr/local/samba).

This patch passes through the smb.conf file, if one was specified.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 09:41:19 +01:00
Tim Beale
1451b696ee selftest: Add new customdc testenv that can load any backup-file
This adds a new testenv that can be used for sandpit/manual testing.
This testenv can be based off any backup-file that you like.

The main use case is large databases. Populating 1000s of users is
time-consuming (it can take hours to create a really large DB). Instead
of having to manually add users to the testenv every time you want to
try something, this allows you to populate the users just once, take a
backup/snapshot of the DB, and then spin up the backup multiple times.

In theory this testenv could be useful for other situations too, e.g.
dealing with a corrupted database, testing DB migration (e.g. 4.7 -->
4.8), or if (for some reason) you wanted to create a realistic
lab-domain within a testenv.

To run-up the testenv you need to specify a BACKUP_FILE environment
variable (the same way we specify the SELFTEST_TESTENV), e.g.
  BACKUP_FILE=/files/backup-10k-ad_dc.tar.bz2 \
    SELFTEST_TESTENV=customdc make testenv

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 09:41:19 +01:00
Douglas Bagnall
d29d2f2f70 pytests: allow blackbox subcommands without a shell
When given a list, it will use the list directly as an argument list,
avoiding shell-expansion and the intermediatory process.

This removes shell expansion trouble, and saves the machine a little
bit of work.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 09:41:19 +01:00
Douglas Bagnall
790acef279 samba-tool ntacl: remove unused imports and variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Nov  1 09:40:02 CET 2018 on sn-devel-144
2018-11-01 09:40:02 +01:00
Douglas Bagnall
5f19bf63dc python/tests/docs: remove unused import, variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:15 +01:00
Douglas Bagnall
eeb1e81e25 python/join: remove unused imports and variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 05:08:15 +01:00
Douglas Bagnall
d177797101 python/upgradehelpers: remove unused provision imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:15 +01:00
Douglas Bagnall
00eaea56f6 samba-tool processes: remove unused imports and variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:15 +01:00
Douglas Bagnall
77b9b5e89d samba-tool domain: remove unused variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:15 +01:00
Douglas Bagnall
4e19b69b4d py/tests/dcerpc_integer: remove dup tests
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:15 +01:00
Douglas Bagnall
5e79630822 autobuild: remove unused variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:15 +01:00
Douglas Bagnall
e37edb7639 python/tests/source: remove useless local variable
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:14 +01:00
Douglas Bagnall
cf3d35c8a9 python/tests/credentials: remove unused variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:14 +01:00
Douglas Bagnall
78f0860cff python/xattr: remove unused variable
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:14 +01:00
Douglas Bagnall
55aaa9c849 python/upgradehelpers: remove unused variable
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:14 +01:00
Douglas Bagnall
d7f26be93c samba-tool dbcheck: remove unused variable
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:14 +01:00
Douglas Bagnall
d5f1ea6077 dbchecker: remove unused variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-01 05:08:14 +01:00
Douglas Bagnall
202be5994e samba-tool spn: remove unused variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:14 +01:00
Douglas Bagnall
866d5b3cd8 samba-tool rodc: remove unused variable
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:14 +01:00
Douglas Bagnall
68a43a189e python/samba3: import passdb in the manner it is used
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:14 +01:00
Douglas Bagnall
1ca5bf32ad python/kcc/graph: import KCCError, which is used somewhere
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:13 +01:00
Douglas Bagnall
1a66eeb124 tests/rodc_rwdc: fix message format
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:13 +01:00
Douglas Bagnall
9055602e63 traffic_replay: Exception has no .message
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:13 +01:00
Douglas Bagnall
9c2447e2d8 py/tests/dcerpc_rpc: Py3 compat integer types
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:13 +01:00
Douglas Bagnall
aafe6bf96a waflib: fix syntax error in string format
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:13 +01:00
Douglas Bagnall
d4e0660354 waflib: add necessary imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:13 +01:00
Douglas Bagnall
7b1d878f4c python/remove_dc: fix Exception construction
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:13 +01:00
Douglas Bagnall
7fafd57998 python/remove_dc: fix formating syntax (missing %s)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:13 +01:00
Douglas Bagnall
7db224bdc2 provision/backend: LDAPBackendResult takes no creds
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:12 +01:00
Douglas Bagnall
71f91ccda4 provision/backend: fix formating syntax
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:12 +01:00
Douglas Bagnall
67c5ab1752 provision: fix string formatting (number of args)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:12 +01:00