1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-27 03:21:53 +03:00
Commit Graph

27 Commits

Author SHA1 Message Date
Tim Beale
2229f4620d drs_utils: Avoid invalid dereference of v8 requests
req.more_flags only exists for v10 requests, so we throw an exception if
we try to dereference that field on a v8 (or v5) request. Unfortunately,
we were checking that we support v10 *after* we had tried to access the
more_flags. This patch fixes up the order of the checks.

This may be a problem trying to replicate with an older Windows DC
(pre-2008R2), and was reported on the samba mailing-list at one point:
https://lists.samba.org/archive/samba/2018-June/216541.html

Unfortunately this patch doesn't help the overall situation at all (the
join will fail because we can't resolve the link target and we can't use
GET_TGT). But it now gives you a more meaningful error, i.e.

  ERROR(runtime): uncaught exception - (8639, "Failed to process 'chunk'
    of DRS replicated objects: DOS code 0x000021bf"
instead of:
  ERROR(<type 'exceptions.AttributeError'>): uncaught exception -
    'drsuapi.DsGetNCChangesRequest8' object has no attribute 'more_flags'

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Tim Beale <timbeale@samba.org>
Autobuild-Date(master): Tue Nov  6 07:15:33 CET 2018 on sn-devel-144
2018-11-06 07:15:33 +01:00
Tim Beale
63bfdb3c11 drs_utils: Fix some long lines
Tweak the code slightly to avoid some 80+ character lines.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-06 03:39:11 +01:00
Tim Beale
fdb6b86b8e drs_util: Improve memory usage when joining large DB
drs_Replicate.replicate() could consume a large amount of memory when
replicating a large DB. This is not a leak - the memory gets freed when
the function returns (i.e. once the partition is fully replicated).
However, while the partition is in the process of being replicated, it
accumulates memory for each replication chunk it receives. This can have
considerable overhead with 1000s of objects/links in the partition.

This was exhausting memory when joining a VM with 1Gb RAM to a DC with
25K users (average ~15 group memberships per user).

It seems that by storing a reference to something that's on the ctr's
talloc tree, it doesn't free up the memory for each ctr message (until
the function actually returns and req is destroyed).

With 10K users (and average 15 group memberships per user), .replicate()
consumed 211Mb of memory, according to talloc.report_full(). With this
patch, it goes down to just the current ctr message (1-2Mb).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Oct 17 08:56:42 CEST 2018 on sn-devel-144
2018-10-17 08:56:42 +02:00
Noel Power
2b4d2810eb s4/torture/drs: PY3 port for samba4.drs.replica_sync_rodc
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett abartlet@samba.org
2018-09-06 15:51:35 +02:00
Joe Guo
cabb299749 PEP8: fix E703: statement ends with a semicolon
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:30 +02:00
Joe Guo
95c36d825c PEP8: fix E401: multiple imports on one line
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:30 +02:00
Joe Guo
211c9a5f85 PEP8: fix E302: expected 2 blank lines, found 1
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:29 +02:00
Joe Guo
5d532543ab PEP8: fix E128: continuation line under-indented for visual indent
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:27 +02:00
Tim Beale
850bba4d32 drs_utils: Always set the GET_TGT flag for clone renames
The DCCloneAndRenameContext replication was a little inefficient, in
that it would essentially replicate the entire DB twice. This was due to
resolving the link targets - it finds a target object it doesn't know
about, so retries the entire replication again with the GET_TGT flag set
this time.

Normally, the repl_meta_data code will use the target object's GUID,
however, it can't do this for cross-partition links (if it hasn't
replicated the target partition yet). The repl_md code can normally
detect that the link is a cross-parition link by checking the base-DN,
however, this doesn't work in the DCCloneAndRenameContext case because
we have renamed the base-DN.

This is not a big deal - it just means extra work. However, because the
domains being backed up could potentially be quite large, it probably
makes sense to just always set the GET_TGT in the rename case and skip
this extra work.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-05 04:01:25 +02:00
Tim Beale
417fe47a91 drs_utils: Add infrastructure to support 'clone with rename'
Our end goal is to create a backup clone of a DB, but rename the
domain/realm so we can startup the backup DC without interferring with
the existing Samba network. The basic strategy to do this is to leverage
DRS replication - by renaming the first object in the partition, all
subsequent objects will automatically be renamed.

This patch adds the infrastructure to do this. I've used object
inheritance to handle the special case of renaming the partition
objects. This means the domain-rename special case doesn't really
pollute the existing DRS replication code. All it needs is a small
refactor to create a new 'process_chunk()' function that the new
sub-class can then override.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-07-03 10:39:14 +02:00
Noel Power
9650deee03 python/samba: Py2|Py3 compat, fix more missed exception tuple issues 2018-06-20 00:18:20 +02:00
Douglas Bagnall
b9a323a27c python/drs_utils: fix repeated typo
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sun Jun 10 21:41:33 CEST 2018 on sn-devel-144
2018-06-10 21:41:33 +02:00
Douglas Bagnall
4885937bf8 samba python libs: convert 'except X, e' to 'except X as e'
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-02-15 00:18:30 +01:00
Andrew Bartlett
dc48fa9822 drs repl: Only print raw DRS replication traffic at level 9
This can be sensitive even with the passwords still encrypted.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13017
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:27 +02:00
Tim Beale
f812c29d40 drs_utils: Add GET_TGT support to 'samba-tool drs replicate --local'
Update drs_Replicate.replicate() so it handles being passed the GET_TGT
flag (more_flags). To do this, we need to always use a v10 GetNCChanges
request (v8 and v10 are essentially the same except for the more_flags).

If the replicate_chunk() call into the C bindings throws an error, check
to see whether the error could be fixed by setting the GET_TGT flag, and
re-send the request if so.

Unfortunately because WERR_DS_DRA_RECYCLED_TARGET isn't documented with
the other AD error codes, I've left it hardcoded for now (Microsoft
should be fixing up their Docs).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
2017-08-18 06:07:12 +02:00
Tim Beale
4bd8467018 drs_utils: HWM in 'samba-tool drs replicate --local' always zero
The code to check for the 'repsFrom' highwatermark didn't have any
effect because the hwm variable was overwritten (initialized to all
zeroes) further down.

Using a zero HWM probably wouldn't have impacted functionality because
we were still correctly using the uptodatenessvector, which should
avoid a full replication.

This was introduced in commit e2ba17d26a, presumably by
accident.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-28 00:25:15 +02:00
Tim Beale
314b96e183 drs: support sync-forced for 'samba-tool drs replicate --local'
The sync-forced option wasn't being passed into the replication request
when the --local option was used. This meant if outbound replication
were disabled on the target DC, then the replicate --local command would
fail.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-28 00:25:14 +02:00
Andrew Bartlett
e2ba17d26a python: Move partial replication logic into drs_utils.py
This allows us to use this easily from join.py soon

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-02-23 11:36:21 +01:00
Bob Campbell
dc0c702f79 drs_utils: use a given highwatermark and uptodateness_vector in replicate()
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
2017-02-14 21:28:25 +01:00
Bob Campbell
8de09d491d drs_utils: return number of replicated objects and links in replicate()
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
2017-02-14 21:28:25 +01:00
Douglas Bagnall
2ad69f4bd0 getncchanges script: use library code, not copied functions.
These functions were duplicates. To be exact, the diff -ub between what
getncchanges had, and what drs_uitls now has is this:

|@@ -1,4 +1,5 @@
|-def do_DsBind(drs):
|+def drs_DsBind(drs):
|     '''make a DsBind call, returning the binding handle'''
|     bind_info = drsuapi.DsBindInfoCtr()
|     bind_info.length = 28
|@@ -32,7 +33,8 @@
|     bind_info.info.supported_extensions	|= drsuapi.DRSUAPI_SUPPORTED_EXTENSION_GETCHGREPLY_V7
|     bind_info.info.supported_extensions	|= drsuapi.DRSUAPI_SUPPORTED_EXTENSION_VERIFY_OBJECT
|     (info, handle) = drs.DsBind(misc.GUID(drsuapi.DRSUAPI_DS_BIND_GUID), bind_info)
|-    return handle
|+
|+    return (handle, info.info.supported_extensions)
|
|
| def drs_get_rodc_partial_attribute_set(samdb):
|@@ -43,7 +45,7 @@
|     attids = []
|
|     # the exact list of attids we send is quite critical. Note that
|-    # we do ask for the secret attributes, but set set SPECIAL_SECRET_PROCESSING
|+    # we do ask for the secret attributes, but set SPECIAL_SECRET_PROCESSING
|     # to zero them out
|     schema_dn = samdb.get_schema_basedn()
|     res = samdb.search(base=schema_dn, scope=ldb.SCOPE_SUBTREE,
|@@ -71,3 +73,4 @@
|     partial_attribute_set.attids         = attids
|     partial_attribute_set.num_attids = len(attids)
|     return partial_attribute_set

while the drs_utils code has changed in moving
drs_get_rodc_partial_attribute_set() out of the class.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-02-10 05:09:08 +01:00
Christian Ambach
dd25aa129b python/drs_utils: do not attempt to parse log level, use parsed value
The log level parameter can contain debug class specific entries.
Do not attempt to parse this as int, but use the values that the
debugging system already parsed

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9945
Signed-off-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2016-09-14 19:31:21 +02:00
Garming
b4d2e1016d drs: Send DRSUAPI_DRS_GET_ALL_GROUP_MEMBERSHIP by default
This flag is not implemented in Samba, however, on an RODC replicating from
Windows, failing to send this flag leaves out group memberships.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2016-06-16 04:40:14 +02:00
Jelmer Vernooij
0c24085317 Fix more pep8 issues in code I touched recently.
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Change-Id: I35f3204bdf5d00b3280d703427ded2fa2163a6f7
2014-10-14 06:44:06 +02:00
Jelmer Vernooij
bbaa739bbd Remove remaining instance of pep8 E211 (too many spaces before operator).
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Change-Id: I9af3bf582bba8fc1094addb12cd0a5ce04406b5b
2014-10-14 06:44:06 +02:00
Andrew Bartlett
a623359fb8 python/drs: Ensure to pass in the local invocationID during the domain join
This ensures (and asserts) that we never write an all-zero GUID as an invocationID
to the database in replPropertyMetaData.

Andrew Bartlett

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2013-09-19 12:25:41 -07:00
Jelmer Vernooij
87afc3aee1 Move python modules from source4/scripting/python/ to python/.
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Mar  2 03:57:34 CET 2013 on sn-devel-104
2013-03-02 03:57:34 +01:00