1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-25 23:21:54 +03:00
samba-mirror/python/samba
Tim Beale fdb6b86b8e drs_util: Improve memory usage when joining large DB
drs_Replicate.replicate() could consume a large amount of memory when
replicating a large DB. This is not a leak - the memory gets freed when
the function returns (i.e. once the partition is fully replicated).
However, while the partition is in the process of being replicated, it
accumulates memory for each replication chunk it receives. This can have
considerable overhead with 1000s of objects/links in the partition.

This was exhausting memory when joining a VM with 1Gb RAM to a DC with
25K users (average ~15 group memberships per user).

It seems that by storing a reference to something that's on the ctr's
talloc tree, it doesn't free up the memory for each ctr message (until
the function actually returns and req is destroyed).

With 10K users (and average 15 group memberships per user), .replicate()
consumed 211Mb of memory, according to talloc.report_full(). With this
patch, it goes down to just the current ctr message (1-2Mb).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Oct 17 08:56:42 CEST 2018 on sn-devel-144
2018-10-17 08:56:42 +02:00
..
emulate emulate/traffic: apply new logger to replace print 2018-10-10 06:16:21 +02:00
gp_parse fdeploy_ini: Generalize the share name SIDs 2018-08-16 23:42:22 +02:00
kcc python/samba: changes needed for samba.tests.samba_tool.visualize PY2/PY3 2018-09-03 03:22:21 +02:00
netcmd samba-tool drs showrepl: do not crash if no dnsHostName found 2018-10-12 15:27:07 +02:00
provision python/samba/provision: PY3 port to run samba.tests.upgradeprovisionneeddc 2018-09-15 15:18:25 +02:00
samba3 PEP8: fix E713: test for membership should be 'not in' 2018-08-24 07:49:31 +02:00
subunit PEP8: fix E701: multiple statements on one line (colon) 2018-08-24 07:49:30 +02:00
tests netcmd: modify Command.get_logger to use get_samba_logger 2018-10-10 06:16:22 +02:00
web_server s4/web_server: PY3: port to python3 2018-09-27 01:54:27 +02:00
__init__.py PEP8: fix E713: test for membership should be 'not in' 2018-08-24 07:49:31 +02:00
colour.py python/colour: add colourizing and switch functions 2018-05-31 01:57:17 +02:00
common.py PEP8: fix E226: missing whitespace around arithmetic operator 2018-08-24 07:49:28 +02:00
compat.py python/samba: PY3 add compat function urllib_join to replace urllib.urljoin 2018-09-27 01:54:27 +02:00
dbchecker.py python/samba: extra ndr_unpack needs bytes function 2018-09-15 15:18:26 +02:00
descriptor.py PEP8: fix E302: expected 2 blank lines, found 1 2018-08-24 07:49:29 +02:00
dnsserver.py PEP8: fix E302: expected 2 blank lines, found 1 2018-08-24 07:49:29 +02:00
domain_update.py python/samba: py2/py3 compatability always decode result of b64encode 2018-05-12 21:38:17 +02:00
drs_utils.py drs_util: Improve memory usage when joining large DB 2018-10-17 08:56:42 +02:00
forest_update.py python: fix the build with python3. 2018-01-16 16:38:23 +01:00
getopt.py PEP8: fix E128: continuation line under-indented for visual indent 2018-08-24 07:49:27 +02:00
gp_ext_loader.py PEP8: fix W391: blank line at end of file 2018-08-24 07:49:31 +02:00
gp_sec_ext.py gpo: add unapply to the gp_sec_ext 2018-10-08 18:17:16 +02:00
gpclass.py gpo PEP8: balance whitespace around equals 2018-10-08 18:17:16 +02:00
graph.py PEP8: fix E303: too many blank lines (2) 2018-08-24 07:49:30 +02:00
hostconfig.py PEP8: fix E302: expected 2 blank lines, found 1 2018-08-24 07:49:29 +02:00
idmap.py PEP8: fix E302: expected 2 blank lines, found 1 2018-08-24 07:49:29 +02:00
join.py join: Support site detection when --server is specified 2018-09-26 07:49:18 +02:00
logger.py samba/logger: add logger module for python 2018-10-10 06:16:21 +02:00
mdb_util.py netcmd: domain backup offline command 2018-08-06 05:37:42 +02:00
ms_display_specifiers.py python/samba: port open to io.open for PY2/PY3 compat 2018-09-03 03:22:20 +02:00
ms_forest_updates_markdown.py PEP8: fix E305: expected 2 blank lines after class or function definition, found 1 2018-08-24 07:49:30 +02:00
ms_schema_markdown.py PEP8: fix E305: expected 2 blank lines after class or function definition, found 1 2018-08-24 07:49:30 +02:00
ms_schema.py python/samba: port open to io.open for PY2/PY3 compat 2018-09-03 03:22:20 +02:00
ndr.py PEP8: fix E302: expected 2 blank lines, found 1 2018-08-24 07:49:29 +02:00
ntacls.py pysmbd: add option to pass a session info to set_nt_acl() function 2018-10-11 10:28:18 +02:00
remove_dc.py python/samba: PY3 port for ridalloc_exop test to work 2018-09-15 15:18:26 +02:00
samdb.py python/samba/netcmd: changes for samab.tests.samba_tool.computer 2018-09-03 03:22:23 +02:00
schema.py PEP8: fix E127: continuation line over-indented for visual indent 2018-09-06 12:10:12 +02:00
sd_utils.py PEP8: fix E231: missing whitespace after ',' 2018-08-24 07:49:28 +02:00
sites.py PEP8: fix E128: continuation line under-indented for visual indent 2018-08-24 07:49:27 +02:00
subnets.py python/samba: Fix py2/3 relative module import issue 2018-09-03 03:22:20 +02:00
tdb_util.py PEP8: fix E302: expected 2 blank lines, found 1 2018-08-24 07:49:29 +02:00
upgrade.py PEP8: improve formatting around bit-wise OR ("|") 2018-08-24 07:49:32 +02:00
upgradehelpers.py python/upgradehelpers: use int not long for PY3 2018-10-10 06:16:22 +02:00
xattr.py PEP8: fix E128: continuation line under-indented for visual indent 2018-08-24 07:49:27 +02:00