IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
So that we don't dereference result->msgs[0] when it doesn't exist.
This can happen when the object has changed in such a way that it no
longer matches the original search query.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13951
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Wed Jun 19 08:16:39 UTC 2019 on sn-devel-184
For TDB databases, the new ORDERED_INTEGER type is disabled along with
repacking at format version 1 if GUID indexing is disabled, so all the new
database features are toggled together. This scheme doesn't work with
MDB because GUID indexing is mandatory when using MDB. However, a
downgrade path is still required so in a previous commit we added
a pack_format_override option which allows a downgrade script to force
the database to use an earlier packing format. But, the new
ORDERED_INTEGER type would still be present in MDB databases so this
patch reads the pack_format_override opaque and converts ORDERED_INTEGER
types in @ATTRIBUTES to INTEGER and doesn't write any indexes of that
type to @INDEXLIST. The @INDEXLIST will be refreshed later, on the first
transaction.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
To reduce the number of potential combinations of database features in
ldb, we want to link all new database features since 4.7. GUID indexing,
ordered integers, and pack format changes will all upgrade together.
This patch makes ordered integers only function if GUID indexing is
enabled. If GUID indexing is disabled, ORDERED_INTEGER will not be
written to @ATTRIBUTES and a syntax's index_format_fn will never be
used.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
We better print "... remote host [Unknown] SID [S-1-5-18] ..."
in 'dsdb_audit' message, this matches what we print for
'dsdb_json_audit'.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13916
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Ensure that the referrals returned in a search request use the same
scheme as the request, i.e. referrals recieved via ldap are prefixed
with "ldap://" and those over ldaps are prefixed with "ldaps://"
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12478
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri May 24 05:12:14 UTC 2019 on sn-devel-184
The callers moved to dsdb_class_by_cn_ldb_val() with
43aa546ecc in 2009.
Found by callcatcher
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This made it much harder to watch under wireshark and is not required (no password setting).
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
metadata.tdb was being locked during transactions, but not during read, and
we should ensure we take all our locks in order for consistency
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13950
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This changes variables that are not the responsiblity of this function, the unlock
implied by partition_del_trans() needs to be done carefully in the right spot.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This allows metadata.tdb to be locked in the correct place in
in the lock order, as partition_reload_if_required() implicitly
calls partition_lock_read().
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This test is run agianst multiple DCs in the same domain, so there can
be a race with replication. Therefore avoid using the same name twice
by adding a random suffix.
This is an improvement to a demonstrator for this bug in TDB:
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13952
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This behaviour is Samba-specific, we have not traditionally cut of responses at 1000
or so as Windows does, and we need to change that behaviour carefully.
This triggers this bug in TDB:
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13952
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Fix use after free detected by AddressSanitizer
AddressSanitizer: heap-use-after-free on address 0x61400026a4a0
at pc 0x7fd555c52f12 bp 0x7ffed7231180 sp 0x7ffed7231170
READ of size 1 at 0x61400026a4a0 thread T0
#0 0x7fd555c52f11 in ldb_should_b64_encode
../../lib/ldb/common/ldb_ldif.c:197
#1 0x7fd539dc9417 in dsdb_audit_add_ldb_value
../../source4/dsdb/samdb/ldb_modules/audit_util.c:491
#2 0x7fd539dc9417 in dsdb_audit_attributes_json
../../source4/dsdb/samdb/ldb_modules/audit_util.c:651
#3 0x7fd539dc6a7e in operation_json
../../source4/dsdb/samdb/ldb_modules/audit_log.c:305
The problem is that at the successful end of these functions
el->values is overwritten with new_values. However get_parsed_dns()
points p->v at the supplied el and it effectively gets used
as a working area by replmd_build_la_val(). So we must duplicate it
because our caller only called ldb_msg_copy_shallow().
The reason this matters is that the audit_log module is
above repl_meta_data in the stack, and tries to log the
ldb_message it saw after the reply (to include the error code).
If that ldb_message is changed it is not only misleading,
it can point to memory that has since gone away.
In this case the memory for the full extended DN in the
member attribute ended up on 'ac', a context lost by
the time repl_meta_data has finished processing.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13941
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 15 05:35:47 UTC 2019 on sn-devel-184
Lower down in this function new_values is assigned over el->values and is
filled in with the values of all the parsed DNs. Therefore it is the natural
talloc parent.
This will allow el->values to be allocated on tmp_ctx in the next commit for
a working area during the function call.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Fix use after free detected by AddressSanitizer
AddressSanitizer: heap-use-after-free on address 0x60f0002b2738
at pc 0x7f89b1a213b5 bp 0x7ffce9528810 sp 0x7ffce9528800
READ of size 8 at 0x60f0002b2738 thread T0
#0 0x7f89b1a213b4 in samldb_rename_search_base_callback
../../source4/dsdb/samdb/ldb_modules/samldb.c:4203
#1 0x7f89d3a0db4a in ldb_module_send_entry
../../lib/ldb/common/ldb_modules.c:793
#2 0x7f89b6f27356 in es_callback
../../source4/dsdb/samdb/ldb_modules/encrypted_secrets.c:1418
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13942
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
When filtering out multiple elements, we end up memmove()ing the same
elements many times over. It is simpler to not do that by keeping track
of how many elements we are keeping.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The bare memmove is not strictly safe at the end of the list.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
ldb_msg_new() is currently the same as talloc_zero(), but it might
not always be.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu May 9 23:54:35 UTC 2019 on sn-devel-184
The last caller was removed in s4-drs: GetNCChanges() to return correct (in AD-way) ATTIDs
(6a51afcfdb) by Kamen Mazdrashki in 2010
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Many parts of Samba use an empty attribute list in requests expecting
all attributes to be returned in the response, which is incorrect. This
patch corrects the instances found by current CI tests. Static analysis
and debugging will need to be done before changing ildap to the correct
semantics.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Mon May 6 05:45:55 UTC 2019 on sn-devel-184
Test for LDAP request with an empty attribute list. LDB responds with
no attributes, but LDAP responds with all attributes. Fix is attached
to the bug below but we can't push it upstream until we've found all
instances of incorrect empty attribute list usage in Samba.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13852
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The dsdb module stack can turn a simple search request into a
complicated tree of sub-queries that include attributes not originally
asked for and excluding those that were. The corresponding replies
might contain unrequested attributes or (for good reasons, according
to some module) hide requested ones. The entire stack is there to
meddle and that is what is does. Except *this* module. It just counts.
To understand dsdb performance it helps to have some idea what
requests and replies are flying too and fro. This module, when
inserted anywhere in the stack, counts the requests and replies
passing through and the attributes they contain. This data is stored
in on-disk tdbs in the private/debug directory.
The module is not loaded by default. To load it you need to patch the
source4/dsdb/samdb/ldb_modules/samba_dsdb.c and put "count_attrs"
somewhere in the module lists in the samba_dsdb_init() function. For
example, to examine the traffic between repl_meta_data and
group_audit_log, you would do something like this around line 316:
"subtree_delete",
"repl_meta_data",
+ "count_attrs",
"group_audit_log",
"encrypted_secrets",
and recompile. Samba will then write to a number of tdb files in the
debug directory as requests and replies pass through. A simple script
is included to read these files. Doing this:
./script/attr_count_read st/ad_dc/private/debug/debug/attr_counts_not_found.tdb
will print a table showing how often various attritbutes were
requested but not found (from the point of view of the module).
A more sophisticated version of the script is coming in the next
commit, but this one is included first because in its simplicity it
documents the storage format reasonably well. The tdb keys are
attribute names, and the values are uint32_t in machine native order.
When the module is included in the stack there will be a very small
decrease in performance.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We run the tests again, trying to modify as a normal user rather than
Administrator.
It turns out that we do not always return the same error code as
Windows, but in all these tests both Windows and Samba always return
some kind of error (as you might hope).
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Do we interpret these the same way as Windows? In many cases, no.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The test was presumably commented out because we fail it, and
known-failing it would have hidden the attr-too-short tests that it
was bundled with. If we disentangle them we can knwn-fail it, which
serves as a TODO list.
(passes against WIN2012R2).
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Found by csbuild.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 1 05:02:22 UTC 2019 on sn-devel-184
When we get here it is very likely that the attribute will not be
preserved, as the preserved ones should have had the flag set, but we
still end up loking through the whole list to confirm. With a binary
search, we end up looking at ~5 attributes to confirm.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Most (perhaps all) attributes that are in the "must not remove" list also
have the PRESERVEONDELETE bit set, and checking bits is much cheaper
than a linear search involving strcasecmp. If we check the bit first
we save work.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The new string conversion wrappers detect and flag errors
which occured during the string to integer conversion.
Those modifications required an update of the callees
error checks.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
Otherwise we may not be able to construct a working schema that's
required to apply the changes.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12204
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Currently schema info's revision field isn't incremented if relax
control is present. This is so that no increment is done during
provision, but we need the relax control in other situations where
the increment is desired, so we should use the provision control instead
to disable schema info update.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13799
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Currently schema info's revision field isn't incremented if relax
control is present. This is so that no increment is done during
provision, but we need the relax control in other situations where the
increment is desired. This patch adds a failing test to expose the
problem.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13799
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Adding ordered integer proto schema handling in kv index cache. This
allows ordered 64 bit integers to be used in cached fields like
@ATTRIBUTES
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Activating <= and >= mdb indexing in samba for int32 and int64 attributes by:
1. Adding index_format_fn to LDB_SYNTAX_SAMBA_INT32 in ldb_samba
2. Cloning the 64bit LDB_SYNTAX_INTEGER type as LDB_SYNTAX_ORDERED_INTEGER
3. Adding index_format_fn to the new type
4. Modifying LargeInteger use the new type in samba schema
5. Bumping the index version to trigger reindexing
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The "oLschema2ldif" program contained multiple bugs triggered by
malformed inputs:
* Iteration beyond list of recognized dsdb syntax OIDs when value wasn't
found (bug 9567)
* NULL pointer dereference when input didn't define a name
* Heap buffer overflows for unterminated token values
Tests are added to reproduce all identified bugs.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=9567
Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Then adapted to use ARRAY_SIZE() consistently as suggested by
metze.
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Apr 3 02:43:07 UTC 2019 on sn-devel-144
The current method require at least the same version as during the build. This is not needed.
Used symbols indicate the minimum is 1.2.0
Bug-Debian: http://bugs.debian.org/886333
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13880
Signed-off-by: Mathieu Parent <math.parent@gmail.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
The original code never dereferenced attrs and only added "*" if attrs
was NULL (not if attrs[0] was NULL).
This causes significant performance issues with the new paged_results
module introduced for 4.10 as the initial GUID search requests no
attributes. This GUID search turns into a search for "*" and ends up
allocating memory for the entire database.
This never appears to cause changes in the final result set, only
intermediate processing.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13836
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Mar 29 18:37:29 UTC 2019 on sn-devel-144
Cmocka test exposing LDB bug where a request with an empty attributes
list returns a response containing all attributes. The bug is in the
ACL module and will be fixed in the next commit.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13836
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The behavior of these functions upon errors depends on the implementation.
The GNU libc implementation seems to return a null hash, but others like
libxcrypt returns a invalid hash string '*0'.
Signed-off-by: Samuel Cabrero <scabrero@suse.de>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
We need a way to rename an object without updating the replication meta
data.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Use a temporary struct as a return value to make the compiler catch all
callers. If we just changed bool->ssize_t, this would just generate a
warning. struct sid_parse_ret will go away in the next commit
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The new unified versions have better debugging and ensure
that both functions continue to have the same control flow.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
The speed test, when it was introduced a few patches ago, was
deliberately slow so that we could see how much better the changes
were. It used 500 users, 50 groups, and 27 computers.
Before the changes, it took this long:
rename ou took 64.373s
rename group took 0.160s
rename user took 0.004s
rename computer took 0.123s
After using the sorted links, it took this long:
rename ou took 12.984s
rename group took 0.161s
rename user took 0.004s
rename computer took 0.122s
And with the final patch to stop the linear search early on success:
rename ou took 11.680s
rename group took 0.089s
rename user took 0.004s
rename computer took 0.128s
"rename ou" is the one we were aiming at. Now that we have done that,
we reduce the size of the test so as not to slow down everyone's
autobuilds.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
In most cases there can only be one link for each GUID. If we assume
that is true, we can skip half the search, on average.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Rename operations can be very slow in large database with many group
memberships, because the linked attributes need to be found and
rewritten for each moved object and the way we did that was naive.
For a while now Samba has kept forward links in sorted order, so
finding group memberships can be an O(log n) rather than O(n)
operation. This patch makes use of that.
The backlinks are not sorted, nor are forward links in old databases,
so we have to use a linear search in those cases.
There is a little bit of extra work to handle the few kinds of forward
links (e.g. msDS-RevealedUsers) that have DN+Binary values.
Tim and Garming came up with the basic idea and a prototype.
Pair-programmed-with: Tim Beale <timbeale@catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Until now the linked attrbutes module has allocated its private data
on a per transaction basis, but we prefer to check the sorted links
feature less often than that. So the private data struct is given
module life time and a transaction member to carry out the old role.
In coming patches, the sorted links flag will be used.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
repl_meta_data.c uses the compatible features attribute of the
"@SAMBA_DSDB" special object to record that linked attributes are
being stored in the database in a sorted order. Soon the
linked_attributes module is going to want to know the same thing, and
in time other modules will want to know about other compatible
features, so we introduce a helper function.
Error checking is slightly improved.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
These tests will ensure that linked attributes continue to be handled
correctly under forthcoming changes. The la_move_ou_tree_big() test
will show that the changes make this much faster, after which it can
perhaps be removed.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
In order to detect an value overflow error during
the string to integer conversion with strtoul/strtoull,
the errno variable must be set to zero before the execution and
checked after the conversion is performed. This is achieved by
using the wrapper function strtoul_err and strtoull_err.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Böhme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
provision_get_schema returns a ldb_context object which is stored
in a python object. As a result the parent python object is never
decrefed and probably not released ever.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Initialise pointers to NULL
* replace talloc_free with TALLOC_FREE
* add goto exit to ensure memory deallocated correctly
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Feb 14 06:53:14 CET 2019 on sn-devel-144
All the uses have been replaced with calls to dsdb_domain_count, so it
is no longer needed.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Replace the call to samdb_search_count with dsdb_domain_count. As this
is the only remaining caller of samdb_search_count, replacing it will
allow the removal of samdb_search_count.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This counts the number of objects that are in the domain,
provided a domain SID was supplied (otherwise it just
counts all the objects).
This routine avoids allocating memory for the full
result set by using a callback.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Also we can defer it past a thing that doesn't need or check for it.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We are already using it in two places, and are about to add a third.
The version in repl_meta_data.c did more work in the case that the
parsed_dns can't really be trusted to conform to the expected format;
this is now a wrapper called get_parsed_dns_trusted_fallback().
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Before "OU=vlvtestou2,%s" % (self.base_dn) was left behind after the
test.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This allows the check password script to reject the username and other
things.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
PyDictSetxxx methods don't steal reference so if the items added
to the dictionary were created just for the purpose of inserting
into the dict then we need to decref them.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Thu Feb 7 17:17:46 CET 2019 on sn-devel-144
The former is just an alias for the latter. samba_add_onoff_option()
better describes what the function actually does, so use that and
remove the alias.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
In large domains with many users, '(objectClass=User)' may as well not
be specified because it's iterating over the entire database.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Generate an appropriate log message in the event of an error
log_group_membership_changes. As the changes have not been applied to
the database, there is no easy way to determine the intended changes.
This information is available in the "dsdbChange" audit messages, to
avoid replicating this logic for what should be a very rare occurrence
we simply log it as a "Failure"
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Jan 19 22:32:05 CET 2019 on sn-devel-144
Add tests to exercise the error handling in
log_group_membership_changes.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
audit_log.c:878:7: error: assuming signed overflow does not occur when
simplifying conditional to constant [-Werror=strict-overflow]
Signed-off-by: Andreas Schneider <asn@samba.org>
This avoids confusing messages, samba_dnsupdate returns the number of
failed updates
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This restores the previous behaviour. It causes (only) the event ID
to be omitted if status != LDB_SUCCESS or there was a problem getting
the group type.
Errors at this stage are exceedingly rare, because the values have
already been checked by the repl_meta_data module, but this is
cosistent with the rest of the module again.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Dec 22 01:58:48 CET 2018 on sn-devel-144
Generate a GroupChange event when a user is created with a PrimaryGroup
membership. Log the windows event id in the JSON GroupChange message.
Event Id's supported are:
4728 A member was added to a security enabled global group
4729 A member was removed from a security enabled global
group
4732 A member was added to a security enabled local group
4733 A member was removed from a security enabled local group
4746 A member was added to a security disabled local group
4747 A member was removed from a security disabled local group
4751 A member was added to a security disabled global group
4752 A member was removed from a security disabled global
group
4756 A member was added to a security enabled universal
group
4757 A member was removed from a security enabled universal
group
4761 A member was added to a security disabled universal
group
4762 A member was removed from a security disabled universal
group
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Testing the new GUID list based paged results module
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Fri Dec 21 11:10:30 CET 2018 on sn-devel-144
Replacing paged results module to use GUID list instead of storing
result list in memory, in order to improve memory performance.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
More vlv testing for cases involving modifying, deleting, and adding records
while observing the effect on already initialised views.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This helps avoid duplicate values and clearly indicates what value to select next.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Dec 17 04:30:39 CET 2018 on sn-devel-144
This is a hold-over from the LDAP backend project, which has not yet been revived.
There will be bigger issues than what to do if the schema changes if this ever comes back
and our schema code is way to complex at the moment.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Add a new "eventId" element to the PasswordChange JSON log messages.
This contains a Windows Event Code Id either:
4723 Password changed
4724 Password reset
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Feedback from real-world users is that they really want raw JSON
strings in the log.
We can not easily remove the leading " " but the other strings above
and before the JSON are really annoying to strip back off
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13714
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
commit: 34ca15fb04 changed the
test so embedded NULLs were avoided when python3 was used.
This was due to the fact the string comparison function
'locale.strcoll' cannot handle embedded NULLs. This commit
a) Restores the test data using embedded NULLs which was
not used depending on the python runtime version
b) Removes the problematic calculation of expected sorting order
and instead uses sort order data stored in files.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Currently the test only runs fully under python2. As an interim
measure while we figure out how to get this to work property under
python3 I have converted the test to planoldpythontest so it also
can run under python2
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Adjust test data to remove use of embedded NULLs.
We are getting the following exception when running the test
under python3
Exception: Exception: Traceback (most recent call last):
File "source4/dsdb/tests/python/sort.py", line 181, in setUp
key=cmp_to_key_fn(locale.strcoll))
ValueError: embedded null character
Looking at the source code for locale.strcoll the lhs & rhs
params are processed using 'PyUnicode_AsWideCharString(s, NULL)'
Documentation for PyUnicode_AsWideCharString states:
"Convert the Unicode object to a wide character string.
The output string always ends with a null character. If size is
not NULL, write the number of wide characters (excluding the
trailing null termination character) into *size. Note that the
resulting wchar_t string might contain null characters, which
would cause the string to be truncated when used with most C
functions. If size is NULL and the wchar_t* string contains null
characters a ValueError is raised."
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Test still fails after applying the following fixes
a) only decode byte or (py2) str objects
b) fix sorted function no longer use cmp func, use compat
cmp_to_key_fn instead
c) convert ldb.bytes returned from ldb search results
d) convert sort_functions.keys() to list so array slicing works
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Fix misc ldb.bytes needing to be stringified for tests to succeeed
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
use Exception.args member as exception is no longer supports
indexing.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Make sure samba-tool is called with correct PYTHON version
* Convert ldb.bytes results to string for comparison with expected
results
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Misc changes mostly around use of ldb.bytes, bytes and string to get
the test to work. Additionally results of a range cannot be combined
with a list using the '+' operator (need to enclose the range with
list).
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Convert ldb.bytes object to string for further processing
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Fix dict_items are not combinable with '+' operator error, need
to convert to list first.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
gensec.Security.update takes bytes as param not string with py3
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Fix various assertTrue/assertEquals so ldb.bytes (PY3)
comparaisons work both in PY3 & PY2
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Fix various assertEquals comparing ldb.bytes with string
when running with PY3
* Fix a couple of tuple assignments to exception (not supported
in PY3)
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Misc changes to ensure samba4.sam.python test will run under
python2/python3
* various objectSID values when formatted need to be treated
as strings for tests.
* DOMAIN_RID_USERS, DOMAIN_RID_DOMAIN_MEMBERS, DOMAIN_RID_DCS,
DOMAIN_RID_READONLY_DCS are all integers (but attibutes are
ldb.bytes in PY3, need to adust various assertEquals.
* Make sure password is encoded correctly
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Clearly the lockOutObservationWindow value is important, and using a
default value of zero doesn't work very well.
This patch adds a better default value (the domain default setting of 30
minutes).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13683
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Wed Nov 28 11:31:14 CET 2018 on sn-devel-144
Fix a remaining place where we were trying to read the
msDS-LockoutObservationWindow as an int instead of an int64.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13683
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Commit 442a38c918 refactored some code into a new
get_lockout_observation_window() function. However, in moving the code,
an ldb_msg_find_attr_as_int64() inadvertently got converted to a
ldb_msg_find_attr_as_int().
ldb_msg_find_attr_as_int() will only work for values up to -2147483648
(about 3.5 minutes in MS timestamp form). Unfortunately, the automated
tests used a low enough timeout that they still worked, however,
password lockout would not work with the Samba default settings.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13683
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Sanity-check that when we use the default lockOutObservationWindow that
user lockout actually works.
The easiest way to do this is to reuse the _test_login_lockout()
test-case, but stop at the point where we wait for the lockout duration
to expire (because we don't want the test to wait 30 mins).
This highlights a problem currently where the default values don't work.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13683
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The previous refactor now means we return early if we don't need to
re-apply isDeleted to the object. The 'else' is redundant and we can
remove it to avoid unnecessary indent.
This patch is basically just a whitespace change. It should not alter
functionality.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Nov 23 08:10:41 CET 2018 on sn-devel-144
It's easier to follow the logic involved here when it's split out into a
separate function.
This patch should not alter the existing logic/functionality.
Note the 'else' case is somewhat redundant, but it avoids excessive
whitespace changes to the function. It'll be tidied up in the next
patch.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We can prevent anyone from inadvertently adding/removing msg->elements[]
in replmd_process_linked_attribute() by just not passing msg into the
function. Currently we only actually need the source DN and a memory
context for reallocating old_el->values.
The warning comment has been moved to a more appropriate place.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This patch should not alter functionality - it is just making memory
assumptions used in replmd_process_linked_attribute() clearer.
When adding/removing msg->elements we have to take care, as this will
invalidate things like the parsed-DN array or old ldb_message_element
pointers. This has always been the case (i.e. f6bc4c08b1),
however, now we need to take even more care, as the msg being modified
is re-used and split across 2 different functions.
Add more code comments to highlight this. We can also free
pdn_list/old_el to prevent them being incorrectly used after realloc.
It seems appropriate to also add a sanity-check that the tmp_ctx alloc
succeeds (which all the other memory hangs off).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Tim Beale <timbeale@samba.org>
Autobuild-Date(master): Wed Nov 21 05:31:10 CET 2018 on sn-devel-144
Quite a bit of time was spent in dsdb_get_deleted_objects_dn()
processing during either a join (~9%) or a full-sync (~28%).
The problem is we're *always* doing the dsdb_get_deleted_objects_dn()
call for each object, regardless of whether it's actually deleted or
not. i.e. we were doing an expensive query and a lot of the time just
ignoring the query result.
If it's not a deleted object we're dealing with, we can just return
early and skip the unnecessary processing.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
When a group has 10,000+ links, get_parsed_dns_trusted() can be costly
(simply the talloc calls alone are expensive). Instead of re-generating
the pdn_list for every single link attribute, we can change to only
re-generate it when we really need to.
When we add a new link, it reallocates old_el->values, and so we need to
recreate the pdn_list because all the memory pointers will have changed.
However, in the other cases, where we're simply updating the existing
link value (or ignoring the update, if it's already applied), we can
continue using the same pdn_list (rather than re-parsing it again).
This would generally only save time with a full-sync - it won't really
help with the join case (because every link processed results in a
realloc).
On a DB with 5000 users, this makes a full-sync about ~13% faster.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We should only need to lookup the msg attribute once per source object.
The old_el->values may change due to link-processing, but old_el itself
should not.
This is not aimed at improving performance, but we need to change how
old_el is used before we can change pdn_list (which is more costly
processing-wise).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
At first glance, this code seemed completely unnecessary. However, it
was added (by commit f6bc4c08b1) for a valid reason: adding the
whenChanged/uSNChanged attributes to the message can cause msg->elements
to be reallocated, which means the old_el pointer (which points to
msg->elements memory) can be out of date.
whenChanged/uSNChanged now get added to the msg last, just before the DB
modify operation. So old_el can no longer become out of date within
replmd_process_link_attribute(), so re-fetching it is now redundant.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Move this closer to where the source object actually gets modified.
The main reason to do this is that adding fields can cause the
msg->elements to be reallocated, which will invalidate all the
old_el and pdn_list pointers which are derived from the msg.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Commit 775054afbe reworked replmd_process_link_attribute() so that
we batch together DB operations for the same source object. However, it
was possible that the object had not actually changed at all, e.g.
- link was already processed by critical-objects-only during join, or
- we were doing a full-sync and processing info that was already
up-to-date in our DB.
In these cases we modified the object anyway, even though nothing had
changed. This patch fixes it up, so we check that the object has
actually changed before modifying the DB.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
In order to share work across related link attribute updates, we need
replmd_process_link_attribute() to let the caller know what actually
changed.
This patch adds an extra return type that'll be used in the next patch.
What we're interested in is: the update was ignored (i.e. it's old news),
a new link attribute was added (because this affects the overall
msg/element memory), and an existing link attribute was modified (due to
how links are actually stored, this includes deleting the link, as in
reality it simply involves setting the existing link to 'inactive').
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
By caching the recycle-bin state we can save ~6% of the join time.
Checking whether the recycle-bin is enabled involves an underlying DSDB
search. We do this ~4 times for each link we replicate (twice for the
link source and target). By caching the recycle-bin's state over the
duration of the replication, we can save 1000s of unnecessary DB
searches.
With 5K users this makes the join time ~5 secs faster.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Tim Beale <timbeale@samba.org>
Autobuild-Date(master): Tue Nov 20 08:40:16 CET 2018 on sn-devel-144
replmd_store_linked_attributes() has gotten in szie and complexity. This
refactors some code out into a separate function to make things a bit
more manageable.
This patch should not alter functionality.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We were passing in the entire src_msg, but all we really need is the
source object's DN (and even then, it's only used in error messages).
Change it so we only pass in what the function actually needs. This
makes it a bit easier to see what src_msg is actually used for.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We receive the links grouped together by source object. We can save
ourselves some work by not looking up the source object for every single
link (if it's still the same object we're dealing with).
We've already made this change to replmd_process_linked_attribute().
This patch makes the same change to replmd_store_linked_attributes().
(We verify that we know about each link source/target as we receive each
replication chunk. replmd_process_linked_attribute() kicks in later as
the transaction completes).
Note some care is needed to hold onto the tmp_ctx/src_msg across
multiple passes of the for loop.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Refactor replmd_verify_linked_attribute() so we split out the link
attribute source/target checks. This patch should not alter
functionality.
The source object check has been moved out to where
replmd_verify_linked_attribute() was called.
replmd_verify_linked_attribute() has been renamed, as it's now only
checking the link target.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Avoid an "else" where we have the early return
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Björn Baumbach <bbaumbach@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
convert various attribute results to str so assert function
as expected for tests.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
+ Misc attributes needed to be converted to strings from bytes to
ensure various asserts work as expected.
+ Fix ndr_unpack call which needs bytes not str
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Python 3 does not have .next(), which we rely on, change the
code slightly so it works in py2/py3 (using builtin next function)
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Fri Nov 2 06:56:24 CET 2018 on sn-devel-144
Add a post fork hook to the service API this will be called:
- standard process model
immediately after the task_init.
- single process model
immediately after the task_init
- prefork process model, inhibit_pre_fork = true
immediately after the task_init
- prefork process model, inhibit_pre_fork = false
after each service worker has forked. It is not run on the service
master process.
The post fork hook is not called in the standard model if a new process
is forked on a new connection. It is instead called immediately after
the task_init.
The task_init hook has been changed to return an error code. This ensures
the post_fork code is only run if the task_init code completed successfully.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Move the source object checks and DB modify operation up a level, so we
only do them once per source object rather than once per link.
This allows LMDB joins to succeed with ~15,000 members in a group.
Previously LMDB would fail with the error:
Failed to apply linked attribute change '(-30792) - MDB_MAP_FULL:
Environment mapsize limit reached at ../lib/ldb/ldb_mdb/ldb_mdb.c:203'
Rewriting the same object ~15000 times seemed to completely fill up
the LMDB 8Gb buffer. Presumably this was because LMDB is 'copy on
write', so it was storing ~15,000 copies of the same object. Strangely,
we don't see this problem writing the backlinks (which this patch won't
have helped with at all, because that's modifying the target object).
Note uSNChanged was only being added to the msg once, so the code has
been modified to replace the usnChanged each time (i.e. remove it and
re-add it).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Nov 1 23:48:21 CET 2018 on sn-devel-144
Eventually we want to combine multiple link attributes, that apply to the
same source object, into a single DB 'modify' operation. This will mean
the memory context needs to hang around until we have performed the DB
operation (instead of allocating a temporary context for each link).
This patch moves the talloc context one level up, so a temp context gets
allocated for each link *group*, instead of for each link *attribute*.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We've grouped the linked attributes by source-object. Next, we want to
avoid duplicated processing for the source object, i.e. we only need to
check the source object exists once, not once per link.
Before we can do this, we need to tease apart
replmd_extract_la_entry_details(), which is doing both source and target
object processing. Split out extracting the target DSDB-DN so that it's
done separately.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Instead of processing each link attribute one at a time, we want to
group them together by source object. This will mean we only have to
look-up the source object once, and only perform one DB 'modify'
operation. With groups with 1000s of members, this will help improve
performance.
This patch takes the first step of group together the links by
source-object. A new 'la_group' struct is added to help track what links
belong to the same source object. The la_list essentially becomes a
'list of lists' now.
Note that only related links *in the same chunk* are only grouped together.
While it is trivial to groups together links that span different
replication chunks, this would be a fairly insignificant efficiency gain,
but seems to have a fairly detrimental memory overhead, once you get
into groups with 10,000+ members.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
During a join of a large DB, processing the linked attributes can take a
long time. The join hangs in 'Committing SAM database' for many minutes
with no indication of whether it's making progress or not.
This patch adds some extra debug to show how far through processing the
linked attributes we are, when there are many thousands of links.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Commit 3117279477 removed a duplicate
test but it removed the wrong one, leaving this dreadful typo.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Add extra tests to ensure better test coverage of log_membership_changes
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Oct 30 20:20:26 CET 2018 on sn-devel-144
Change check_version to display the expected, actual along with the
line and name of the failing test, rather than the line in check_version
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Change check_timestamp to display the expected, actual along with the
line and name of the failing test, rather than the line in
check_timestamp.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Rename the parameter names and adjust the return codes from dn_compare
so that:
dn_compare(a, b) =>
LESS_THAN means a is less than b.
GREATER_THAN means a is greater than b.
Thanks to metze for suggesting the correct semantics for dn_compare
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13664
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
The group audit code incorrectly logs member additions and deletions.
Thanks to metze for the debugging that isolated the issue, and for
suggesting the fix to dn_compare.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13664
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Oct 30 10:32:51 CET 2018 on sn-devel-144
This is important, otherwise we'll loose the <SID=> component of the
linked attribute.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This will be used by dbcheck in the next commits.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This will be used to fix missing <SID=> components in future.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is not always going to work, and is not guaranteed to be
consistent even between minor versions.
Here is a simple counterexample:
>>> a = 'hello'
>>> a is 'hello'
True
>>> a is 'hello'.lower()
False
>>> a == a.lower()
True
Possibly it always works for the empty string, but we cannot rely
on that.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Oct 29 23:13:36 CET 2018 on sn-devel-144
The tests differ in two lines, thus:
def test_subClassOf(self):
- """ Testing usage of custom child schamaClass
+ """ Testing usage of custom child classSchema
[...]
-governsId: 1.3.6.1.4.1.7165.4.6.2.6.3.""" + str(random.randint(1, 100000)) + """
+governsId: 1.3.6.1.4.1.7165.4.6.2.6.7.""" + str(random.randint(1, 100000)) + """
with the governsId OID changed because it was noticed they were colliding.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Correctly handle "ldb://" and "mdb://" schemes in the file path when
determining the path for the encrypted secrets key file.
When creating a new user and specifying the local file path of the
sam.ldb DB, it was possible to create an account that you could not
login with. The path for the key file was incorrectly calculated
for the "ldb://" and "mdb://" schemes, the scheme was not stripped from
the path and the subsequent open of the key file failed.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13653
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Oct 19 09:34:46 CEST 2018 on sn-devel-144
When creating a new user and specifying the local file path of the
sam.ldb DB, it's possible to create an account that you can't actually
login with.
This commit contains tests to verify the bug.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13653
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Most of the DSDB modules only want to check the existence of a control,
rather than access the control itself. Adding a helper function allows
the code to ask more natural-sounding yes/no questions, and tidies up
an ugly-looking long-line in extended_dn_out.c.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Fri Oct 12 07:23:26 CEST 2018 on sn-devel-144
Previously, this code used to live inside the loop, so the
checked_reveal_control was needed to save ourselves unnecessary work.
However, now that the code has been moved outside the loop, the
checked_reveal_control variable is just unnecessary complication.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Add the 'reveal_internals' controls when performing objectclass-based
checks of mandatory attributes. This prevents the extended_dn DSDB
module from suppressing attributes that point to deleted (i.e.
non-existent/expunged) objects.
This ensures that, when modifying an object (and often not even
touching the mandatory attribute) that the fact that an attribute is a
DN, and the DN target is deleted, that the schema check will still pass.
Otherwise a fromServer pointing at a dead server can cause failures,
i.e. you can't modify the affected object at all, because the DSDB
thinks a mandatory attribute is missing.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
The fromServer attribute is slightly unique, in that it's a DN (similar
to a one-way link), but it is also a mandatory attribute.
Currently, if fromServer gets a bad value (i.e. a dead server that has
been expunged), the DSDB rejects any attempts to modify the associated
nTDSConnection object (regardless of whether or not you're actually
changing the fromServer attribute).
This patch adds a test-case that demonstrates how the DB can get into
such a state.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Python int is at least a C long; Python long disappears in Py3.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <nopower@suse.com>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Oct 10 09:28:20 CEST 2018 on sn-devel-144
As an after-thought to commit 563e454e8c, we thought it
might be a good idea to add a test case that requests an non-existent
attribute in the attribute-filter as well the search-filter.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This patch should not alter functionality. It is just updating the Samba
code to better match the Windows specification docs.
When fixing Samba BUG #13434, the Microsoft behaviour wasn't clearly
documented, so we made a best guess based on observed behaviour.
The problem was an exception was made to allow "objectClass=*" searches
to return objects, even if you didn't have Read Property rights for the
object's objectClass attribute. However, the logic behind what
attributes were and weren't covered by this exception wasn't clear.
I made a guess that it was attributes belonging to the Public Info
property-set that also have the systemOnly flag set.
Microsoft have confirmed the object visibility behaviour. It turns out
that an optimization is made for the 4 attributes that are always
present for every object (i.e. objectClass, distinguishedName,
name, objectGUID). They're updating their Docs to reflect this.
Now that we know the Windows logic, we can update the Samba code.
This simplifies the code somewhat.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
+ fix a couple of flake8 warnings
+ add some extra code comments (particularly around the cases where the
child class overrides a particular method, to avoid confusion when
browsing the code).
+ assert_not_in_result() was duplicated (it's only needed for the deny
ACL tests)
+ skip redundant if in dirsync's assert_search_result() (it always has
to use the base-DN - we never pass it this as an args).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Python 2.6 wants "{0}".format(x), not "{}".format(x).
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Seems the underlying c code expects binary blob, so.. we should
handle str for py2 and byte for py3
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Sep 6 15:50:17 CEST 2018 on sn-devel-144
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Salt principal for the interdomain trust is krbtgt/DOMAIN@REALM where
DOMAIN is the sAMAccountName without the dollar sign ($)
The salt principal for the BLA$ user object was generated wrong.
dn: CN=bla.base,CN=System,DC=w4edom-l4,DC=base
securityIdentifier: S-1-5-21-4053568372-2049667917-3384589010
trustDirection: 3
trustPartner: bla.base
trustPosixOffset: -2147483648
trustType: 2
trustAttributes: 8
flatName: BLA
dn: CN=BLA$,CN=Users,DC=w4edom-l4,DC=base
userAccountControl: 2080
primaryGroupID: 513
objectSid: S-1-5-21-278041429-3399921908-1452754838-1597
accountExpires: 9223372036854775807
sAMAccountName: BLA$
sAMAccountType: 805306370
pwdLastSet: 131485652467995000
The salt stored by Windows in the package_PrimaryKerberosBlob
(within supplementalCredentials) seems to be
'W4EDOM-L4.BASEkrbtgtBLA' for the above trust
and Samba stores 'W4EDOM-L4.BASEBLA$'.
While the salt used when building the keys from
trustAuthOutgoing/trustAuthIncoming is
'W4EDOM-L4.BASEkrbtgtBLA.BASE', which we handle correct.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13539
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Alexander Bokovoy <ab@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Sep 5 03:57:22 CEST 2018 on sn-devel-144
This is similar to dsdb_trust_xref_tdo_info(), but will also work
if we ever support more than one domain in our forest.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We should not overwrite it within the function.
Currently it doesn't matter as we don't have multiple domains
within our forest, but that will change in future.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This means we can have a long observation window for many of the tests and
so make them much more reliable. Many of these cause frustrating flapping
failures in our CI systems.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Sep 3 06:14:55 CEST 2018 on sn-devel-144
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
