IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
The variable name "ac" typically implies the async context, and the long-life
private context is normally denoted private, not context. This aligns better
with other modules.
talloc_get_type_abort() is now also used.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This is only used for selftest, to send out the log messages for checking.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
We still build some of the ldb_modules even when we are not a DC, so we must
split up the DSDB_MODULE_HELPERS.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This is not a general purpose profiling solution, but these JSON
logs are already being generated and stored, so this is worth adding.
This will allow administrators to identify long running
transactions, and identify potential performance bottlenecks.
This complements a similar patch set to log authentication duration.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Jun 25 11:16:18 CEST 2018 on sn-devel-144
This is not a general purpose profiling solution, but these JSON logs are already being
generated and stored, so this is worth adding.
Some administrators are very keen to know how long authentication
takes, particularly due to long replication transactions in other
processes.
This complements a similar patch set to log the transaction duration.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
../source4/dsdb/samdb/ldb_modules/samldb.c: In function ‘samldb_add’:
../source4/dsdb/samdb/ldb_modules/samldb.c:424:6: error: ‘found’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if (found) {
^
../source4/dsdb/samdb/ldb_modules/samldb.c:348:11: note: ‘found’ was declared here
bool ok, found;
^~~~~
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13437
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Jun 13 13:40:56 CEST 2018 on sn-devel-144
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Jun 9 17:42:38 CEST 2018 on sn-devel-144
Replace uses of the string "sessionInfo" with the constant
DSDB_SESSION_INFO, and "networkSessionInfo" with the constant
DSDB_NETWORK_SESSION_INFO.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Log details of Group membership changes and User Primary Group changes.
Changes are logged in human readable and if samba has been built with
JANSSON support in JSON format.
Replicated updates are not logged.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add audit logging of DSDB operations and password changes, log messages
are logged in human readable format and if samba is commpile with
JANSSON support in JSON format.
Log:
* Details all DSDB add, modify and delete operations. Logs
attributes, values, session details, transaction id.
* Transaction roll backs.
* Prepare commit and commit failures.
* Summary details of replicated updates.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13462
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Jun 4 20:58:01 CEST 2018 on sn-devel-144
Copy the dsdb_control_password_acl_validation into the reply so that it
is available to the audit_logging module. The audit logging module
uses it to differentiate between password change and reset operations.
We include it in the result for failed request to allow the logging of
failed attempts.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
partition_copy_all uses ldb_wait to wait for the update to the primary
partition to complete, when updating a special dn. If a module higher
up the chain inserts a callback, the code blocks in ldb_wait and does
not complete. This change replaces the ldb_wait logic with a callback.
Currently there is no code that triggers this bug, however the up coming
audit logging changes do trigger this bug.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
The sort was written back when the module did not operate recursivly
over the tree. Now it is just confusing, so replace with useful
comments.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Instead, use the actual found attribute (less error prone).
This is an attempt to fix:
./source4/dsdb/repl/replicated_objects.c:945 Failed to prepare commit of transaction:
attribute isDeleted: invalid modify flags on CN=g1_1527558311141,CN=Users,DC=samba,DC=example,DC=com: 0x0
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
In a typical user login query, the code tries to work out the PSO 2-3
times - once for the msDS-ResultantPSO attribute, and then again for the
msDS-User-Account-Control-Computed & msDS-UserPasswordExpiryTimeComputed
constructed attributes.
The PSO calculation is reasonably expensive, mostly due to the nested
groups calculation. If we've already constructed the msDS-ResultantPSO
attribute, then we can save ourselves extra work by just re-fetching the
result directly, rather than expanding the nested groups again from
scratch.
The previous patch improves efficiency when there are no PSOs in the
system. This should improve the case where there are PSOs that apply to
the users. (Unfortunately, it won't help where there are some PSOs in
the system, but no PSO applies to the user being queried).
Also updated sam.c so the msDS-ResultantPSO gets calculated first,
before the other constructed attributes.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Wed May 23 10:09:11 CEST 2018 on sn-devel-144
The new PSO code adds some additional overhead in extra lookups. To
avoid penalizing existing setups, we can short-circuit the PSO
processing and return early if there are no actual PSO objects in the
DB. The one-level search should be very quick, and it avoids the need to
do more complicated PSO processing (i.e. expanding the nested groups).
The longer-term plan is to rework the tokenGroups lookup so that it only
gets done once, and the result can then be reused by the resultant-PSO
code (rather than computing the nested-groups again). However, in the
short-term, a slight decrease in performance is the price for any users
that want to deploy PSOs.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
When calculating the Password-Expiry-Time, we should use the PSO's
max-password-age setting, if one applies to the user.
This is code may be inefficient, as it may repeat the PSO-lookup work
several times (once for each constructed attribute that tries to use
it). For now, I've gone for the simplest code change, and efficiency can
be addressed in a subsequent patch (once we have a good test to measure
it).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Honour the settings in the PSO when changing the password, i.e.
msDS-PasswordComplexityEnabled, msDS-PasswordHistoryLength, etc.
The password_hash code populates dsdb_control_password_change_status's
domain_data with the password settings to use - these are currently
based on the settings for the domain.
Now, if the password_hash code has worked out that a PSO applies to the
user, we override the domain settings with the PSO's values.
This change means the password_settings tests now pass.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Anonymous structs and 80 character line-lengths don't mix well. Allow
the struct to be referenced directly.
With the introduction of PSOs, the password-settings are now calculated
per-user rather than per-domain. I've tried to reflect this in the
struct name.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
When a user's password-hash is modified, we need the PSO settings for
that user, so that any lockout settings get applied correctly.
To do this, we query the msDS-ResultantPSO in the user search. Then, if
a PSO applies to the user, we add in a extra search to retrieve the
PSO's settings. Once the PSO search completes, we continue with the
modify operation.
In the event of error cases, I've tried to fallback to logging the
problem and continuing with the default domain settings. However,
unusual internal errors will still fail the operation.
We can pass the PSO result into dsdb_update_bad_pwd_count(), which means
the PSO's lockout-threshold and observation-window are now used. This is
enough to get the remaining lockout tests passing.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
msDS-User-Account-Control-Computed uses the effective-lockoutDuration to
determine if a user is locked out or not. If a PSO applies to the user,
then the effective-lockoutDuration is the PSO's msDS-LockoutDuration
setting. Otherwise it is the domain default lockoutDuration value.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
If a PSO applies to a user, use its lockOutThreshold/Duration settings
instead of the domain setting. When we lookup a user, we now include the
msDS-ResultantPSO attribute. If the attribute is present for a user,
then we lookup the corresponding PSO object to get the lockOutThreshold/
Duration settings.
Note: This is not quite enough to make the PSO lockout tests pass, as
msDS-User-Account-Control-Computed is still constructed based on the
domain lockoutDuration setting rather than the PSO.
Updating the password_hash.c code properly will be done in a subsequent
commit.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Add support for the msDS-ResultantPSO constructed attribute, which
indicates the PSO (if any) that should apply to a given user. First we
consider any PSOs that apply directly to a user. If none apply directly,
we consider PSOs that apply to any groups the user is a member of. (PSO
lookups are done by finding any 'msDS-PSOAppliesTo' links that apply to
the user or group SIDs we're interested in.
Note: the PSO should be selected based on the RevMembGetAccountGroups
membership, which doesn't include builtin groups. Looking at the spec,
it appears that perhaps our tokenGroups implementation should also
exclude builtin groups. However, in the short-term, I've added a new
ACCOUNT_GROUPS option to the enum, which is only used internally for
PSOs.
The PSO test cases (which are currently only checking the constructed
attribute) now pass, showing that the correct msDS-ResultantPSO value is
being returned, even if the corresponding password-policy settings are
not yet being applied.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Those wishing to build without gpgme support need simply to build --without-gpgme
This In general, we prefer that optional libraries be required by default
so that they are not accidentially missed, particularly in packages.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
During the new samba-tool domain backup restore the NTDS GUID changes
as the server is taken over by the new DC record.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
construct_generic_token_groups() currently works out the entire group
membership for a user, including the primaryGroupID. We want to do the
exact same thing for the msDS-ResultantPSO constructed attribute.
However, construct_generic_token_groups() currently adds the resulting
SIDs to the LDB search result, which we don't want to do for
msDS-ResultantPSO.
This patch splits the bulk of the group SID calculation work out into
a separate function that we can reuse for msDS-ResultantPSO. basically
this is just a straight move of the existing code. The only real change
is the TALLOC_CTX is renamed (tmp_ctx --> mem_ctx) and now passed into
the new function (so freeing it if an error conditions is hit is now
done in the caller).
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
We'll reuse this code for working out the msDS-ResultantPSO, so
references to 'tokenGroups' in error messages would be misleading.
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
The password attributes are defined as literal in two places in the
password_hash code. They will also be needed to support password change
logging. This patch replaces the individual definitions with a shared
constant.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
A usage in GetDCNameEx2 could return the wrong result. This may need to
be fixed in other places.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13365
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We have to also take it out in the partitions code when we load the
partition backends.
This ensures that the init handlers hold a whole-db lock just as the
search code does.
To ensure the locking count in schema_load is balanced, the
private data is now created in the first lock_read() call.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13379
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Rather than refusing the reload based on making cached sequence numbers match
just load it once at the time the DB is globally locked, if required.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13379
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This should trigger slightly less often and is the more correct place, as
we only load it during the first lock when not in a transaction.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13379
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This matches the earlier check of p && p->normalise.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13379
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This call needs to be done at the very first chance, in this case
during the first call to the lock_read() hook, otherwise the
schema_data module can't find the schema.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13379
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This will get the private data on the first call, allowing that not to be
the init() hook.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13379
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This avoids starting a transaction in schema_load_init() and allows it
to operate with a read lock held, which will avoid locking issues
(deadlock detected due to lock odering if we do not have a global
read lock).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13379
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Simply do not decrypt anything until the init call is run.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13379
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This will allow us to lock the databases for read during all of the Samba init
hooks.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13379
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
The net result of this is only that userPassword values (which were
world readable when set) would still be visible after userPassword
started setting the main DB password.
In AD, those values become hidden once the dSHeuristics bit is set,
but Samba lost that when fixing a performance issue with
f26a2845bd
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13378
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
When heimdal encounters a MIT krb5.conf that it does not understand,
it would emit an "ldb operations error". Sadly this does not help
or communicate to the administrator the root cause of the issue.
Improve the error message for when krb init fails during password_hash.c
Signed-off-by: William Brown <william@blackhats.net.au>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
"continue" in a do-while loop jumps to the "while"-check, so "id_exists" needs
to be initialized by that point.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13367
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This whole routine assumes module!=NULL, both in the successful as
well as in error cases. So checking for module!=NULL is confusing both
the reader as well as Coverity.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Which was failing with GNUTLS_E_SHORT_MEMORY_BUFFER - The given memory
buffer is too short to hold parameters.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13352
Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Fri Mar 23 07:25:30 CET 2018 on sn-devel-144
We may have a dn in '<SID=...>' form and ldb_dn_get_linearized()
just gives in empty string.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13300
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
This implements the handling for FPO-enabled attributes, see
[MS-ADTS] 3.1.1.5.2.3 Special Classes and Attributes:
FPO-enabled attributes: member, msDS-MembersForAzRole,
msDS-NeverRevealGroup, msDS-NonMembers, msDS-RevealOnDemandGroup,
msDS-ServiceAccount.
Note there's no msDS-ServiceAccount in any schema (only
msDS-HostServiceAccount and that's not an FPO-enabled attribute
at least not in W2008R2)
msDS-NonMembers always generates NOT_SUPPORTED against W2008R2.
See also [MS-SAMR] 3.1.1.8.9 member.
We now create foreignSeurityPrincipal objects on the fly (as needed).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13300
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
We have several schema related tests, which already prove
that for the defaultObjectCategory attribute.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13307
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
It's quite likely that there're more than one attribute and we may
already altered values.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13307
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
We have several tests which already test that, we can avoid doing
searches at all in that case.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13307
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Mar 13 23:48:28 CET 2018 on sn-devel-144
This is not strictly needed to fig bug 13272, but it makes sense to also
fix this while fixing the overall ACL checking logic.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13272
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This is used to pass information about which password change operation (change
or reset) the acl module validated, down to the password_hash module.
It's very important that both modules treat the request identical.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13272
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Will be used to pass "user password change" vs "password reset" from the
ACL to the password_hash module, ensuring both modules treat the request
identical.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13272
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This is needed, because a later commit will let the acl module add a
control to the change request msg and we must ensure that this is only
done once.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13272
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This change ensures we correctly treat the following LDIF
dn: cn=testuser,cn=users,...
changetype: modify
delete: userPassword
add: userPassword
userPassword: thatsAcomplPASS1
as a password reset. Because delete and add element counts are both
one, the ACL module wrongly treated this as a password change
request.
For a password change we need at least one value to delete and one value
to add. This patch ensures we correctly check attributes and their
values.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13272
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
By default, use tdb, but otherwise read the value from backendStore.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reload the schema just after getting the tranaction lock
but before the transaction counter is bumped.
This ensures we reload the schema exactly once but with
the DB locked.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
It appears that there was a race condition between searching for the
attribute & class definitions, and searching for the schema object, if
the schema was changed in-between the two searches.
This is likely the cause of ldap_schema being flapping.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12889
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
With the lmdb patches, I have cleanly observed the database being read
in between the commit of the metadata.tdb and the eventual commits of
the individual partitions.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
During a read lock, we find ourselves seeing an unchanged schema, but
reading any updates to the metadata.tdb (in the case of lmdb, where
reads do not block writes).
The alternative is to read-lock the entire metadata.tdb, however, this
allows more concurrency by allowing reads not to block writes.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is critical as otherwise we can read a sequence number in advance
of the data that it represents and so have a false cache.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We already rely on gnutls in order to implement SSL/TLS, so using that
to speed up crypto like aes gcm 128 is fine, but as we already have
code for that algorithm, we should use that instead of adding a new
dependency to libnettle.
Some (I guess newer versions) of gnutls use nettle internally, so
we may end up using that code, but we should not have a direct dependency.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13276
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13031
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
No error code was being set in this case, and so, we would commit the
HWM and UDV without actually having all the updates.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13269
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Thu Feb 15 10:18:42 CET 2018 on sn-devel-144
No error code was being set in this case, and so, we would commit the
HWM and UDV without actually having all the updates.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13269
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is really critical bug, it removes valid linked attributes.
When a DC was provisioned/joined with a Samba version older than 4.7
is upgraded to 4.7 (or later), it can happen that the garbage collection
(dsdb_garbage_collect_tombstones()), triggered periodically by the 'kcc' task
of 'samba' or my 'samba-tool domain tombstones expunge' corrupt the linked attributes.
This is similar to Bug #13095 - Broken linked attribute handling,
but it's not triggered by an originating change.
The bug happens in replmd_modify_la_delete()
were get_parsed_dns_trusted() generates a sorted array of
struct parsed_dn based on the values in old_el->values.
If the database doesn't support the sortedLinks compatibleFeatures
in the @SAMBA_DSDB record, it's very likely that
the array of old_dns is sorted differently than the values
in old_el->values.
The problem is that struct parsed_dn has just a pointer
'struct ldb_val *v' that points to the corresponding
value in old_el->values.
Now if vanish_links is true the damage happens here:
if (vanish_links) {
unsigned j = 0;
for (i = 0; i < old_el->num_values; i++) {
if (old_dns[i].v != NULL) {
old_el->values[j] = *old_dns[i].v;
j++;
}
}
old_el->num_values = j;
}
old_el->values[0] = *old_dns[0].v;
can change the value old_dns[1].v is pointing at!
That means that some values can get lost while others
are stored twice, because the LDB_FLAG_INTERNAL_DISABLE_SINGLE_VALUE_CHECK
allows it to be stored.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Changes to provision and join to create a database with
encrypted_secrets enabled and a key file generated.
Also adds the --plaintext-secrets option to join and provision commands
to allow the creation of unencrypted databases.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Encrypt the samba secret attributes on disk. This is intended to
mitigate the inadvertent disclosure of the sam.ldb file, and to mitigate
memory read attacks.
Currently the key file is stored in the same directory as sam.ldb but
this could be changed at a later date to use an HSM or similar mechanism
to protect the key.
Data is encrypted with AES 128 GCM. The encryption uses gnutls where
available and if it supports AES 128 GCM AEAD modes, otherwise nettle is
used.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
When we upgrade a schema from 2008R2 to 2012R2, we want to apply all the
changes in a single transaction - if we can't apply all the updates then
we don't want to be left with a schema halfway in between the two.
However, as we apply each LDIF update, we also want to refresh the
schema. There are 2 reasons for this:
1. The adprep .LDIF files provided by Microsoft have some writes to
schemaUpdateNow in them.
2. Microsoft uses attribute OIDs in their adprep .LDIF files, which
Samba doesn't handle so well. However, we can replace the OIDs with the
attribute's ldapDisplayName and they work fine. But to do this, we need
to query the schema to map the OID to attribute name. And to query the
schema successfully, the schema needs to be refreshed after the new
attribute object has been added.
Basically this patch avoids bailing out during the dsdb_schema_refresh()
if we are writing schemaUpdateNow as part of a larger transaction.
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Adprep schema adds backlinks, but they do not have the NOT_REPLICATED
bit. We need to force this in locally to ensure we have it.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Remove the unique constraint on the objectSID index, and enable the
unique_object_sids module.
This allows duplicate objectSIDs on foreign security principals, and
disallows duplicates for local objectSIDs
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13004
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
New module that sets the LDB_FLAG_INTERNAL_UNIQUE_VALUE on all local
objectSIDS and ensure it is cleared for any foreign security principals.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13004
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
This should not happen, but stopping all replication because of it is a pain.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri Nov 24 19:53:50 CET 2017 on sn-devel-144
USER is memberOf GROUP and they both were deleted on W2K8R2 AD. Domain join ends
with error below.
Failed to apply records: ../source4/dsdb/samdb/ldb_modules/repl_meta_data.c:421
8: Failed to remove backlink of memberOf when deleting CN=USER\0ADEL:a1f2a2cc-1
179-4734-b753-c121ed02a34c,CN=Deleted Objects,DC=samdom,DC=intern: dsdb_module_
search_dn: did not find base dn CN=GROUP\0ADEL:030d0be1-3ada-4b93-8371-927f2092
3116,CN=Deleted Objects,DC=samdom,DC=intern (0 results): Operations error
Failed to commit objects: WERR_GEN_FAILURE/NT_STATUS_INVALID_NETWORK_RESPONSE
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13120
Signed-off-by: Andrej Gessel <Andrej.Gessel@janztec.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This will be used by dbcheck to fix duplicate link values.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This will be used by dbcheck to fix duplicate link values.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
In schema_load_init, we find that the writing of indices is not locked
in any way. This leads to race conditions. To resolve this, we need to
have a new state (SCHEMA_COMPARE) which can report to the caller that we
need to open a transaction to write the indices.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Mon Oct 30 04:16:42 CET 2017 on sn-devel-144
Now both routines avoid the escape/unescape implicit in ldb_dn_add_child_fmt()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
This will allow it to be used in common with replmd_conflict_dn()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
This makes it clearer that we are just replacing the RDN and ensures we do not
somehow create multiple components inside ldb_dn_add_child_fmt().
We also avoid an escape/un-escape round-trip.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
replmd_add_fix_la() was already making the same check; here we move it
a bit earlier.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Because we already have a sorted parsed_dn list, this is a simple
linear scan.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
replmd_build_la_val() is creating a new link attribute. In this case,
the RMD_ORIGINATING_USN and RMD_LOCAL_USN are always going to be the
same thing, so we don't need to pass them in as 2 separate parameters.
This isn't required for any bug fix, but is just a general code
tidy-up.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
replmd_build_la_val() and replmd_set_la_val() are pretty much identical.
Keep the replmd_build_la_val() API (as it makes it clearer we're
creating a new linked attribute), but replace the code with a call to
replmd_set_la_val().
This isn't required for any bug fix, but is just a general tidy-up to
avoid code duplication.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The initial value for RMD_VERSION is one on Windows. The MS-DRSR spec
states the following in section 5.11 AttributeStamp:
dwVersion: A 32-bit integer. Set to 1 when a value for the attribute is
set for the first time. On each subsequent originating update, if the
current value of dwVersion is less than 0xFFFFFFFF, then increment it
by 1; otherwise set it to 0
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13059
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
replmd_build_la_val() is used to populate a new link attribute value
from scratch. The version parameter is always passed in as the initial
value (zero), and deleted is always passed in as false.
For cases (like replication) where we want to set version/deleted to
something other than the defaults, we can use replmd_set_la_val()
instead.
This patch changes these 2 parameters to variables instead.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13059
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Now that the code is all in one place we can refactor it to make it
slightly more readable.
- added more code comments
- tweaked the 'no conflict' return logic to try to make what it's checking
for more obvious
- removed conflict_pdn (we can just use active_pdn instead)
- added a placeholder variable and tweaked a parameter name
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Return immediately if there's no conflict, which reduces nesting.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Link conflict handling is a corner-case. The logic in
replmd_process_linked_attribute() is already reasonably busy/complex.
Split out the handling of link conflicts into a separate function so
that it doesn't detract from the core replmd_process_linked_attribute()
logic too much.
This refactor should not alter functionality.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Currently the code only handles the case where the received link
attribute is a new link (i.e. pdn == NULL). As well as this, we need to
handle the case where the conflicting link already exists, i.e. it's a
deleted link that has been re-added on another DC.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The previous patch to handle link conflicts was simply overriding the
received information and marking the link as deleted. We should be doing
this as a separate operation to make it clear what has happened, and so
that the new (i.e. inactive) link details get replicated out.
This patch changes it so that when a conflict occurs, we immediately
overwrite the received information to mark it as deleted, and to update
the version/USN/timestamp/originating_invocation_id to make it clear
that this is a new change.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
replmd_set_la_val() and replmd_build_la_val() are almost identical. When
we were processing the replicated link attributes we were calling one
function if the link was new, and a different one if the link existed.
I think we should be able to get away with using replmd_set_la_val() in
both cases.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
All the other talloc_asprintf()s in this function use the mem_ctx, but
for some reason the vstring was using the dsdb_dn->dn. This probably
isn't a big deal, but might have unintentional side-effects.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
These two functions are almost identical. The main difference between
them is the RMD_ADDTIME. replmd_set_la_val() tries to use the
RMD_ADDTIME of the old_dsdb_dn. Whereas replmd_build_la_val() always
uses the time passed in.
Change replmd_set_la_val() so it can accept a NULL old_dsdb_dn (i.e. if
it's a new linked attribute that's being set). If so, it'll end up using
the nttime parameter passed in, same as replmd_build_la_val() does.
Also update replmd_process_linked_attribute (which used to use
replmd_build_la_val()) to now pass in a NULL old_dsdb_dn. There
shouldn't be a difference in behaviour either way, but this exercises
the code change.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
If 2 DCs independently set a single-valued linked attribute to differing
values, Samba should be able to resolve this problem when replication
occurs.
If the received information is better, then we want to set the existing
link attribute in our DB as inactive.
If our own information is better, then we still want to add the received
link attribute, but mark it as inactive so that it doesn't clobber our
own link.
This still isn't a complete solution. When we add the received attribute
as inactive, we really should be incrementing the version, updating the
USN, etc. Also this only deals with the case where the received link is
completely new (i.e. a received link conflicting with an existing
inactive link isn't handled).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is the first part of the fix for resolving a single-valued link
conflict.
When processing the replication data for a linked attribute, if we don't
find a match for the link target value, check if the link is a
single-valued attribute and it currently has an active link. If so, then
use the active link instead.
This change means we delete the existing active link (and backlink)
before adding the new link. This prevents the failure in the subsequent
dsdb_check_single_valued_link() check that was happening previously
(because the link would end up with 2 active values).
This is only a partial fix. It stops replication from failing completely
if we ever hit this situation (which means the test is no longer
hitting an assertion when replicating). However, ideally the existing
active link should be retained and just marked as deleted (with this
change, the existing link is overwritten completely).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The previous refactor makes it obvious that we aren't actually using
this variable for anything.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is precursor work for supporting single-link conflicts.
Split out the code to check if the link update is newer. It's now safe
to call this from the main codepath. This also means we can combine the 2
calls to get the seqnum into a single common call.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The code to add the backlink is the same in both the 'if' and the 'else'
case, so move it outside the if-else block.
(We're going to rework this block of code quite a bit in order to
support single-value linked attribute conflicts, aka bug #13055).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13055
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This problem was noticed when 2 DCs added the same linked attribute at
roughly the same time. One DC would have a later timestamp than the
other, so it would re-apply the same link information. However, when it
did this, replmd_update_la_val() would incorrectly increment the
RMD_VERSION for the attribute. We then end up with one DC having a
higher RMD_VERSION than the others (and it doesn't replicate the new
RMD_VERSION out).
During replication RMD_VERSION is used to determine whether a linked
attribute is old (and should be ignored), or whether the information is
new and should be applied to the DB. This RMD_VERSION discrepancy could
potentially cause a subsequent linked attribute update to be ignored.
Normally when a local DB operation is performed, we just pass in a
version of zero and get replmd_update_la_val() to increment what's
already in the DB. However, we *never* want this to happen during
replication - we should always use the version we receive from the peer
DC.
This patch fixes the problem by separating the API into two:
- replmd_update_la_val(): we're updating a linked attribute in the DB,
and so as part of this operation we always want to increment the
version number (the version no longer need to be passed in because
we can work it out from the existing DB entry).
- replmd_set_la_val(): we want to set a linked attribute to use the
exact values we're telling it, including the version. This is what
replication needs to use.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13038
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Sep 26 09:36:48 CEST 2017 on sn-devel-144
A modify of both @INDEXLIST and @ATTRIBUTES will still trigger two re-index passes
but that is a task for later.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=9527
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Sep 20 12:29:49 CEST 2017 on sn-devel-144
This allows debugging of why the LDB failed to start up.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
We display warnings if a target object is missing but it's still OK to
continue the replication. Currently we need to check the target twice -
once to verify it when we first receive it, and once when we actually
commit it (we can't skip the 2nd check altogether because in the join
case, they could occur quite far apart).
One annoying side-effect is we get the same warning message coming out
twice in these special cases.
In the cases where we're checking the dsdb_repl_flags, we can actually
just bypass the verification checks for the target object (if it doesn't
exist we still continue anyway). This may save us a tiny bit of
unnecessary work.
For cross-partition links, we can limit logging these warnings to when
the objects are actually being committed. This avoids spurious warnings
in the join case (i.e. we receive the link before we receive the target
object's partition, but we have received all partitions by the time we
actually commit the objects).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
While running the selftests, I noticed a case where DC replication
unexpectedly sends a linked attribute for a deleted object (created in
the drs.ridalloc_exop tests). The problem is due to the
msDS-NC-Replica-Locations attribute, which is a (known) one-way link.
Because it is a one-way link, when the test demotes the DC and deletes
the link target, there is no backlink to delete the link from the source
object.
After much debate and head-scratching, we decided that there wasn't an
ideal way to resolve this problem. Any automated intervention could
potentially do the wrong thing, especially if the link spans partitions.
Running dbcheck will find this problem and is able to fix it (providing
the deleted object is still a tombstone). So the recommendation is to
run dbcheck on your DCs every 6 months (or more frequently if using a
lower tombstone lifetime setting).
However, it does highlight a problem with the current GET_TGT
implementation. If the tombstone object had been expunged and you
upgraded to 4.8, then you would be stuck - replication would fail
because the target object can't be resolved, even with GET_TGT, and
dbcheck would not be able to fix the hanging link. The solution is to
not fail the replication for an unknown target if GET_TGT has already
been set (i.e. the dsdb_repl_flags contains
DSDB_REPL_FLAG_TARGETS_UPTODATE).
It's debatable whether we should add a hanging link in this case or
ignore/drop the link. Some cases to consider:
- If you're talking to a DC that still sends all the links last, you
could still get object deletion between processing the source object's
links and sending the target (GET_TGT just restarts the replication
cycle from scratch). Adding a hanging link in this case would be
incorrect and would add spurious information to the DB.
- Suppose there's a bug in Samba that incorrectly results in an object
disappearing. If other DCs then remove any links that pointed to that
object, it makes recovering from the problem harder. However, simply
ignoring the link shouldn't result in data loss, i.e. replication won't
remove the existing link information from other DCs. Data loss in this
case would only occur if a new DC were brought online, or if it were a
new link that was affected.
Based on this, I think ignoring the link does the least harm.
This problem also highlights that we should really be using the same
logic in both the unknown target and the deleted target cases.
Combining the logic and moving it into a common
replmd_allow_missing_target() function fixes the problem. (This also has
the side-effect of fixing another logic flaw - in the deleted object
case we would unnecessarily retry with GET_TGT if the target object was
in another partition. This is pointless work, because GET_TGT won't
resolve the target).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
This adds basic DRS_GET_TGT support. If the GET_TGT flag is specified
then the server will use the object cache to store the objects it sends
back. If the target object for a linked attribute is not in the cache
(i.e. it has not been sent already), then it is added to the response
message.
Note that large numbers of linked attributes will not be handled well
yet - the server could potentially try to send more than will fit in a
single repsonse message.
Also note that the client can sometimes set the GET_TGT flag even if the
server is still sending the links last. In this case, we know the client
supports GET_TGT so it's safe to send the links interleaved with the
source objects (the alternative of fetching the target objects but not
sending the links until last doesn't really make any sense).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
This re-work of our LDIF printing avoids some of the privacy issue from
printing the full LDIF at level 4, while showing the entry that actually fails.
Instead, with e3988f8f74 we now print the DN
only at level 4, then the full message at 8.
With this patch on failure, we print the redacted failing message at 5.
While all of the DRS replication data is potentially sensitive
the passwords are most sensitive, and are now not printed unencrypted.
This discourages users from sending the full failing trace, as the
last entry is much more likely the issue.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
This is used in the client and in the server
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This re-work of our LDIF printing avoids some of the privacy issue from
printing the full LDIF at level 4, while showing the entry that actually fails.
Instead, we print the DN only at level 4, then the full message at 8.
While all of the DRS replication data is potentially sensitive
the passwords are most sensitive, and are now not printed unencrypted.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This avoids printing un-encrypted secret values in logs, and while links are not likely
secret, this avoids a future copy and paste using ldb_ldif_message_string() again.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This avoids printing un-encrypted secret values in logs
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This drove me to strace before I understood what it really meant.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Previously Samba would just drop cross-partition links where the link
target object is unknown. Instead, what we want to do is try to add the
forward link for the GUID specified. We can't add the backlink because
we don't know the target, however, dbcheck should be able to fix any
missing backlinks.
The new behaviour should now mean dbcheck will detect the problem and be
able to fix it. It's still not ideal, but it's better than dropping the
link completely.
I've updated the log so that it has higher severity and tells the user
what they need to do to fix it.
These changes now mean that the selftests now detect an error - instead
of completely dropping the serverReference, we now have a missing
backlink. I've updated the selftests to fix up any missing
serverReference backlinks before running dbcheck.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
We are going to end up supporting 2 different server schemes:
A. the old/default behaviour of sending all the linked attributes last,
at the end of the replication cycle.
B. the new/Microsoft way of sending the linked attributes interleaved
with the source/target objects.
Normally if we're talking to a server using the old scheme-A, we won't
ever use the GET_TGT flag. However, there are a couple of cases where
it can happen:
- A link to a new object was added during the replication cycle.
- An object was deleted while the replication was in progress (and
the linked attribute got queued before the object was deleted).
Talking to an Samba DC running the old scheme will just cause it to
start the replication cycle from scratch again, which is fairly
harmless. However, there is a chance that the same thing can happen
again, in which case the replication cycle will fail (because GET_TGT
was already set).
Even if we're using the new scheme (B), we could still potentially hit
this case, as we can still queue up linked attributes between requests
(group memberships can be larger than what can fit into a single
replication chunk).
If GET_TGT is set in the GetNcChanges request, then the local copy of
the target object should always be up-to-date when we process the linked
attribute. So if we still think the target object is deleted/recycled at
this point, then it's safe to ignore the linked attribute (because we
know our local copy is up-to-date). This logic matches the MS spec logic
in ProcessLinkValue().
Not failing the replication cycle may be beneficial if we're trying to
do a full-sync of a large database. Otherwise it might be time-consuming
and frustrating to repeat the sync unnecessarily.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
The server-side can potentially send the linked attribute before the
target-object. This happens on Microsoft, and will happen on Samba once
server-side GET_TGT support is added. In these cases there is a hole
where the Samba client can silently drop the linked attribute.
If the old copy of the target object was deleted/recycled, then the
client can receive the new linked attribute before it realizes the target
has now been reincarnated. It silently ignores the linked attribute,
thinking its receiving out of date information, when really it's the
client's copy of the target object that's out of date.
In this case we want to retry with the GET_TGT flag set, which will
force the updated version of the target object to be sent along with the
linked attribute. This deleted/recycled target case is the main reason
that Windows added the GET_TGT flag.
If the server sends all the links at the end, instead of along with the
source object, then this case can still be hit. If so, it will cause the
server to restart the replication from the beginning again. This is
probably preferential to silently dropping links.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
There was a bug in my previous patch where the code would verify
*all* links in the list, rather than just the ones that are new. And it
would do this for every replication chunk it received, regardless of
whether there were actually any links in that chunk.
The problem is by the time we want to verify the attributes, we don't
actually know which attributes are new. We can fix this by moving where
we store the linked attributes from the start of processing the
replication chunk to the end of processing the chunk. We can then verify
the new linked attributes at the same time we store them.
Longer-term we may want to try to apply the linked attribute at this
point. This would save looking up the source/target objects twice, but
it makes things a bit more complicated (attributes will usually apply at
this point *most* of the time, but not *all* the time).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Windows replication can send the linked attribute before it sends the
source object. The MS-DRSR spec says that in this case the client should
resend the GetNCChanges request with the GET_ANC flag set. In my testing
this resolves the problem - Windows will include the source object for the
linked attribute in the same replication chunk.
This problem doesn't happen with Samba-to-Samba replication, because the
source object for the linked attribute is guaranteed to have already been
sent.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Currently we only check that the target object is known at the end of
the transaction (i.e. the .prepare_commit hook). It's too late at this
point to resend the request with GET_TGT. Move this processing earlier
on, after we've applied all the objects (i.e. off the .extended hook).
In reality, we need to perform the checks at both points. I've
split the common code that gets the source/target details out of the
la_entry into a helper function. It's not the greatest function ever,
but seemed to make more sense than duplicating the code.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
If the DRS client received a linked attribute that it couldn't resolve
the target for, then it would just ignore that link and keep going. That
link would then be lost forever (although a full-sync would resolve
this). Instead of silently ignoring the link, fail the transaction.
This *can* happen on Samba, but it is unusual. The target object and
linked-attribute would need to be added while a replication is still in
progress. It can also happen fairly easily when talking to a Windows DC.
There are two import exceptions to this:
1). Linked attributes that span partitions. We can never guarantee that
we will have received the target object, because it may be in a partition
we haven't replicated yet. Samba doesn't have a great way of handling
this currently, but we shouldn't fail the replication (because that breaks
basic join tests). Just skip that linked attribute and hope that a
subsequent full-sync will fix it.
(I queried Microsoft and they said resolving cross-partition linked
attributes is a implementation-specific problem to solve. GET_TGT won't
resolve it)
2). When the replication involves a subset of objects, e.g.
critical-only. In these cases, we don't increase the highwater-mark, so
it is probably not such a dire problem if we don't add the link. In the
case of critical-only, we will do a subsequent full sync which will then
add the links.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
We want to re-use this code to check that the linked attribute's target
object exists *before* we try to commit the transaction. This will allow
us to re-request the block with the GET_TGT flag set.
This splits checking the target object exists into a separate function.
Minor changes of note:
- the 'parent' argument was passed to replmd_process_linked_attribute()
as NULL, so I've just replaced where it was used in the refactored code
with NULL.
- I've tweaked the "Failed to find GUID" error message slightly to display
the attribute ID rather than the attribute name (saves repeating
lookups and/or passing extra arguments).
- Tweaked the replmd_deletion_state() logic - it only made sense to call
it in the code block where we actually found the target
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
gcc error: ‘result’ may be used uninitialized
This wont happen, because ldb will return and error, but the compiler
doesn't understand this.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12930
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
If we do not call ldb_module_done() then we do not know that up_req->callback()
has been called, and ldb_next_request() will call the callback again.
If called twice, the new ldb_lock_backend_callback() in ldb 1.2.0 will segfault.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12904
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Aug 1 07:52:38 CEST 2017 on sn-devel-144
The previous patch set this incorrectly to NETLOGON_NT_VERSION_1
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This adds support for DRSUAPI_DS_NAME_FORMAT_USER_PRINCIPAL and
DRSUAPI_DS_NAME_FORMAT_SERVICE_PRINCIPAL as desired formats.
This also causes the test in cracknames.py to no longer fail.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12842
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Jul 24 11:10:26 CEST 2017 on sn-devel-144
Previously, when a GUID was desired to
cracknames, it would include recycled objects as well. This would
sometimes result in two objects being returned from a query which is
supposed to return a unique GUID. For example, if a deleted user had
the same sAMAccountName as a non-deleted user and cracknames was used to
find the GUID of this account, it would return two GUIDs, and so would
fail with DRSUAPI_DS_NAME_STATUS_NOT_UNIQUE.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12842
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
The metadata partition (sam.ldb) lock is not
enough to block another process in prepare_commit(),
because prepare_commit() is a no-op, if nothing
was changed in the specific backend.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jun 30 06:23:39 CEST 2017 on sn-devel-144
This is only required if you supply SHOW_RECYCLED or SHOW_DELETED. Note
that any add does trigger this (through callbacks in the modules in acl,
objectclass etc.).
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This happens during provision, however due to the fact that the first
search in the rootDSE init does not check return codes, this was done
implicitly (and coincidentally).
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This helps when we improve show_deleted in a way that the fake database in samba3sam can not cover
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
There's no point in creating a temporary ldb_context as
the only callers already have a valid struct ldb_context for
the local sam.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
There's no point in creating a temporary ldb_context as
all direct callers already have a valid struct ldb_context for
the local sam.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The schema can go away unless the second argument (the memory context) is supplied
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri Jun 16 23:43:46 CEST 2017 on sn-devel-144
It may well be the same as the next one we need to check, so we can
avoid parsing it again.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jun 16 07:39:24 CEST 2017 on sn-devel-144
This should help a lot for large one-level searches and for subtree searches that are of
flat tree structures
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
A replace leads to CONSTRAINT_VIOLATION while an add causes
ATTRIBUTE_OR_VALUE_EXISTS. For this we need to check the mod type
before the replmd_modify_la_* calls because they change everything
into a replace.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is simple enough because we already have the sorted list.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
repl_meta_data knows whether linked attributes are appropriately
[un-]duplicated, and this is how it tells ldb_tdb that.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This caused failures against vampire_dc (on large-dc), likely due to
more frequent replication propagating the record before it was renamed.
The DC ran out of RIDs and RID allocation causes schema replication,
which failed.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12841
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
If we just call ldb_request_done() then we never call the callback.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Provide a XSI-compliant strerror_r on GNU based systems.
The default GNU strerror_r is not XSI-compliant, this patch wraps the
GNU-specific call in an XSI-compliant wrapper.
This reverts 18ed32ce0821d11c0c06d82c07ba1c27b0c2b886 which tried to
make Heimdal use roken, rather than libreplace for strerror_r.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This ditches a particular aspect of thread safety, but I doubt that
ldb is really thread safe. So in practice, I think we should not
see harm from this.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Jun 13 05:06:49 CEST 2017 on sn-devel-144
The modify operations on the rootDSE turn into IRPC messages, and these need
to be handled on the global event context, not the per-operation context
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 31 10:47:46 CEST 2017 on sn-devel-144
Add tests of the WDigest values using ldap. This allows the tests to be
run against Windows, to validate the calculated values.
Tests validated against Windows Server 2012 R2
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Generate sha256 and sha512 password hashes and store them in
supplementalCredentials
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The sam.ldb file does not contain the same kind of data as the partitions, we do not wish to
mix these results. This also avoids taking out locks on the sam.ldb file.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue May 23 05:18:04 CEST 2017 on sn-devel-144
Instead, write it once in the module init, if required, and after a
modify to the schema partition is detected
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
We must, when starting the transaction and preparing to commit the transaction, take
out the locks in the same order across all the databases, otherwise we may deadlock.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
These changes will get clobbered by RWDCs through replication. This
behaviour is required for lockoutTime to enforce the password lockout
locally on the RODC (and is consistent with Windows).
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Previously, you could add or delete and cause replication conflicts on
an RODC. Modifies are already partly restricted in repl_meta_data and
have more specific requirements, so they cannot be handled here.
We still differ against Windows for modifies of non-replicated
attributes over LDAP.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12008
The Windows protocol test suites check that a particular DC is used when
sending referrals.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12008
refactored to make it easier to add extra password hashes.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This allows us to search against binary DN using only the attributeID in
the case of msDS-RevealedUsers (as it appears right at the beginning).
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This allows restriction of auditing attributes from being wiped.
Modifications of the RID Set must be done as SYSTEM.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is important for multi-valued DN+Binary (or DN+String) attributes,
as otherwise they will be considered duplicates.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Bob Campbell <bobcampbell@catalyst.net.nz>
This is required if we are to search them with a binsearch.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This appears quite expensive (particularly in provision), and also
unnecessary.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Mar 10 15:34:39 CET 2017 on sn-devel-144
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Feb 28 13:55:42 CET 2017 on sn-devel-144
This deferred handling was already removed, for performance, from
everything but the add case.
We now remove the normal local add case (an originating update), eg
LDAP add from the transaction commit and insted do it on the ADD
operation callback (replmd_op_callback()).
To keep things simple, we make up the extended DN with the GUID and
SID as the object does not actually exist in the DB at the time we
prepare backlink. This also allows us to avoid another search in the
(much more common) modify case.
We rely on transactions to clean up the add of the object if the
backlink fails, thankfully unlike in replication replmd_add() is
normally the only operation in a transaction, and we have alredy
confirmed the link target exists during get_parsed_dns().
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Mon Feb 27 07:12:02 CET 2017 on sn-devel-144
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Feb 23 15:30:35 CET 2017 on sn-devel-144
Signed-off-by: Chris Lamb <chris@chris-lamb.co.uk>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Chris Lamb <chris@chris-lamb.co.uk>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
The previous code assumed that only plain DNs could be linked attributes.
We need to look over the list of attribute values and find the value
that causes this particular backlink to exist, so we can remove it.
We do not know (until we search) of the binary portion, so we must
search over all the attribute values at this layer, using the
parsed_dn_find() routine used elsewhere in this code.
Found attempting to demote an RODC in a clone of a Windows 2012R2
domain, due to the msDS-RevealedUsers attribute.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11139
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Feb 14 06:14:35 CET 2017 on sn-devel-144
This allows us to know that the output of get_parsed_dns_trusted() is sorted, as an
upgraded attribute of FL2000 links would not otherwise be sorted in the DB
This allows us to delete linked objects that have a forward link from a
FL2000 style linked attribute once the DN+Binary patches land.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
This eliminates a lot of duplicate code and allows us to know that we will
have a set of FL2003 style links in the parsed DNs to operate on
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
This allows us to conform to MS-ADTS 3.1.1.2.3.2, where the OID
1.2.840.113556.1.2.49 can be specified as the mAPIID of a new attribute
in the schema in order to automatically assign it an unused mAPIID.
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
As per MS-ADTS 3.1.1.2.3.1, this allows specifying the OID
1.2.840.113556.1.2.50 as the linkID of a new linked attribute in the
schema in order to automatically assign it an unused even linkID.
Specifying the attributeID or ldapDisplayName of an existing forward
link will now also add the new linked attribute as the backlink of that
existing link.
This also prevents adding duplicate linkIDs. Previously, we could run
into issues when trying to delete backlinks with duplicate linkIDs.
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11139
If it is there, we assume linked attributes are stored in a sorted
order.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This allows us to detect modification by a Samba version prior to
the introduction of the compatibleFeatures logic as this flag will
be stripped by the schema load code of older Samba versions.
Therefore if it is not present, then remove all
compatibleFeatures.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Because @INDEXLIST is rewritten by all Samba versions, we can detect
that we have opened the database with an older version that does not
support the feature flags by the absense of this in @INDEXLIST
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This will allow us to introduce new database features that are
backward compatible from the point of view of older versions of Samba,
but which will be damaged by modifying the database with such a
version.
For example, if linked attributes are stored in sorted order in 4.7,
and this change, without any values in current_supportedFeatures is
itself included in 4.6, then our sortedLinks are backward compatible
to that release.
That is with 4.6 (including this patch) which doesn't care about
ordering -- but a downgraded 4.7 database used by 4.6 will be broken
when later used with 4.7. If we add a 'sortedLinks' feature flag in
compatibleFeatures, we can detect that.
This will allow us to determine if the database still contains
unsorted links, as that information allows us to make the code
handling links much more efficient.
We won't add the actual flag until all the code is in place.
Andrew wrote the actual code and Douglas wrote the tests, and they
cross-reviewed.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Piar-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
selftest: check for database features flags
This is where forward links get added when they get added with an
object.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This roughly follows the pattern in the 2009 commit
0d5d7f5847 by the Andrews Tridgell and Bartlett, which dealt
with zero GUIDs in replmd_add_fix_la(). That function is about to use
get_parsed_dns() [see next commit], and the other users of
get_parsed_dns don't really want to see zero guids, so it is simpler
to test here.
This makes hitting the GUID_all_zero branch of parsed_dn_find() even
more unlikely.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is where linked attributes get added during a replication.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We use a merge-like algorithm, which gives us a slight algorithmic
improvement (O(m + n) vs O(m log(n) + n log(m))) and keeps the results
sorted.
Here's an example. There are existing links to {A C D* F*} where D*
and F* represent deleted links, and we want to replace them with {B C
E F}.
existing: A C D* E F*
| | |
replacements: B C E F
result: A* B C D* E F
This is what happens to each link:
A gets deleted to A*.
B gets added.
C is retained, with possible extended DN changes.
D* stays in the list as a deleted link
E is retained like C
F is undeleted.
Backlinks are created in the case of B and F
The backlink for A is deleted
The backlinks are not changed for C and E or D* (D* has none)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This function fills out the DN and GUID fields of an unparsed
parsed_dn struct, which was happening in a few other places already.
In some places the GUID was not being filled out, which would probably
cause problems if the sorted_links switch was turned on.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
With the old binary search, we didn't get a pointer to the found
value, just a yes or no answer as to its existence. That meant we
ended up searching in both directions to find the links to be deleted.
As a consequence we needed to parse out the GUID of every existing
link, even if it wasn't being deleted.
Here we do it in one pass.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Links come over the wire as if sorted by memcmp() on the binary blobs,
not as sorted by GUID_compare(). Until a few patches ago, a newly
joined DC would have its linked attributes in the memcmp order. This
restores that behaviour.
This comparison could be made more efficient by storing the GUID in
the original state, but it does not seem to be a bottleneck.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Because both the list of added links and the list of existing links
are sorted, it is possible to interlace the two and obtain a merged
sorted list.
We avoid a great amount of talloc_realloc()ing by observing that the
merged list can't be longer than the sum of the two lists.
In the (common) case where there are many existing links but few being
added, we avoid parsing most of the existing link DNs and GUIDs if the
sorted_links feature flag is set.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Elsewhere we use the dsdb_dn pointer as a flag indicating parsed-ness,
so we have to be consistent.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
If we know that links from the database are in sorted order (via the
replmd_private->sorted_links flag), we can avoid actually parsing them
until it is absolutely necessary.
In many cases we are adding a single link to a long list. The location
of the single link is found via a binary search, so we end up parsing
log(N) DNs instead of N.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This will be initialised to false (zero) by default and will later come
from the compatibleFeatures in @SAMBA_DSDB
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Because we now load the dns with get_parsed_dns_trusted we have
to manually explode them in the upgrade tests.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This will allow us to maintain the list of links in sorted order.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is in preparation for improvements in our handling of linked
attributes where we make changes to the pointer in the process of
comparing it (for caching purposes).
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
