1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-27 03:21:53 +03:00
Commit Graph

2181 Commits

Author SHA1 Message Date
Björn Jacke
4c34a2d725 traffic.py: avoid inefficient string concatenations
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-09-24 12:22:44 +00:00
Björn Jacke
465e518d6c classicupgrade: fix a a bytes-like object is required, not 'str' error
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14136

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Björn Baumbach <bb@samba.org>

Autobuild-User(master): Björn Jacke <bjacke@samba.org>
Autobuild-Date(master): Mon Sep 23 12:58:20 UTC 2019 on sn-devel-184
2019-09-23 12:58:20 +00:00
Björn Jacke
dd0837c9a8 user.py: import tempfile module only where needed
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-09-21 19:33:25 +00:00
Garming Sam
a0fe965c36 param: Allow rpc server dynamic port range to roundtrip
Originally this parameter only set two other parameters, but never set
the original string. By setting the string as well, we can make it be
emitted by testparm -v correctly (instead of ''), and set it back as the
value for the parameter.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
2019-09-13 11:15:31 +00:00
Mathieu Parent
a03bafbcd5 Spelling fixes s/overriden/overridden/
Signed-off-by: Mathieu Parent <math.parent@gmail.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-09-01 22:21:26 +00:00
Andrew Bartlett
a4f0a6519c samba-tool domain provision: Remove experimental OpenLDAP support
This feature has long been obsolete, remaining only in the hope
that it might be revived in the future.

Specifically, in 2011 the S4 OpenLDAP backend HOWTO was removed:

 commit 1d46325af8
 Author: Andrew Bartlett <abartlet@samba.org>
 Date:   Wed Apr 27 22:42:29 2011 +1000

     Remove outdated S4 OpenLDAP backend HOWTO.

There is a project to revive this, hosted here:

https://github.com/Symas/samba
and
https://github.com/Symas/samba_overlays

However discussions at SambaXP with Nadezhda Ivanova
indicate a new approach with slapd being started by Samba
and taught to read native Samba ldb files is more likely
in the short term.

This has the advantage that Samba's provision and offline
tooling would not need to change, with the solution looking
more like how BIND9_DLZ has access to the Samba DB.

If any of this is required then reverting these patches will be
the least of the difficulties in bringing this to production.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Nadezhda Ivanova <nivanova@symas.com>
2019-08-30 08:32:30 +00:00
Aaron Haslett
961f07fb76 rpc samr: EnumDomainUsers perf improvement
EnumDomainUsers currently takes too long, significantly slowing down
calls to winbind's getpwent which is a core unix API. The time is taken
up by a GUID lookup for every record in the cached result. The advantages
of this approach are:
1. It meets the specified requirement that if a record yet to be returned
	by a search in progress (with a resume handle) is deleted or
	modified, the future returned results correctly reflect the
	new changes.
2. Memory footprint for a search in progress is only 16 bytes per record.

But, those benefits are not worth the significant performance hit
of the lookups, so this patch changes the function to run the search
and cache the RIDs and names of all records matching the search when
the request is made. This makes the memory footprint around 200 bytes
per record or up to 2MB per concurrent search for a 100k user database.
The speedup achieved by this change is around 50%, and in tandem with
some winbindd improvements as part of the same task has achieved around
15x speedup for getpwent.

The lost specification compliance is unlikely to cause a problem for any
known usage of this RPC call.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-08-30 07:08:36 +00:00
Andrew Bartlett
b4816861f2 s4-dns: Deprecate BIND9_FLATFILE and remove "rndc command"
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Aug 22 21:24:00 UTC 2019 on sn-devel-184
2019-08-22 21:24:00 +00:00
Björn Jacke
4364231625 python: use os.urandom, which is available in python by definition
os.urandom also uses CSPRNG methods like getrandom() when the underlying OS
provides those.

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-08-22 12:28:30 +00:00
Aaron Haslett
6dcf00ba0a downgradedatabase: installing script
Installing downgrade script so people don't need the source tree for it.

Exception added in usage test because running the script without arguments
is valid. (This avoids the need to knownfail it).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-08-20 03:40:28 +00:00
Tim Beale
fdaaee8d3a downgradedatabase: rename to samba_downgrade_db
Just so that it's slightly less of a mouthful for users.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-08-20 03:40:28 +00:00
Tim Beale
a8cdbe0b82 tests: Avoid hardcoding relative filepath
If we move the test file, the test will break.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-08-20 03:40:28 +00:00
David Mulder
4be5ffdca6 Prevent samba-tool online backup crash
On some GPOs, getting a files ntacl throws an
NT_STATUS_ACCESS_DENIED. Catch and log the
failure when this happens.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14088

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Tim Beale <timbeale@samba.org>
2019-08-17 03:50:39 +00:00
Günther Deschner
4b88e1d997 tests/dcerpc/raw_protocol: Add more tests for DCERPC_AUTH_LEVEL_PACKET
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Signed-off-by: Guenther Deschner <gd@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Aug  1 16:59:02 UTC 2019 on sn-devel-184
2019-08-01 16:59:02 +00:00
Stefan Metzmacher
e0e68ea255 tests/dcerpc/raw_protocol: split test_spnego_integrity_request into 2 parts
This can be a generic test that can be used for more auth_levels.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-08-01 15:40:17 +00:00
Stefan Metzmacher
ff2f20439c tests/dcerpc/raw_protocol: split test_spnego_connect_request() into 2 parts
This can be a generic test that can be used for more auth_levels.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-08-01 15:40:17 +00:00
Garming Sam
7d2875bd70 tests: Add samba_upgradedns to the list of possible cmds
This will be used to test the replication scenario with no DNS partitions

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14051

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-31 02:50:24 +00:00
Garming Sam
d90ccce597 netcmd: Allow drs replicate --local to create partitions
Currently, neither the offline (--local) or online (normal replica sync)
methods allow partition creation post-join. This overrides the Python
default to not create the DB, which allows TDB + MDB to work.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14051

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-31 02:50:24 +00:00
Tim Beale
9d2fd08249 netcmd: Better error message for backup with no RID pool
Add a better error message (and what to do about it) if the user tries
to back up a DC that hasn't initialized its RID pool yet.

Seems to be a fairly common problem hit by users.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14048
RN: Added more informative error message if the 'samba-tool domain
backup' command fails due to no RID pool being present on the DC.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Jul 24 07:07:01 UTC 2019 on sn-devel-184
2019-07-24 07:07:01 +00:00
Tim Beale
256684c7a8 join: Use a specific attribute order for the DsAddEntry nTDSDSA object
Joining a Windows domain can throw an error if the HasMasterNCs
attribute occurs before msDS-HasMasterNCs. This patch changes the
attribute order so that msDS-HasMasterNCs is always first.

Previously on python2, the dictionary hash order was arbitrary but
constant. By luck, msDS-HasMasterNCs was always before HasMasterNCs, so
we never noticed any problem. With python3, the dictionary hash order
now changes everytime you run the command, so the order is
unpredictable.

To enforce a order, we can change to use an OrderedDict, which will
return the keys in the order they're added.

I've asked Microsoft to clarify the protocol requirement here WRT
attribute order. However, in the meantime we may as well fix the problem
for users.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14046
RN: When trying to join a Windows domain (with functional level 2008R2)
as an AD domain controller, the 'samba-tool domain join' command could
throw a python exception: 'RuntimeError ("DsAddEntry failed")'. When
this problem occurred, you would also see the message "DsAddEntry failed
with status WERR_ACCESS_DENIED info (8363, 'WERR_DS_NO_CROSSREF_FOR_NC')"
in the command output. This issue has now been resolved. Note that this
problem would only occur on Samba v4.10 when using the Python3 packages.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Jul 24 04:18:21 UTC 2019 on sn-devel-184
2019-07-24 04:18:21 +00:00
Tim Beale
630857c71e traffic_replay: Avoid DB full scans in LDAP searches
When generating LDAP search traffic, a full DB scan can be very costly.
Avoiding full-scan LDAP searches means that we can run traffic_replay
against a 100K user DB and get some sane results.

Because the traffic_learner doesn't record the LDAP search filter at all,
the traffic_replay LDAP searches default to being full scans.
Doing full scans meant that the LDAP search was usually the first packet
type to exceed the max latency and fail the test. It could also skew
results for the other packet types by creating big demands on memory/CPU/
DB-lock-time.

It's hard to know for sure exactly what real-world LDAP searches will
look like, but let's assume full scan searches will be fairly rare.
In traffic-model files we've collected previously, some of the
attributes are fairly unique (e.g. pKIExtendedKeyUsage), and as there
are some LDAP queries specified in MS specs (such as MS-GPOL and
MS-WCCE), it allows us to infer what the search filter might be.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-24 02:24:27 +00:00
Gary Lockyer
7abfa6778f traffic replay test: Populate total_converations and instance_id
Ensure that the total_conversations and instance_id attributes are
assigned a value in the replay contexts passed to test cases.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-24 02:24:27 +00:00
Gary Lockyer
6d6fe49938 traffic replay: Store the instance id in the replay context
Store the traffic runner instance id in the replay context.  Will be
used in subsequent commits.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-24 02:24:27 +00:00
Tim Beale
6b1f9b4a5f traffic_replay: Make use of SCOPE_BASE explicit
i.e. avoid hard-coded numbers.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-24 02:24:27 +00:00
Tim Beale
9f504fd5a3 traffic_replay: Store total conversations on the replay context
This is useful info to know, and will be used in subsequent commits.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-24 02:24:27 +00:00
Douglas Bagnall
3383390b4b tests/py-segfault: add messaging.deregister()
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-22 22:20:25 +00:00
Douglas Bagnall
c3a52e4c40 py segfault test: registry.open_hive
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-22 22:20:25 +00:00
Garming Sam
0bcfc550b1 gp_inf: Read/write files with a UTF-16LE BOM in GptTmpl.inf
Regression caused by 16596842a6

[MS-GPSB] 2.2 Message Syntax says that you have to write a BOM which I
didn't do up until this patch. UTF-16 as input encoding was marked much
higher up in the inheritance tree, which got overriden with the Python 3
fixes. I've now marked the encoding much more obviously for this file.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14004

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Fri Jul 19 02:20:47 UTC 2019 on sn-devel-184
2019-07-19 02:20:47 +00:00
Gary Lockyer
1c4b9ba36c prefork restart tests: Use echo server for back off
Use the echo server for the restart back off tests in the
samba.tests.prefork_restart tests instead of the kdc.  The kdc is not enabled
when the ADDC is built to run MIT Kerberos.  Changing the test to use
the echo server means it can be run when MIT Kerberos is enabled.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-07-16 08:13:12 +00:00
Douglas Bagnall
192386ede6 pyldb: remove ldb.open, which was never survivable
There was no way to call ldb.open without evoking signal 11, so it is
unlikely anyone was using it.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-07-10 04:32:13 +00:00
Douglas Bagnall
3af57daa84 py segfault test: ldb.open
There seems to be no way of using ldb.open without causing a segfault

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-07-10 04:32:13 +00:00
Douglas Bagnall
545e95386f py segfault test: ldb.register_module
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-07-10 04:32:13 +00:00
Douglas Bagnall
9fc8e2a6d0 tests/usage: test for --help consistency
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-05 01:05:20 +00:00
Douglas Bagnall
272a6c3c30 tests/samba-tool: test --help consistency
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-05 01:05:20 +00:00
Douglas Bagnall
027d35bda1 python/tests: helper function for checking --help consistency
Check that --help output doesn't contradict itself by assigning the same
option string to different meanings (which *does* happen in the ldb tools).

This will be used in the samba-tool help tests and the usage tests.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-05 01:05:20 +00:00
Douglas Bagnall
089034628b tests/usage: generalise to cover non-python scripts
It is not as simple as running everything executable, because for example
.so library files are marked as executable.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-05 01:05:20 +00:00
Douglas Bagnall
854e554c99 tests/usage: python scripts --help should be helpful
We want to be sure it says *something* and returns success.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-05 01:05:20 +00:00
Douglas Bagnall
538ffe1960 tests: ensure that most python scripts have usage text
When a script is run with the wrong arguments, it should at least say
something like this:

    Usage: samba-foo [OPTIONS]

For many samba scripts, especially without a server environment, having
no arguments is the wrong arguments.

Here we look for every executable file with '#![...]python[3]' on the
first line, and exclude certain files and directories that have excuses
to fail the test. For example, many selftest scripts are stream-oriented
and will hang forever waiting for stdin, which is not an error. Some
test modules are designed so they can be optionally run from the command
line, but this option is typically only used by the developer who is
writing them.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-05 01:05:19 +00:00
Gary Lockyer
7a0a2fab0c domain join: set ldb "transaction_index_cache_size" option
Set the "transaction_index_cache_size" on a join to improve
performance. These setting reduced a join to a 100k user domain from
105 minutes to 44 minutes.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-04 10:02:23 +00:00
Gary Lockyer
4335f4b4e4 domain join: enable ldb batch mode
Enable ldb "batch_mode" transactions duting a join to improve
performance.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-04 10:02:23 +00:00
Björn Baumbach
78e78a4092 samba-tool tests: add tests for contact management
Signed-off-by: Björn Baumbach <bb@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-04 02:07:21 +00:00
Björn Baumbach
3f10c8f25c samba-tool: implement contact management commands
Usage: samba-tool contact <subcommand>

Contact management.

Available subcommands:
  create  - Create a new contact.
  delete  - Delete a contact.
  edit    - Modify a contact.
  list    - List all contacts.
  move    - Move a contact object to an organizational unit or container.
  show    - Display a contact.

Signed-off-by: Björn Baumbach <bb@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-04 02:07:21 +00:00
Björn Baumbach
3a97122ff5 samba-tool tests: add test for 'samba-tool group edit' command
Signed-off-by: Björn Baumbach <bb@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-04 02:07:21 +00:00
Björn Baumbach
b2fb02585f samba-tool group: add 'edit' command to edit an AD group object
Same like the samba-tool user edit command.

Signed-off-by: Björn Baumbach <bb@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-04 02:07:21 +00:00
Björn Baumbach
6c26abd981 samba-tool tests: add test for 'samba-tool computer edit' command
Signed-off-by: Björn Baumbach <bb@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-04 02:07:21 +00:00
Björn Baumbach
2c64a25fc0 samba-tool computer: add 'edit' command to edit an AD computer object
Similar to the samba-tool user edit command.

Signed-off-by: Björn Baumbach <bb@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-04 02:07:20 +00:00
Björn Baumbach
08742d1df8 samba-tool user edit: avoid base64 encoded strings in editable ldif if possible
Use clear text arguments strings if possible. Makes it more comfortable
for users to edit the user objects attributes.

Remove test from knownfail:
  samba.tests.samba_tool.user_edit.change_attribute_force_no_base64

Signed-off-by: Björn Baumbach <bb@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-04 02:07:20 +00:00
Björn Baumbach
33fe481760 samba-tool tests: add test for 'samba-tool user edit', using LDB_FLAG_FORCE_NO_BASE64_LDIF
Test to edit a user: Change attributes with LDB_FLAG_FORCE_NO_BASE64_LDIF

Signed-off-by: Björn Baumbach <bb@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-04 02:07:20 +00:00
Björn Baumbach
4954118b81 samba-tool user edit: simplify code
Use "None"-changetype here, instead of "Add". This avoids the need to
remove the changetype line afterwards.

Signed-off-by: Björn Baumbach <bb@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-04 02:07:20 +00:00
Björn Baumbach
64e37e3822 samba-tool user edit: use ldb methods to create ldif to modify user
Remove tests from knownfail:
  samba.tests.samba_tool.user_edit.add_attribute_base64
  samba.tests.samba_tool.user_edit.add_attribute_base64_control
  samba.tests.samba_tool.user_edit.change_attribute_base64_control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14003

Signed-off-by: Björn Baumbach <bb@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-04 02:07:20 +00:00