1
0
mirror of https://github.com/samba-team/samba.git synced 2025-02-08 05:57:51 +03:00

1785 Commits

Author SHA1 Message Date
Joe Guo
24b1aa9da2 logger: use color automatically for a tty
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Nov 21 10:46:20 CET 2018 on sn-devel-144
2018-11-21 10:46:20 +01:00
Douglas Bagnall
c4bd785ac1 samba-tool ldapcmp: print DNs on predictable order
Rather than unstable hash order. Ideally we'd do them in proper DN order.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:20 +01:00
Joe Guo
7ebbd3731d netcmd/ldapcmp: make code pythonic
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:20 +01:00
Joe Guo
26f0992c24 netcmd/ldapcmp: promote re object to global
Then we can reuse the re obj.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:20 +01:00
Joe Guo
7d9282bf7c netcmd/ldapcmp: use set instead of list to compare attrs
This will simplify the logic and improve performance.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:20 +01:00
Joe Guo
1085550985 netcmd/ldapcmp: fix wrong way for string copy
Two mistakes here:

- res[:-1] will copy but lost the last char
- string is immutable in python, there is no need to copy it explicitly

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:20 +01:00
Joe Guo
2851cd5bcc netcmd/ldapcmp: pass --skip-missing-dn to LDAPBase
This option has default value False, and was actually not passed down from cli
to LDAPBase. However, LDAPBase.__init__ has default value True for it.

After the change, a few tests using ldapcmp are affected.
Add --skip-missing-dn explicitly to keep the behavior consistent,
otherwise test will fail.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:19 +01:00
Joe Guo
bbd082e792 netcmd/ldapcmp: avoid modifying data while looping on dict
Just define another dict for return value, seems no need to modify
original dict.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:19 +01:00
Joe Guo
0471387089 netcmd/ldapcmp: rm unused global var summary
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:19 +01:00
Joe Guo
f821f84f2e netcmd/ldapcmp: use set instead of list to find missing DNs
This simplify the logic and improve performance a lot.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:19 +01:00
Joe Guo
86882bd12e netcmd/ldapcmp: avoid list comprehension in for loop
The list comprehension will repeat for each item.
For large database, this make the command freeze.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:19 +01:00
Joe Guo
e71d0d7120 netcmd/ldapcmp: add choices arg to --view option
So we don't need to validate ourselves.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:19 +01:00
Joe Guo
263f020784 netcmd/ldapcmp: add choices arg to --scope option
So we don't need to validate ourselves.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:18 +01:00
Joe Guo
683342b7a0 netcmd/ldapcmp: rename __eq__ to diff
This method actually changed both objects and print info.
__eq__ is not a proper name and is not designed for this case.
Rename to diff.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:18 +01:00
Joe Guo
5f47c04cf3 netcmd/ldapcmp: fix typo for Bundle
Bundel -> Bundle

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-21 07:46:18 +01:00
Gary Lockyer
fa3ea1cfc1 tests samr: Extra tests for samr_EnumDomainUserss
Add extra tests to test the content returned by samr_EnumDomainUsers,
and tests for the result caching added in the following commit.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-20 22:14:17 +01:00
Gary Lockyer
7dd7800a88 test samr: Extra tests for samr_EnumDomainGroups
Add extra tests to test the content returned by samr_EnumDomainGroups,
and tests for the result caching added in the following commit.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-20 22:14:17 +01:00
Gary Lockyer
d9f7b6d63f tests samr: remove PEP8 warnings
Remove PEP8 warnings from the samr tests.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-20 22:14:17 +01:00
Gary Lockyer
9a1277c1ec tests samr: Extra tests for samr_QueryDisplayInfo
Add extra tests to test the content returned by samr_QueryDisplayInfo,
which is not tested for the ADDC.  Also adds tests for the result
caching added in the following commit.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-20 22:14:16 +01:00
Garming Sam
ac90c9faa7 sync_passwords: Remove dirsync cookie logging for continuous operation
Under normal operation, users shouldn't see giant cookies in their logs.
We still log the initial cookie retrieved from the cache database, which
should still be helpful for identifying corrupt cookies.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13686

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-20 22:14:16 +01:00
Tim Beale
008449d99f netcmd: Flush replUpToDateVector when restoring offline backup
The replUpToDateVector could be incorrect after an offline backup was
restored. This means replication propagation dampening doesn't work
properly. In the worst case, a singleton DC would have no
replUpToDateVector at all, and so *all* objects created on that DC get
replicated every time a new DRS connection is established between 2 DCs.
This becomes a real problem if you used that singleton DC to create 100K
objects...

This patch flushes the replUpToDateVector when an offline backup gets
restored. We need to do this before we add in the new DC and remove the
old DCs.

Note that this is only a problem for offline backups. The online/rename
backups are received over DRS, and as part of the replication they
receive the latest replUpToDateVector from the DC being backed up.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-20 01:33:33 +01:00
Tim Beale
5ca2726a40 netcmd: Small backup refactor to avoid compatiblity problems
It will be easy to forget that the backupType marker doesn't exist on
v4.9. However, this seems like a dumb reason not to support v4.9
backup-files. Add a wrapper function to avoid potential problems
cropping up in future.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-20 01:33:33 +01:00
Tim Beale
6e5ebbfb65 netcmd: Add backupType marker to backed-up DB
We are starting to hit restore cases that are only applicable to a
particular type of backup. We already had a marker to differentiate
renames, but differentiating offline backups would also be useful.

Note that this raises a slight compatibility issue for backups created
on v4.9, as the marker won't exist. However, it's only offline backups
we will use this marker for (at the moment), and this option doesn't
exist on v4.9, so there's no problem.

Removing the markers has been refactored out into a separate function to
handle the optional presence of the new marker.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-20 01:33:33 +01:00
Tim Beale
65e7476a87 tests: Add assertion that replUpToDateVector is present after backup
We noticed that offline backups were missing a replUpToDateVector for
the original DC, if the backup was taken on a singleton DC. This patch
adds an assertion to the existing test-cases to highlight the problem.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-20 01:33:33 +01:00
Noel Power
a5c8ab277b python/samba/tests: Ensure samba_dnsupdate called with correct python version
We need to examine the contents of PYTHON env variable which should defined the
python version to be used when running tests.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-14 05:07:17 +01:00
Noel Power
8bf82d9f85 python/samba/tests: PY3 Fix str/bytes issue for json.loads
Python 3.4 seems to need a string

    parsed = json.loads (out_jsobj)
  File "/usr/lib/python3.4/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

however Python 3.5 seems to be happy to consume bytes (or string)

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-14 05:07:17 +01:00
Noel Power
f176461cf2 python/samba: PY3 fix can't compare string with int
Testing a string against an int value is illegal and
is not necessary in this case, this patch removes the
problematic test.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-14 05:07:17 +01:00
Noel Power
32a490d681 python/samba/netcmd: PY3 port samba4.blackbox.ldapcmp_restore
Convert attribute from bytes to str
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-14 05:07:16 +01:00
Noel Power
fdb3aa487f python/samba/netcmd: PY3 port samba.tests.samba_tool.edit
Need to write bytes to file

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-14 05:07:16 +01:00
Noel Power
b16043f57d python/samba/tests: PY3 port samba.tests.blackbox.netads_json
convert output from check_output to text
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-14 05:07:16 +01:00
Noel Power
108109360a python/samba/tests: Port samba.tests.blackbox.samba_dnsupdate for PY2/PY3
Make sure either the output of tests and/or the item we are searching match
in type. Output of cmd in python3 is bytes, depending on the was the test is
written it may be easier just to convert all output or just a single string
that is used in the test

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-14 05:07:16 +01:00
Noel Power
0ef6e3da4b testprogs/blackbox: PY3 bulk change for python scripts use correct python
Change all instance where python scripts are called so that the
correct python version as specified by $PYTHON is used

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-14 05:07:16 +01:00
Noel Power
5b4c664856 python/samba/provision: PY3 port samba4.dlz_bind9.version
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-14 05:07:15 +01:00
Tim Beale
b161b3a891 mdb_util: Better error message if lmdb-utils not installed
mdb_copy() was dutifully checking the PATH for the mdb_copy executable,
then, if it didn't find it, blindly proceeding anyway and trying to run
a non-existent executable. This resulted in a cryptic error:

  ERROR(<type 'exceptions.OSError'>): uncaught exception - [Errno 2] No
    such file or directory

Add in an extra check that we actually find the executable and raise a
better human-readable exception if we don't.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Nov  9 21:07:47 CET 2018 on sn-devel-144
2018-11-09 21:07:47 +01:00
Andreas Schneider
2d512b278e debug: Use debuglevel_(get|set) function
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Nov  8 11:03:11 CET 2018 on sn-devel-144
2018-11-08 11:03:11 +01:00
Tim Beale
a0b5f4b7b0 traffic_replay: Rework machine accounts to remove redundant code
generate_users_and_groups() now generates the machine acounts as well as
the user accounts, so it seems there's no need to also have
generate_traffic_accounts(), which does the same job.

Instead, we can just pass through the number of machine acounts to
generate_users_and_groups() and delete the other function.

Also updated generate_users_and_groups() so that machine_accounts is
no longer optional (we want to create machine accounts in all cases).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07 17:55:09 +01:00
Tim Beale
1d7fb66f76 traffic_replay: Make packet generation work on a pre-populated DB again
Generate separate machine accounts for populating a large DB vs
replaying network traffic.

We want to use different userAccountControl flags in each of the above
cases (i.e. commit 3338a3e257fa9f28). However, this means that once you
use the --generate-users-only option, you can't replay network packets
against the machine accounts.

We can avoid this problem by creating separate machine accounts for each
of 2 different cases, e.g. STGM-0-x machines for traffic-replay, and
PC-0-x machines for padding out the database.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07 17:55:09 +01:00
Tim Beale
c7fe481477 traffic_replay: Make sure naming assumptions are in a single place
The traffic_replay group/user/machine account names follow a standard
format. This adds a function to generate the machine-name. It also makes
sure the existing user_name() function gets called in all applicable
places.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07 17:55:09 +01:00
Tim Beale
51917fc07f traffic_replay: Move 'traffic account' flag up a level
We create machine accounts for 2 different purposes:
1). For traffic generation, i.e. testing realistic network packets.
2). For generating a realistic large DB.

Unfortunately, we want to use different userAccountControl flags for
the 2 different cases. Commit 3338a3e257fa9f28 changed the flags used
for case #2, but this breaks case #1.

The problem is generate_users_and_groups() is called in both cases,
so we want the 'traffic account' flag passed into that function.
This ensures that the machine accounts get created with the appropriate
userAccountControl flags for the particular case you want to test.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07 17:55:08 +01:00
Tim Beale
85b6d88989 traffic_replay: Move machine account creation
I was assuming that generate_users_and_groups() only gets called in the
--generate-users-only case. However, it also gets called in the default
traffic replay case.

This patch reworks the code so that the number of machine accounts to
create gets passed in, and the 'create 25% more computers than users'
assumption only applies to the --generate-users-only case.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07 17:55:08 +01:00
Douglas Bagnall
fd089c37a1 samba-tool gpo: convert pseudo-method into method
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07 17:55:08 +01:00
Douglas Bagnall
9461033094 samba-tool gpo: add helper method for tmpdir construction
A few of the gpo commands use an identical temporary directory structure
that can be constructed using shared code.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07 17:55:08 +01:00
Douglas Bagnall
e3320b6d3d samba-tool gpo: do not use predictable temp directory
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07 17:55:08 +01:00
Douglas Bagnall
a9682f94d4 python/upgradehelpers: use mkstemp, not mktemp
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07 17:55:08 +01:00
Tim Beale
2229f4620d drs_utils: Avoid invalid dereference of v8 requests
req.more_flags only exists for v10 requests, so we throw an exception if
we try to dereference that field on a v8 (or v5) request. Unfortunately,
we were checking that we support v10 *after* we had tried to access the
more_flags. This patch fixes up the order of the checks.

This may be a problem trying to replicate with an older Windows DC
(pre-2008R2), and was reported on the samba mailing-list at one point:
https://lists.samba.org/archive/samba/2018-June/216541.html

Unfortunately this patch doesn't help the overall situation at all (the
join will fail because we can't resolve the link target and we can't use
GET_TGT). But it now gives you a more meaningful error, i.e.

  ERROR(runtime): uncaught exception - (8639, "Failed to process 'chunk'
    of DRS replicated objects: DOS code 0x000021bf"
instead of:
  ERROR(<type 'exceptions.AttributeError'>): uncaught exception -
    'drsuapi.DsGetNCChangesRequest8' object has no attribute 'more_flags'

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Tim Beale <timbeale@samba.org>
Autobuild-Date(master): Tue Nov  6 07:15:33 CET 2018 on sn-devel-144
2018-11-06 07:15:33 +01:00
Tim Beale
63bfdb3c11 drs_utils: Fix some long lines
Tweak the code slightly to avoid some 80+ character lines.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-06 03:39:11 +01:00
Tim Beale
02d41fea33 join: Fix join large-DB timeout against Windows
The LDAP connection can also timeout when trying to join a Windows DC
with a very large database. However, in this case Windows gives a
slightly different error message (NT_STATUS_CONNECTION_RESET instead of
NT_STATUS_CONNECTION_DISCONNECTED).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13612

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-06 03:39:11 +01:00
Noel Power
fc047c2cf4 python/samba/gp_parse: PY2/PY3 Decode only when necessary
In python2 we decode str types in load_xml, in python3 these are
str class(s) which we cannot decode.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-11-05 20:05:24 +01:00
Noel Power
6476ef589e python/samba/tests/samba_tool: PY2/PY3 compat port for test
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-11-05 20:05:24 +01:00
Noel Power
16596842a6 python/samba/gp_parse: Fix mulitple encode step with write_section
In python2 as far as I can see GptTmplInfParser.write_binary more
or less works by accident.

write_binary creates a writer for the 'utf8' codec, such a writer
should consume unicode and emit utf8 encoded bytes. This writer
is passed to each of the sections managed by GptTmplInfParser as
follows

    def write_binary(self, filename):
        with codecs.open(filename, 'wb+',
                         self.encoding) as f:
            for s in self.sections:
                self.sections[s].write_section(s, f)

And each section type itself is encoding its result to 'utf-16-le'
e.g.
    class UnicodeParam(AbstractParam):
         def write_section(self, header, fp):
            fp.write(u'[Unicode]\r\nUnicode=yes\r\n'.encode(self.encoding)

But this makes little sense, it seems like sections are encoded to one
encoding but the total file is supposed to be encoded as ut8??? Also
having an encoding per ParamType doesn't seem correct.

Bizarely in PY2 this works and it actually encodes the whole file as utf-16le
In PY3 you can't do this as the writer wants to deal with strings not bytes
(after the extra encode phase in 'write_section'.

So, changes here are to remove the unnecessary encoding in each 'write_section'
method, additionally in GptTmplInfParser.write_binary the
codecs.open call now uses the correct codec (e.g. 'utf-16-le') to write

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-11-05 20:05:24 +01:00