1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-11 05:18:09 +03:00
Commit Graph

107049 Commits

Author SHA1 Message Date
Lumir Balhar
1ac5bf21a1 python: samba.auth: Port samba.auth to Python 3 compatible form
Port samba.auth Python module to Python 3 compatible form and
enable tests execution with Python 3.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:11 +01:00
Lumir Balhar
47b014899a python: wscript_build: Build some DCE/RPC modules with Python 3
Samba.auth Python module depends on a lot of DCE/RPC modules which
have to be built with Python 3 to make port of samba.auth to
Python 3 possible.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:11 +01:00
Lumir Balhar
a40d7f18dd python: samba.dcerpc: Port security module to Python 3 comp. form
Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:11 +01:00
Douglas Bagnall
e481aed28d dcerpc/misc tests: asset GUID ordering in python 2 and 3
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:11 +01:00
Lumir Balhar
9843ccef87 python: samba.tests.dcerpc.misc: Port and enable tests
Port tests of samba.dcerpc.misc module to Python 3 compatible form
and enable their execution with Python 3.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:11 +01:00
Lumir Balhar
3b1c0ba98a python: samba.dcerpc: Port RPC related stuff to Python 3
Port RPC related stuff like samba.dcerpc.misc and samba.dcerpc
Python modules and pyrpc_util to Python 3 compatible form.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:11 +01:00
Lumir Balhar
2de5fa45e0 python: pidl: Port Python interface generator
Port PIDL generator of Python interfaces to generate interfaces in
Python 3 compatible form.

Python 2.7 is now required, so we can use PyCapsule in both versions.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:11 +01:00
Lumir Balhar
5edb0b4ebd python: samba.tests: Enable Python 3 tests for ported modules
Enable tests with Python 3 for Python 3 compatible modules.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:11 +01:00
Andrew Bartlett
b7a5308612 buildtools: Work around a . being in the target name when building python3 helpers
The pyparam_util module becomes pyparam_util.cpython_35m_x86_64_linux_gnu but
the command line parser for -D stops at the first .

That we even set -DSTATIC_subsystem_MODULES_PROTO for these subsystems without
any modules ever declared is left for another time

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:11 +01:00
Lumir Balhar
fe8bba5f81 python: wscript_build: Build some modules for Python 3
Update a few wscript_build files to build Python 3-compatible modules
for Python 3.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:11 +01:00
Lumir Balhar
6fa125e121 python: Make top-level samba modules Python 3 compatible
New file compat.py will help with porting to Python 3. For now, it
contains only PY3 variable based on six.PY3 which simplifies
condition mentioned below.

The added `if not PY3` conditions enable us to bootstrap running
tests with Python 3 even if most modules are not ported yet.
The plan is to move modules outside this condition as they are ported.
The `PY3` condition is currently used only in tests and for
the samba._ldb module which is not ported yet and has a lot of
dependencies.

The other changes are related to differences between Python 2 and 3.
Python 2.6 introduced the `0o` prefix for octal literals as an
alternative to plain `0`. In Python 3, support for plain `0` is
dropped and octal literals have to start with `0o` prefix.
Python 2.6 introduced a clearer `except` syntax:
`except ExceptionType as target:` instead of
`except ExceptionType, target:`. In Python 3, the old syntax
is no longer allowed.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:11 +01:00
Lumir Balhar
211df4a1e4 python: samba.tests.dcerpc: Move Class RawDCERPCTest to separated file.
The class is quite big, used in only one place, and it complicates
situation around bootstrapping of Python 3 port.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:10 +01:00
Lumir Balhar
9c55bb9f65 python: samba.tests.glue: Add new tests for samba._glue.
Add new file with tests of samba._glue module.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:10 +01:00
Lumir Balhar
b454b0903c python: samba._glue: Port samba._glue module to Python 3.
Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:10 +01:00
Lumir Balhar
5123f15072 python: samba.tests.param: Add missing tests
Add some new tests of samba.param Python bindings.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:10 +01:00
Lumir Balhar
31cef92cd3 python: samba.param: Port param module to Python 3
Port Python bindings of samba.param module to
Python3-compatible form.

Because native Python file objects are officially
no longer backed by FILE*, API of some _dump()
functions is changed. File argument is now
optional and contains only name of file. Stdout
is default if no file name is specified. Otherwise
opening and closing files is done on C layer
instead of Python.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:10 +01:00
Lumir Balhar
035e6dce4f python: samba.tests.credentials: Python 3 compatible tests
Port test of pycredentials to Python 3 compatible form.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:10 +01:00
Lumir Balhar
1dab2b493e python: samba.credentials: Port pycredentials.c to Python3-compatible form.
Port Python bindings of samba.credentials module to
Python3-compatible form using macros from py3compat.h.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:10 +01:00
Andrew Bartlett
583ff0a0e3 lib/ldb: Enable use of a python3 pyldb-util system library
To do this, we have to install a .pc file for the python3 pyldb-util

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Incorportaing fixes by Petr Viktorin <pviktori@redhat.com>

Signed-off-by: Petr Viktorin <pviktori@redhat.com>
2017-03-10 07:31:10 +01:00
Andrew Bartlett
f5cafee0c7 talloc: use the system pytalloc-util for python3 as well
This involves installing a .pc file for the python3 library as well

To get the .pc file generated and installed is quite a mission, we
have to rework the talloc build system to ensure that the second 'env'
created for EXTRA_PYTHON has everything set up on it, the
TALLOC_VERSION in particular.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Incorportaing fixes by Petr Viktorin <pviktori@redhat.com>

Signed-off-by: Petr Viktorin <pviktori@redhat.com>
2017-03-10 07:31:10 +01:00
Douglas Bagnall
132cbad9ce scripts/traffic_summary: documentation typo
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:10 +01:00
Douglas Bagnall
4485ba5f3c ./examples/scripts/SambaConfig.py: fix typo in "continue"
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:10 +01:00
Douglas Bagnall
d424c7d5af python/examples/winreg: two variable name typos on a single line
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:10 +01:00
Douglas Bagnall
b7cd0f2843 python sites/subnets: correctly spell variable name
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:10 +01:00
Douglas Bagnall
cedb991e6f python provision: FDSBackend takes forced uri
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:10 +01:00
Douglas Bagnall
b9c56142b5 python/remove_dc: avoid using non-existent variable
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:10 +01:00
Douglas Bagnall
732233ff3d samba-tool domain: correctly spell variable name
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:09 +01:00
Douglas Bagnall
495383c9a6 python/join: correct spelling of "ctx.del_noerror"
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:09 +01:00
Douglas Bagnall
0779d66718 selftest: remove unused broken client.py
Nothing uses this, and pyflakes points out it is unusable:

./selftest/client.py:60: undefined name 'prefix_abs'
./selftest/client.py:69: undefined name 'opts'
./selftest/client.py:70: undefined name 'interfaces'

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:09 +01:00
Douglas Bagnall
765275ce9a gitignore: add some hidden files
.gdb_history is generated by gdb,
.emacs* are generated by emacs, and
.clang* by clang.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:09 +01:00
Douglas Bagnall
17dce5d84f perftests/ad_dc_search: do less work in expensive member searches
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:09 +01:00
Douglas Bagnall
bb4ad8dffe pyldb: p3k readiness: allow single unicode string in msg element
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:09 +01:00
Uri Simchoni
2772fa4fbf talloc: fix doxygen of talloc_move
talloc_move cannot fail.

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Mar 10 07:30:40 CET 2017 on sn-devel-144
2017-03-10 07:30:40 +01:00
Volker Lendecke
59b580f6c7 auth_ntdomain3: Correctly handle !authoritative
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 03:28:26 +01:00
Volker Lendecke
7d3c197e61 auth_winbind4: Correctly handle !authoritative
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 03:28:26 +01:00
Volker Lendecke
9abc80e0e4 auth_winbind3: Correctly handle !authoritative
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 03:28:26 +01:00
Ralph Boehme
d3e2e41995 s3/smbd: add my copyright to open.c
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:26 +01:00
Ralph Boehme
fd03420c4f s4/torture: some tests for kernel oplocks
Bug: https://bugzilla.samba.org/show_bug.cgi?id=7537

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:26 +01:00
Ralph Boehme
6e54d8d2bd s3/selftest: adopt config.h check from source4
No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=7537

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:26 +01:00
Ralph Boehme
b35a296a27 s3/smbd: fix deferred open with streams and kernel oplocks
I noticed smbd can get stuck in an open() call with kernel oplocks
enabled and named streams (provided by vfs_streams_xattr):

- client opens a file and with an exclusive oplock

- client starts writing to the file

- client opens an existing stream of the file

- the smbd process gets stuck in an open()

What happens is:

we had setup a locking.tdb record watch in defer_open(), the watch was
triggered, we reattempted the open and got stuck in a blocking open
because the oplock holder (ourselves) hadn't given up the oplock yet.

Cf e576bf5310 for the commit that added
the kernel oplock retry logic. tldr: with kernel oplocks the first open
is non-blocking, but the second one is blocking.

Detailed analysis follows.

When opening a named stream of a file, Samba internally opens the
underlying "base" file first. This internal open of the basefile suceeds
and does *not* trigger an oplock break (because it is an internal open
that doesn't call open() at all) but it is added as an entry to the
locking.tdb record of the file.

Next, the stream open ends up in streams_xattr where a non-blocking
open() on the base file is called. This open fails with EWOULDBLOCK
because we have another fd with a kernel oplock on the file.

So we call defer_open() which sets up a watch on the locking.tdb record.

In the subsequent error unwinding code in open_file_ntcreate() and
callers we close the internal open file handle of the basefile which
also removes the entry from the locking.tdb record and so *changes the
record*.

This fires the record watch and in the callback defer_open_done() we
don't check whether the condition (oplock gone) we're interested in is
actually met. The callback blindly reschedules the open request with
schedule_deferred_open_message_smb().

schedule_deferred_open_message_smb() schedules an immediate tevent event
which has precedence over the IPC fd events in messaging, so the open is
always (!) reattempted before processing the oplock break message.

As explained above, this second open will be a blocking one so we get
stuck in a blocking open.

It doesn't help to make all opens non-blocking, that would just result
in a busy loop failing the open, as we never process the oplock break
message (remember, schedule_deferred_open_message_smb() used immediate
tevent events).

To fix this we must add some logic to the record watch callback to check
whether the record watch was done for a kernel oplock file and if yes,
check if the oplock state changed. If not, simply reschedule the
deferred open and keep waiting.

This logic is only needed for kernel oplocks, not for Samba-level
oplocks, because there's no risk of deadlocking, the worst that can
happen is a rescheduled open that fails again in the oplock checks and
gets deferred again.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=7537

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:26 +01:00
Ralph Boehme
1a6c82e5d5 s3/smbd: all callers of defer_open() pass a lck
No change in behaviour. Update the function comment explaining how it
works and relies on lck for a record watch.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=7537

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:26 +01:00
Ralph Boehme
7fa2f11594 s3/smbd: remove async_open arg from defer_open()
All remaining callers pass false.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=7537

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:25 +01:00
Ralph Boehme
ad8c36125f s3/smbd: fix schedule_async_open() timer
schedule_async_open() was calling defer_open with sharemode lock = NULL,
as a result there was never an active 20 s timeout.

This has been broken since the commits in

$ git log --reverse -p -10 8283fd0e00

Just roll our own deferred record instead of calling defer_open() and
also set up timer that, as a last resort, catches stuck opens and just
exits for now.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=7537

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:25 +01:00
Ralph Boehme
beaba62228 s3/smbd: add and use retry_open() instead of defer_open() in two places
Add a new function that does an immediate open rescheduling.

The first deferred open this commit changes was never scheduled, as the
scheduling relies on a timeout of the watch on the sharemode lock.

This has been broken since the commits in

$ git log --reverse -p -10 8283fd0e00

That patchset added the dbwrap watch record logic to defer_open() and
removed the timers.

I'm doing this mainly to untangle the defer_open() logic which is
complicated by the lck arg.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=7537

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:25 +01:00
Ralph Boehme
b17ff9b181 s3/smbd: simplify defer_open()
Add a helper function deferred_open_record_create() that creates a
deferred_open_record and let all callers pass all needed arguments
individually.

While we're at it, enhance the debug message in defer_open() to print
all variables.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=7537

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:25 +01:00
Ralph Boehme
8580adc1d9 s3/smbd: req is already validated at the beginning of open_file_ntcreate()
req can't be NULL because the if condition surrounding this code checks
!(oplock_request & INTERNAL_OPEN_ONLY).

Bug: https://bugzilla.samba.org/show_bug.cgi?id=7537

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:25 +01:00
Ralph Boehme
f5631f6b35 s3/smbd: add comments and some reformatting to open_file_ntcreate()
No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=7537

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:25 +01:00
Ralph Boehme
6924e72ade s3/smbd: add const to get_lease_type() args
Bug: https://bugzilla.samba.org/show_bug.cgi?id=7537

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:25 +01:00
Ralph Boehme
fe473f805a s3/wscript: fix Linux kernel oplock detection
Fix a copy/paste error, the Linux kernel oplocks check was copied from
the change notify support check.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=7537

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-03-10 03:28:25 +01:00
Volker Lendecke
373e0bdf12 winbindd: Remove an unused #define
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Fri Mar 10 00:00:15 CET 2017 on sn-devel-144
2017-03-10 00:00:15 +01:00