1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-15 23:24:37 +03:00

112861 Commits

Author SHA1 Message Date
Noel Power
c46fd168fc python/samba/netcmd: Protect variable that can be None
In py3 None variable cannot be compared with '>' '<' etc operators

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-07-13 01:12:24 +02:00
Noel Power
352eee57ae python/samba/netcmd: fix py2/py3 bytes usage for replace
base64.b64encode returns bytes in py3 make sure associated replace
uses 'b' for strings passed to replace

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-07-13 01:12:24 +02:00
Noel Power
e915d18e55 s4/torture/drs/python: Py2/Py2 fix tab/space also incorrect unicode usage
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-07-13 01:12:24 +02:00
Noel Power
7b170206b4 lib/ldb: Implement a bytes derived object for attributes py2/py3
ldb attributes are either bytes (py3) or str (py2)

Some places in the code do str(res[0]['attribute'][0])
which results in
   'result' (py2)
  b'result' (py3)

or more commonly the attribute is used to construct a string e.g.
   "blah=" + res[0]['attribute'][0] + ",foo,bar=...."

giving
   "blah=result,foo,bar=...." (py2)
and very unhelpfully
   "blah=b'result',foo,bar=...." (py3)

lots of code already constructs various strings for passing to other
api using the above. To avoid many excessive
    res[0]['attribute'][0].decode('utf8')

code like 'res[0]['attribute'][0]'

will now return LdbBytes (a new object subclassing 'bytes') in py3
instead of bytes. This object has a custom '__str__' method which
attempts to return a string decoded to uft8. In Py2 this will behave as
it did previously (this is the safer option at the moment)

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-07-13 01:12:24 +02:00
Noel Power
337ae8cd58 python/samba/tests: remove Py2 specific imports.
Remove some python2 specific import, probably this was due to
previous unavailability for some c-modules in python3

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-07-13 01:12:24 +02:00
Noel Power
7d43571169 python/samba/emulate: Fix some more missed exception tuple assignments
In python3 we need to change

    except LdbError as e:
-        (status, _) = e
to
    except LdbError as e:
+        (status, _) = e.args

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-07-13 01:12:24 +02:00
Noel Power
d5cd9af7b5 python/samba/tests: Py2/Py3 port for hexdump
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-07-13 01:12:24 +02:00
Noel Power
5b96e34981 python/samba/tests: Py2/Py3 allow import of ndr_(un)pack to work
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-07-13 01:12:24 +02:00
Joe Guo
9b164637f0 samdb: use int for get and set methods instead of digit str
This will make the API work in a nature way.

Also, because of a defect in ldb API, code like `res[0]["maxPwdAge"][0]`
will return bytes even in Python3, which will cause trouble. By casting
the value to int, we avoid the str/bytes issue.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-13 01:12:24 +02:00
David Mulder
f3358f048d gpo: Specify samba module when importing from gpclass
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jul 13 01:05:22 CEST 2018 on sn-devel-144
2018-07-13 01:05:22 +02:00
David Mulder
b05b6198a3 gpo: Don't duplicate guids in the apply log
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:23 +02:00
David Mulder
de8b30a726 gpo: Add user policy extensions
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:23 +02:00
David Mulder
126b789268 gpo: Dynamically load gp_exts
This loads Group Policy Client Side Extensions
similar to the way that they are loaded on a
Windows client. Extensions are installed to a
configuration file in the samba cache path where
they receive a unique GUID matched with the path
to the python gp_ext file. Classes which inherit
from the gp_ext class (as defined in gpclass.py)
will be dynamically loaded.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:23 +02:00
David Mulder
fb6c250bde gpo: Tests for gp_ext register/unregister
Adds testing for the gp_ext register and
unregister functions, as well as testing
the list function.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:23 +02:00
David Mulder
4436b6703c gpo: add list_gp_extensions for listing registered gp extensions
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:23 +02:00
David Mulder
b4f09af27a gpo: add unregister_gp_extension for unregistering gp extensions
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:23 +02:00
David Mulder
387dc358d3 gpo: add register_gp_extension for registering gp extensions
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:23 +02:00
David Mulder
46629f5321 param: Add python binding for lpcfg_state_path
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:23 +02:00
David Mulder
2412c70481 gpo: Offline policy application via cache
Read policy files from the cache, rather than
the sysvol. This enables offline policy apply.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:23 +02:00
David Mulder
57faf35cf8 gpo: Read GPO versions locally, not from sysvol
Non-kdc clients cannot read directly from the
sysvol, so we need to store the GPT.INI file
locally to read each gpo version.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:22 +02:00
David Mulder
4c7348e44d python: Allow forced signing via smb.SMB()
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:22 +02:00
David Mulder
2494e33822 gpo: Disable python3 testing
The gpo module doesn't work in python3 yet,
causing this test to fail on python3.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13525

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:22 +02:00
David Mulder
ce9ac51ee2 gpo: Fix asserts in gpo testing
These tests weren't using python's unit testing
asserts.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 22:11:22 +02:00
Stefan Metzmacher
3eee52b44d pthreadpool: allocate glue->tctx on glue as memory context.
This means it will go aways together with glue and thte event context.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Jul 12 17:18:01 CEST 2018 on sn-devel-144
2018-07-12 17:18:00 +02:00
Stefan Metzmacher
25756425aa pthreadpool: maintain a global list of orphaned pthreadpool_tevent_jobs
Instead of leaking the memory forever, we retry the cleanup,
if other pthreadpool_tevent_*() functions are used.

pthreadpool_tevent_cleanup_orphaned_jobs() could also be called
by external callers.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
fa070d9007 pthreadpool: make use of pthreadpool_stop() in pthreadpool_tevent_destructor()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
791c05144e pthreadpool: add pthreadpool_tevent_job_cancel()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
245d684d28 pthreadpool: split out pthreadpool_tevent_job from pthreadpool_tevent_job_state
This makes it much easier to handle orphaned jobs,
we either wait for the immediate tevent to trigger
or we just keep leaking the memory.

The next commits will improve this further.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
cdbad9041b pthreadpool: let pthreadpool_tevent_job_send() fail with an invalid pool
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
f19552e239 pthreadpool: split out a pthreadpool_stop() from pthreadpool_destroy()
This can be used in combination with pthreadpool_cancel_job() to
implement a multi step shutdown of the pool.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
5976841614 pthreadpool: don't process further jobs when shutting down
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
4e54543b54 pthreadpool: add pthreadpool_cancel_job()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
e4dfd3da94 pthreadpool: add pthreadpool_tevent_max_threads() and pthreadpool_tevent_queued_jobs()
These can be used to implement some kind of flow control in the caller.
E.g. unless pthreadpool_tevent_queued_jobs() is lower than
pthreadpool_tevent_max_threads() is good to prepare new jobs.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
505d298e81 pthreadpool: add pthreadpool_max_threads() and pthreadpool_queued_jobs() helpers
These can be used to implement some kind of flow control in the caller.
E.g. unless pthreadpool_queued_jobs() is lower than
pthreadpool_max_threads() is good to prepare new jobs.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
76474a6fad pthreadpool: expand test_create() to check unlimited, sync and one thread pool
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
f1dac71a86 pthreadpool: fix helgrind error in pthreadpool_free()
We need to pthread_mutex_lock/unlock the pool mutex
before we can destroy it.

The following test would trigger this.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
c9f54db109 pthreadpool: use talloc_zero() in tests_cmocka.c setup_pthreadpool_tevent()
This was found with valgrind.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
e45d33e92e pthreadpool: use strict sync processing only with max_threads=0
Otherwise it's an error if not at least one thread is possible.

This gives a much saner behaviour and doesn't end up with
unexpected sync processing.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
03830a3226 pthreadpool: consitently use unlock_res for pthread_mutex_unlock() in pthreadpool_add_job()
This makes further restructuring easier to implement and understand.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
65faef959d s3:messages: explicitly use max_thread=unlimited for pthreadpool_tevent_init() in messaging_dgm_init()
Currently 0 also means unlimited, but that will change soon,
to force no thread and strict sync processing.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
53a9f3cac6 pthreadpool: explicitly use max_thread=unlimited for pthreadpool_tevent_init() tests
Currently 0 also means unlimited, but that will change soon,
to force no thread and strict sync processing.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
5e723bc6f1 pthreadpool: use unsigned for num_idle, num_threads and max_threads
These can't get negative.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
19e4a08577 pthreadpool: correctly handle pthreadpool_tevent_register_ev() failures
It returns errno values instead of setting 'errno'.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Ralph Boehme
c310647e56 smbd: remove unused change_to_root_user() from brl_timeout_fn()
This is handled by using the root_ev_ctx in order to register
the timer event.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
d0b1f96f08 smbd: remove unused change_to_root_user() from smbd_sig_hup_handler()
This is handled by using the root_ev_ctx in order to register
the signal event.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
e37e41b3ca smbd: avoid explicit change_to_user() in defer_rename_done() already done by impersonation
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
1b804f7ae2 smbd: implement smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers
This makes sure we're doing the correct impersonation for async
requests, which is a requirement to start adding path based
async SMB_VFS calls.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
0dcaa0707b smbd: make use of smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers
For now they just add debugging, but that will change shortly.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
5285966e67 smbd: add simple noop smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers
As a start these are just wrappers arround
smbd_impersonate_debug_create(), without any real impersonation.
But this will change shortly.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:17 +02:00
Stefan Metzmacher
23319ef5a2 smbd: add smbd_impersonate_debug_create() helper
This will be used to implement no-op impersonation
for the create_conn_struct_as_root() case were we
don't really have other unrelated events in the loop
and only need a valid tevent wrapper context to avoid
double free on the raw event context on teardown.

This also adds useful debugging instead of being
a full no-op wrapper.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:17 +02:00