1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-25 06:04:04 +03:00

6439 Commits

Author SHA1 Message Date
Stefan Metzmacher
9d31bbf979 tevent: use talloc_zero_size() for the private state in tevent_context_wrapper_create()
This is watch tevent_req_create() uses and what callers of
tevent_context_wrapper_create() would therefore also expect.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-24 17:38:25 +02:00
Ralph Boehme
ba991da434 tevent: fix CID 1437974 dereference after null check
Probably a copy/paste error from the tevent_debug() statement a few
lines above as at this place we want to pass main_ev directly to
tevent_debug() anyway.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Jul 17 16:21:21 CEST 2018 on sn-devel-144
2018-07-17 16:21:21 +02:00
Ralph Boehme
3761d42e4f tevent: fix CID 1437976 dereference before null check
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-07-17 13:33:06 +02:00
Noel Power
7b170206b4 lib/ldb: Implement a bytes derived object for attributes py2/py3
ldb attributes are either bytes (py3) or str (py2)

Some places in the code do str(res[0]['attribute'][0])
which results in
   'result' (py2)
  b'result' (py3)

or more commonly the attribute is used to construct a string e.g.
   "blah=" + res[0]['attribute'][0] + ",foo,bar=...."

giving
   "blah=result,foo,bar=...." (py2)
and very unhelpfully
   "blah=b'result',foo,bar=...." (py3)

lots of code already constructs various strings for passing to other
api using the above. To avoid many excessive
    res[0]['attribute'][0].decode('utf8')

code like 'res[0]['attribute'][0]'

will now return LdbBytes (a new object subclassing 'bytes') in py3
instead of bytes. This object has a custom '__str__' method which
attempts to return a string decoded to uft8. In Py2 this will behave as
it did previously (this is the safer option at the moment)

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-07-13 01:12:24 +02:00
Stefan Metzmacher
3eee52b44d pthreadpool: allocate glue->tctx on glue as memory context.
This means it will go aways together with glue and thte event context.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Jul 12 17:18:01 CEST 2018 on sn-devel-144
2018-07-12 17:18:00 +02:00
Stefan Metzmacher
25756425aa pthreadpool: maintain a global list of orphaned pthreadpool_tevent_jobs
Instead of leaking the memory forever, we retry the cleanup,
if other pthreadpool_tevent_*() functions are used.

pthreadpool_tevent_cleanup_orphaned_jobs() could also be called
by external callers.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
fa070d9007 pthreadpool: make use of pthreadpool_stop() in pthreadpool_tevent_destructor()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
791c05144e pthreadpool: add pthreadpool_tevent_job_cancel()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
245d684d28 pthreadpool: split out pthreadpool_tevent_job from pthreadpool_tevent_job_state
This makes it much easier to handle orphaned jobs,
we either wait for the immediate tevent to trigger
or we just keep leaking the memory.

The next commits will improve this further.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
cdbad9041b pthreadpool: let pthreadpool_tevent_job_send() fail with an invalid pool
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
f19552e239 pthreadpool: split out a pthreadpool_stop() from pthreadpool_destroy()
This can be used in combination with pthreadpool_cancel_job() to
implement a multi step shutdown of the pool.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
5976841614 pthreadpool: don't process further jobs when shutting down
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
4e54543b54 pthreadpool: add pthreadpool_cancel_job()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
e4dfd3da94 pthreadpool: add pthreadpool_tevent_max_threads() and pthreadpool_tevent_queued_jobs()
These can be used to implement some kind of flow control in the caller.
E.g. unless pthreadpool_tevent_queued_jobs() is lower than
pthreadpool_tevent_max_threads() is good to prepare new jobs.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
505d298e81 pthreadpool: add pthreadpool_max_threads() and pthreadpool_queued_jobs() helpers
These can be used to implement some kind of flow control in the caller.
E.g. unless pthreadpool_queued_jobs() is lower than
pthreadpool_max_threads() is good to prepare new jobs.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
76474a6fad pthreadpool: expand test_create() to check unlimited, sync and one thread pool
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
f1dac71a86 pthreadpool: fix helgrind error in pthreadpool_free()
We need to pthread_mutex_lock/unlock the pool mutex
before we can destroy it.

The following test would trigger this.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
c9f54db109 pthreadpool: use talloc_zero() in tests_cmocka.c setup_pthreadpool_tevent()
This was found with valgrind.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
e45d33e92e pthreadpool: use strict sync processing only with max_threads=0
Otherwise it's an error if not at least one thread is possible.

This gives a much saner behaviour and doesn't end up with
unexpected sync processing.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
03830a3226 pthreadpool: consitently use unlock_res for pthread_mutex_unlock() in pthreadpool_add_job()
This makes further restructuring easier to implement and understand.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
53a9f3cac6 pthreadpool: explicitly use max_thread=unlimited for pthreadpool_tevent_init() tests
Currently 0 also means unlimited, but that will change soon,
to force no thread and strict sync processing.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
5e723bc6f1 pthreadpool: use unsigned for num_idle, num_threads and max_threads
These can't get negative.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
19e4a08577 pthreadpool: correctly handle pthreadpool_tevent_register_ev() failures
It returns errno values instead of setting 'errno'.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
3eecdbcc38 ldb: version 1.4.1
* add some missing return value checks
* Fix several mem leaks in ldb_index ldb_search ldb_tdb (bug#13475)
* ldb_tdb: Use mem_ctx and so avoid leak onto long-term memory
  on duplicated add. (bug#13471)
* ldb: Fix memory leak on module context (bug#13459)
* Refused build of Samba 4.8 with ldb 1.4 (bug #13519)
* Prevent similar issues in the future at configure time (bug #13519)

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Jul 12 07:43:22 CEST 2018 on sn-devel-144
2018-07-12 07:43:22 +02:00
Stefan Metzmacher
df858ec17e talloc: version 2.1.14
* Fix some typos in the comments
* Remove extra 0x prefix for the "%p" format specifiers,
  avoiding 0x0x0 strings in the output.
* make sure we link extra-python versions of libraries

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 04:32:09 +02:00
Stefan Metzmacher
b9efc5a628 tdb: version 1.3.16
* Fix build on AIX
* Python3 compatibility fixes
* Use tdb_wipe_all in "erase" command
* Harden allocating the tdb recovery area
* Make sure the hash size fits
* Harden tdb_check_used_record against overflow
* Harden tdb_rec_read
* Handle TDB_NEXT_LOCK_ERR in tdb_traverse_internal
* Fix build warnings

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 04:32:09 +02:00
Andrew Bartlett
52efa79653 ldb: Refuse to build Samba against a newer minor version of ldb
Samba is not compatible with new versions of ldb (except release versions)

Other users would not notice the breakages, but Samba makes many
more assuptions about the LDB internals than any other package.

(Specifically, LDB 1.2 and 1.4 broke builds against released
Samba versions)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13519

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-07-12 04:32:08 +02:00
Andrew Bartlett
1a559fd6a9 ldb: Ban ldb 1.4.x with Samba 4.8 and earlier
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13519

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-07-12 04:32:08 +02:00
Aaron Haslett
00002b8df9 dns: custom match rule for DNS records to be tombstoned
A custom match rule for records to be tombstoned by the scavenging process.
Needed because DNS records are a multi-valued attribute on name records, so
without a custom match rule we'd have entire zones into memory to search for
expired records.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 04:31:54 +02:00
Gary Lockyer
b841da04e2 dns: Reformat DNS with clang-format
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-12 04:31:52 +02:00
Gary Lockyer
d871e0c84c smb.conf: add dns_zone_scavenging
Add parameter dns_zone_scavenging to control dns zone scavenging.
Scavenging is disabled by default, as due to
https://bugzilla.samba.org/show_bug.cgi?id=12451 the ageing properties of
existing DNS entries are incorrect.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
2018-07-12 04:31:51 +02:00
Volker Lendecke
86a548c693 torture: Test tevent_req_profile
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-11 23:04:23 +02:00
Volker Lendecke
1e206553eb lib: Add tevent_req_profile helpers
Print and marshall/unmarshall tevent_req_profile structs

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-11 23:04:22 +02:00
Volker Lendecke
9847848cca lib: Multi-line a long line in wscript_build
Why? I'll add another file in a later commit

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-11 23:04:22 +02:00
Stefan Metzmacher
f4fe3f7766 tevent: version 0.9.37
* simplify "poll" and "poll_mt" backends
* make tevent_abort() reachable for backends
* add tevent_common_invoke_*_handler() functions
* add tevent_context_same_loop() function
* add tevent_context_wrapper_create() infrastructure
* add tevent_req_profile infrastructure

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:22 +02:00
Volker Lendecke
3dae5061b5 tevent: Add tevent_req_profile
This allows detailed reporting where a tevent_req spends its time

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-11 23:04:22 +02:00
Ralph Boehme
6189446abe tevent: add a test that frees wrapper_ev with pending events
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-11 23:04:22 +02:00
Ralph Boehme
4991c935c0 tevent: add a simple wrapper test
This checks that for all supported event types the before and after
handlers are called.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-11 23:04:21 +02:00
Stefan Metzmacher
ac9569b1a6 tevent: add tevent_context_wrapper_create() infrastructure
This allows to specify wrapper tevent_contexts, which adds the ability
to run functions before and after the event handler functions.

This can be used to implement impersonation hooks
or advanced debugging/profiling hooks.

We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:21 +02:00
Stefan Metzmacher
e239cbc1fd tevent: make use of #include "system/threads.h"
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:21 +02:00
Stefan Metzmacher
072e3b28ce tevent: split out tevent_common_invoke_fd_handler()
We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:21 +02:00
Stefan Metzmacher
6740718e0e tevent: split out tevent_common_invoke_immediate_handler()
We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:20 +02:00
Stefan Metzmacher
a85ee852ac tevent: split out tevent_common_invoke_timer_handler()
As side effect this avoids tricks with an extra
tevent_common_timed_deny_destructor().

We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:20 +02:00
Stefan Metzmacher
ddab21987d tevent: split out tevent_common_invoke_signal_handler()
As side effect this avoids tricks with tevent_se_exists_destructor() to
figure out if the event handler removed itself.

We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:20 +02:00
Stefan Metzmacher
d1b347c4ca tevent: simplify tevent_signal_destructor()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:20 +02:00
Stefan Metzmacher
601fd81ef2 tevent: use talloc_zero() in tevent_signal.c
This might not be strictly required, but it might
avoid problems in future...

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:20 +02:00
Stefan Metzmacher
8a947939e1 tevent: simplify tevent_cleanup_pending_signal_handlers()
Calling tevent_signal_destructor() does the same as se->event_ctx is already
NULL.

This also makes sure we correctly cleanup the SA_SIGINFO array.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:20 +02:00
Stefan Metzmacher
58fa08c805 tevent: add tevent_common_check_double_free() helper function
This will be used to generically support TALLOC_FREE() on
event which are currently running.

It aborts on every explicit talloc_free(), but ignores implicit
cleanup when the talloc parent is about to go.

We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:19 +02:00
Stefan Metzmacher
157df4da26 tevent: add tevent_threaded_schedule_immediate_destructor that just aborts
This will be active while the event is part of the ev->scheduled_immediates
list.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:19 +02:00
Stefan Metzmacher
0b91f6f0e4 tevent: use _tevent_schedule_immediate() to move events from a thread to the main_ev
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-11 23:04:19 +02:00