1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-14 19:24:43 +03:00

107414 Commits

Author SHA1 Message Date
Ralph Boehme
26018947f9 s4/torture: add a test for compound SMB2 FIND requests
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
9c95eca0f4 s3/smbd: add "smbd:find async delay usec" to SMB2 FIND
This is just a hack for selftest that will be used in subsequent commits
for torturing compound find requests.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
d99bd1c48b s3/smbd: make write time fetching async
Finally use the new async dbwrap_parse_record_send/recv() functions
respectively the fetch_share_mode_send/recv wrappers for fetching the
write time from locking.tdb.

Previously for a directory with n files we would sit idle in the
directory enumeration loop fo n * m seconds waiting for responses from
ctdb, where m is the response time in seconds for a dbwrap request via
ctbd.

This is known to kill performance and we even have a parameter
"smbd:search ask sharemode" that can be used to disable fetching the
write time from locking.tdb.

Using fetch_write_time_send() works this way: in the directory
enumeration loop that calls smbd_dirptr_lanman2_entry() to marshall the
directory entries we

1. call fetch_write_time_send() after calling smbd_dirptr_lanman2_entry
   passing a pointer to the current position in the marshall buffer.

2. If fetch_write_time_send() has set the out parameter "stop", we exit
   the enumeration loop. This is necessary because we only send dbwrap
   requests but don't consume the results. This has the potential to
   deadlock so we must stop sending requests as soon as our ctdb send
   queue is full.

3. In the fetch_write_time_done() callback, if the recv function got a
   locking.tdb record, we push the write time into the marshall buffer
   at the offet saved in the request state.

This new feature is still off by default as it doesn't
give any improvement in the non-clustered usecase.
"smbd:async search ask sharemode" can be used to activate it,
which makes only sense with "clustering = yes" (execept for testing).

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
682f1b7d65 s3/smbd: enable processing SMB2 requests async internally
The idea is to allow the implementation of an SMB2 request to tell the
main SMB2 processing engine that it wants to handle a requests
asynchronously internally.

This has two use cases:

- it allows (internal) async processing of compound requests that would
  otherwise be rejected by the SMB2 processing engine

- it preserves sync semantics at the SMB layer, some clients might not
  expect arbitrary SMB2 requests going async

Not used for now, will be used in laters commit for async SMB2 FIND
requests.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
d1f8d3e18b s3/smbd: ask_sharemode is not needed for info_level SMB_FIND_FILE_NAMES_INFO
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
f589919d67 s3/smbd: add file_id return arg to smbd_dirptr_lanman2_entry
Not used for now, needed for async write_time updates in
smbd_smb2_query_directory_send().

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
06891c11d2 s3/locking: add fetch_share_mode_send/recv
The boolean out parameter "queued" tells the caller whether the
async request is blocked in a full send queue:

false := request is dispatched
true  := send queue is full, request waiting to be dispatched

This is useful in a clustered Samba environment where the async dbwrap
request is sent over a socket to the local ctdbd.

If the send queue is full and the caller was issuing multiple async
dbwrap requests in a loop, the caller knows it's probably time to stop
sending requests for now and try again later.

This will be used in subsequent commits in
smbd_smb2_query_directory_send() when implementing async write time
updates. Directories may contain umpteen files so we send many requests
to ctdb without going through tevent and reading the responses which
has the potential to deadlock.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
cc56fcd0e5 dbwrap_watch: add parse_record_send/recv wrappers
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
b3f8e9a62c dbwrap: add dbwrap_parse_record_send/recv
The req_state parameter tells the caller whether the async request is
blocked in a full send queue:

req_state >= DBWRAP_REQ_DISPATCHED := request is dispatched
req_state < DBWRAP_REQ_DISPATCHED := send queue is full

This is useful in a clustered Samba environment where the async dbwrap
request is sent over a socket to the local ctdbd.

If the send queue is full and the caller was issuing multiple async
dbwrap requests in a loop, the caller knows it's probably time to stop
sending requests for now and try again later.

This will be used in subsequent commits in
smbd_smb2_query_directory_send() when implementing async write time
updates. Directories may contain umpteen files so we send many requests
to ctdb without going through tevent and reading the responses which has
the potential to deadlock.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
2813b53e1a dbwrap_ctdb: implement parse_record_send()/recv()
This mainly works like the sync version, but calls ctdbd_parse_send/recv
instead.

We use one global ctdb connection that is used exclusively for async
requests.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
6a1734ebfd dbwrap_ctdb: factor out a db_ctdb_try_parse_local_record() function
Pair-programmed-with: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
0a0c50d361 ctdb_conn: add ctdbd_parse_send/recv
Implement the ctdb packet layer for async parse send/recv with tevent.

ctdbd_setup_fde() will is used to create an fde from the
connection fd and will be called from dbwrap_ctdb.

ctdbd_parse_send() and ctdbd_parse_recv() will be used by dbwrap_ctdb
for async packet sending and receiving.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
4449844096 dbwrap: add parse_record_send/recv to struct db_context
The implementation comes next.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
27acbb26ee dbwrap: add enum dbwrap_req_state
This will be used by async dwrap_parse_send() as an out argument, giving
the caller an indication about the state of the request.

This is can be useful for the caller if it is a sync function and sends
multiple async dbwrap requests. As it's a sync function it won't return
to the main tevent event loop and so the async dbwrap recv function are
not called.

As a result the function may deadlock: our receive queue may already be
full with results from a peer, the peer might be blocked in his send
queue (because we're not receiving), the peer therefor doesn't read from
his receive queue so our send queue will block as well.

To inform the caller of this situation "send queue full" we return this
state information to the caller of the dbwrap send function.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:16 +02:00
Ralph Boehme
1fcb66efe0 lib/util: add and use iov_concat
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-18 22:54:15 +02:00
Jeremy Allison
1e8e048bf0 lib: param: Remove lpcfg_register_defaults_hook().
Completely unused functionality. Gets rid of another
talloc_autofree_context(). Updated WHATSNEW to make
this clear.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
2017-04-18 22:54:15 +02:00
Volker Lendecke
4fe2b24b34 smbldap: Move ldapsam_privates to pdb_ldap.h
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Apr 18 22:52:03 CEST 2017 on sn-devel-144
2017-04-18 22:52:03 +02:00
Volker Lendecke
91e9ff9975 smbldap: pdb_ipa is gone
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-18 18:54:13 +02:00
Jan Engelhardt
e5f2dfacae build: correct package dependencies
The wscript_build files convey what header files belong to which
logical package. For example,

    # lib/util/wscript_build:
    bld.SAMBA_LIBRARY('samba-util',
                      public_headers='... data_blob.h ...'

    # auth/credentials/wscript_build:
    bld.SAMBA_LIBRARY('samba-credentials',
                      public_headers='credentials.h',

Now, credentials.h #includes <util/data_blob.h> and therefore,
samba-credentials.pc must have a Requires: samba-util.

Similarly for other parts.

Signed-off-by: Jan Engelhardt <jengelh@inai.de>
Reviewed-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-18 18:54:13 +02:00
Jeremy Allison
76b351e907 s3:smbd: Fix incorrect use of sys_getgroups()
Second arg must be NULL when first arg is 0 (it is in all other places).

Bug report and patch from Hanno Böck <hanno@hboeck.de>

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12747

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Apr 18 15:43:02 CEST 2017 on sn-devel-144
2017-04-18 15:43:02 +02:00
Jeremy Allison
60af864f75 s3:lib: Fix incorrect logic in sys_broken_getgroups()
If setlen == 0 then the second argument must be ignored.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12747

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-18 11:47:17 +02:00
Jeremy Allison
600f8787e3 lib: debug: Avoid negative array access.
Report and patch from Hanno Böck <hanno@hboeck.de>.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12746

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-18 11:47:17 +02:00
Jeremy Allison
bf8f7a36bf lib:charset: Remove use of talloc_autofree_context() for global_iconv_handle
All other callers use NULL here anyway, so there's no
need to use a special context for get_iconv_handle().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-18 11:47:17 +02:00
Jeremy Allison
35b23711e8 lib:charset: Make global_iconv_handle private
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-18 11:47:17 +02:00
Jeremy Allison
2a4d07b999 lib: param: Remove the last external use of global_iconv_handle by calling the utility function reinit_iconv_handle().
Add an error check.

This *looks* like a logic change, but it is not.

The only change is the addition of the error return check.

The reason is that the changed function, reload_charcnv(),
is the *only* function that sets lp_ctx->iconv_handle. And
it does so just before setting global_iconv_handle = lp_ctx->iconv_handle.

Calling the utility function reinit_iconv_handle()
instead merely sets global_iconv_handle first, then
assigns it (as the return) to lp_ctx->iconv_handle.

So all this is doing is reversing the order of
setting global_iconv_handle and lp_ctx->iconv_handle
to the same thing.

Even the removal of the lines:

-       struct smb_iconv_handle *old_ic = lp_ctx->iconv_handle
-       if (old_ic == NULL) {
-               old_ic = global_iconv_handle;

has no effect, as remember that lp_ctx->iconv_handle
is only ever set to the same value as global_iconv_handle,
and once this function has been run once, lp_ctx->iconv_handle != NULL.

This allows us finally to make global_iconv_handle private
to the C source file that defines it.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-18 11:47:17 +02:00
Jeremy Allison
766e9ff05e lib: param: Use utility functions to get rid of two more uses of global_iconv_handle.
Add error return checking.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-18 11:47:17 +02:00
Jeremy Allison
6086b0e5a2 s3:param: Use new utility function to hide use of global_iconv_handle
Add error return check.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-18 11:47:17 +02:00
Jeremy Allison
b4e4042b37 s3:lib:charcnv: Remove use of global global_iconv_handle
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-18 11:47:17 +02:00
Jeremy Allison
c28e2c937a lib:charset: Add utility functions reinit_iconv_handle() and free_iconv_handle(void)
Not yet used. Will enable us to make global_iconv_handle private.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-18 11:47:17 +02:00
Jeremy Allison
3afbdb7a0e lib: Remove smb_iconv_handle_reinit_lp()
It's merely a wrapper for smb_iconv_handle_reinit(),
only used in one place and smb_iconv_handle_reinit()
is already called from lib/param/loadparm.c.

Removing this will make it easier to make global_iconv_handle
private state to lib/util/charset/codepoints.c later.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-18 11:47:17 +02:00
Uri Simchoni
05d83ccf7a vfs_acl_xattr: avoid needlessly supplying a large buffer to getxattr()
When obtaining the security descriptor via getxattr(), first try
optimistically to supply a buffer of 4K, and if that turns out
to be too small, determine the correct buffer size.

The previous behavior of falling back to a 64K buffer encountered
problem with Linux prior to version 3.6, due to pyisical memory
fragmentation. With those kernels, as long as the buffer is 8K or
smaller, getting the xattr is much less prone to failure due to
memory fragmentation.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12737

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Apr 18 04:41:16 CEST 2017 on sn-devel-144
2017-04-18 04:41:16 +02:00
Uri Simchoni
7b775abd92 vfs_acl_xattr: factor out fetching of an extended attribute
Pure refactoring - add a function that fetches an extended attribute
based on either the file descriptor or the file name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12737

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-04-18 00:43:20 +02:00
Uri Simchoni
4dfa2d6a09 vfs_xattr_tdb: handle case of zero size.
With getxattr(), passing a zero buffer size is a
way of obtaining actual xattr size.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12737

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-04-18 00:43:20 +02:00
Uri Simchoni
5017dfeef2 selftest: test fetching a large ACL from vfs_acl_xattr
Add a test that fetches an ACL whose size is larger than 4K.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12737

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-04-18 00:43:20 +02:00
Andreas Schneider
0e0f56485e docs: Update idmap_rid manpage
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-04-18 00:43:20 +02:00
Jeremy Allison
ab597b9a47 s4: server: Use state as the talloc context for open_schannel_session_store.
Ensure it's freed on all error paths.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Apr 17 23:10:06 CEST 2017 on sn-devel-144
2017-04-17 23:10:06 +02:00
Jeremy Allison
f24ecb035b s4: server: Remove use of talloc_autofree_context as the parent of event_ctx.
Use state->event_ctx as the parent of the initial imessaging context.

Now we control all exit paths, we can call TALLOC_FREE(state)
on all of them.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-04-17 19:13:07 +02:00
Jeremy Allison
fc8f858c8c s4: messaging: When talloc_free()'ing an event context, only remove msg_dgm_ref's that point to *that* context.
Defensive programming change. Not strictly needed to prevent
any crash/error.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-04-17 19:13:07 +02:00
Jeremy Allison
5fe996bc1f s4: server: Add a tevent signal handler for SIGTERM.
Simplify by removing global state we don't need now
we're called by tevent (and in the short window where
we're installed by CatchSignal but before we install
the tevent handler we don't need the complex global
state handling as we have no forked children).

We now have access to struct server_state on all
exit paths - next commits will stop using talloc autofree context.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-04-17 19:13:07 +02:00
Jeremy Allison
139793ce06 s4: server: Add error return checks for tevent_add_fde, tevent_add_timer.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-04-17 19:13:07 +02:00
Jeremy Allison
1ae7028e47 s4: server: Plumb server_state through the irpc messaging for samba_terminate().
Use it in the message print to avoid a "unused variable" compile error.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-04-17 19:13:07 +02:00
Jeremy Allison
d721b8d86b s4: server: Use server_state as a parameter to max_runtime_handler, not just name.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-04-17 19:13:07 +02:00
Jeremy Allison
cc14187c5c s4: server: Use server_state as a parameter to stdin handler, not just name.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-04-17 19:13:07 +02:00
Jeremy Allison
a7d0e54969 s4: server: Create a server 'state' struct.
No logic changes, will be used to move allocated
pointers off the talloc autofree context in a later commit.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-04-17 19:13:07 +02:00
Jeremy Allison
33f599737d s4: server. Whitespace and 80+ column cleanup.
No logic changes.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-04-17 19:13:06 +02:00
Jeremy Allison
61157388e7 s4: messaging. Minor cleanup. Check for error returns on imessaging_register calls.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2017-04-17 19:13:06 +02:00
Noel Power
5c04521b8e s3:tests: Add test for illegal value detection for 'name resolve order'
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12739

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Apr 13 15:20:04 CEST 2017 on sn-devel-144
2017-04-13 15:20:04 +02:00
Noel Power
8050db2303 param: Check for valid values of 'name resolve order' option
This variable is populated by a list of values where each value should
be a known option. This patch ensures that illegal values are detected.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12739

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
2017-04-13 11:26:28 +02:00
Garming Sam
63c6209f5c updaterefs: Do not open transaction even when unnecessary
This can be called during GetNCChanges (a generally read-only call), it
is not wise to be blocking the database for no reason.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Thu Apr 13 11:25:06 CEST 2017 on sn-devel-144
2017-04-13 11:25:06 +02:00
Garming Sam
d055b20298 drepl_server: Allow refresh of partitions on UpdateRef
When we call UpdateRef, the push replication will not begin until the
drepl_server has done its periodic refresh. If UpdateRefs is called, we
should just send an IRPC message to call the refresh.

NOTE: This has the same dependencies and issues as repl_secrets in
auth_sam.c in terms of IRPC implementation.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-04-13 07:29:18 +02:00