IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Finally: use the new dos_mode_at_send() in the directory enumeration
loop. This means that fetching the DOS attributes for directory entries
is done asynchronously with regard to the enumeration loop.
As the DOS attribute is typically read from an extended attribute in the
filesytem, this avoids sequentially blocking on IO. If the IO subsystem
is slow servicing these request, enabling async processing can result in
performance improvements.
A parametric option
smbd:async dosmode = true | false (default: false)
can be used to enable the new async processing.
Simulating slow IO with usleep(5000) in the synchronous and asynchronous
versions of SMB_VFS_GET_DOS_ATTRIBUTES(), the results of enumerating a
directory with 10,000 files are:
smbd:async dosmode = no:
$ time bin/smbclient -U slow%x //localhost/test -c "ls dir\*" > /dev/null
real 0m59.597s
user 0m0.024s
sys 0m0.012s
smbd:async dosmode = yes:
$ time bin/smbclient -U slow%x //localhost/test -c "ls dir\*" > /dev/null
real 0m0.698s
user 0m0.038s
sys 0m0.025s
Performance gains in real world workloads depends on whether the actual
IO requests can be merged and parallelized by the kernel. Without such
wins at the IO layer, the async processing may even be slower then the
sync processing due to the additional overhead.
The following parameters can be used to adapt async processing behaviour
for specific workloads and systems:
aio max threads = X (default: 100)
smbd:max async dosmode = Y (default: "aio max threads" * 2)
By default we have at most twice the number of async requests in flight
as threads provided by the underlying threadpool. This ensures a worker
thread that finishes a job can directly pick up a new one without going
to sleep.
It may be advisable to reduce the number of threads to avoid scheduling
overhead while also increasing "smbd:max async dosmode".
Note that we disable async processing for certain VFS modules in the VFS
connect function to avoid the overhead of triggering the sync fallback
in dos_mode_at_send(). This is done for VFS modules that implement the
sync SMB_VFS_GET_DOS_ATTRIBUTES(), but not the async version (gpfs), and
for VFS modules that don't share a real filesystem where fchdir() can be
used (ceph, gluster). It is disabled for catia, because we realized that
the catia name translation macros used on
fsps (CATIA_FETCH_FSP_[PRE|POST]_NEXT) have a bug (#13547).
We use threadpool = smb_vfs_ev_glue_tp_chdir_safe() and then
pthreadpool_tevent_max_threads(threadpool) to get the number of maximum
worker threads which matches the pool used by the low level
SMB_VFS_GETXATTRAT_[SEND|RECV] implementation in vfs_default.
This is a terrible abstraction leak that should be removed in the future
by maybe making it possible to ask a VFS function which threadpool it
uses, internally suporting chaining so VFS function FOO that internally
uses BAR can forward the question to BAR.
On a hyphotetical system that had a getxattrat(dirfd, path, ...)
syscall and at the same time doesn't support per-thread current working
directories (eg FreeBSD doesn't have the latter) but has support for
per-thread-credentials, pthreadpool_tevent_max_threads() on the
tp_chdir_safe threadpool returns 1.
So when hooking the hyphotetical getxattrat() into the async
SMB_VFS_GETXATTRAT_[SEND|RECV] implementation in an VFS module, the
implementation could use the tp_path_safe threadpool, but the SMB2
layer would use the wrong threadpool in the call to
pthreadpool_tevent_max_threads(), resulting in no parallelism.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This paves the way for adding async functions into the enumeration loop.
Best viewed with: git show -w
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
The next commit will add code that must be run if status is NT_STATUS_OK
or STATUS_MORE_ENTRIES.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This finally uses "get_dosmode" as passed in from the SMB2 layer, but
all callers still pass true, so no change in behaviour.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
We apply some post processing to the dosmode returned from the VFS
function. Move this to a seperate function which will be reused in the
next commit in the async dos_mode_send/recv post processing.
Best viewed with: git show --histogram
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
In preperation of adding an async version of get_ea_dos_attribute() that
will then call parse_dos_attribute_blob() too.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
exit_server_common() can be called also in smbd processes without a
smbXsrv_client structure, e.g. the parent or some background tasks.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Jul 26 01:29:38 CEST 2018 on sn-devel-144
This will be used to in order to pass down the
impersonation magic from the SMB layer through
the SMB_VFS layer.
This includes the following options:
smbd:force sync user path safe threadpool
smbd:force sync user chdir safe threadpool
smbd:force sync root path safe threadpool
smbd:force sync root chdir safe threadpool
They can be used in order to test the non linux code
path on linux, once we get code that makes full use
of the new infrastructure.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
This adds VFS helper functions and that work on a struct smb_vfs_ev_glue
object which bundles two event contexts and a few threadpools.
This will be used to streamline the use of impersonating wrappers
in the SMB_VFS.
Notice the verbose comments in source3/smbd/vfs.c.
This will allow us to introduce path based async operations
to the SMB_VFS layer.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
This just simulates a threadpool, but executes the
job functions inline (blocking) in the main thread.
This will be used to work arround some OS limitations,
e.g. if per thread credentials or per thread working directory
are not supported.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This should in future not be used directly, we'll provide
wrapper pools, which will provide impersonation for
path based async calls.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
For multi channel connections we should increment the disconnect count
also if we're not closing the last channel.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This means the pending requests are destroyed before tevent_context
impersonation wrapper are destroyed.
Otherwise, with a pending struct tevent_req that is a child of
client->xconn like this in exit_server_common():
conn[ipv4:127.0.0.11:40745] reason[NT_STATUS_CONNECTION_RESET] at ../source3/smbd/smb2_server.c:4015
full talloc report on 'struct smbXsrv_connection' (total 6085 bytes in 43 blocks)
struct smbd_smb2_request contains 648 bytes in 1 blocks (ref 0) 0x55ed41634740
struct smbd_smb2_request contains 3438 bytes in 32 blocks (ref 0) 0x55ed416331e0
struct tevent_req contains 1824 bytes in 20 blocks (ref 0) 0x55ed41635860
struct smb_filename contains 206 bytes in 2 blocks (ref 0) 0x55ed41635560
lease_v2_complex2.dat contains 22 bytes in 1 blocks (ref 0) 0x55ed4161b950
struct smbd_smb2_create_state contains 1386 bytes in 16 blocks (ref 0) 0x55ed41635a10
struct deferred_open_record contains 804 bytes in 12 blocks (ref 0) 0x55ed41633090
struct defer_open_state contains 764 bytes in 11 blocks (ref 0) 0x55ed41634f10
struct tevent_req contains 748 bytes in 10 blocks (ref 0) 0x55ed41636390
struct tevent_timer contains 96 bytes in 1 blocks (ref 0) 0x55ed41636ad0
struct dbwrap_watched_watch_state contains 420 bytes in 7 blocks (ref 0) 0x55ed41636540
struct tevent_req contains 296 bytes in 5 blocks (ref 0) 0x55ed41636700
struct messaging_filtered_read_state contains 64 bytes in 3 blocks (ref 0) 0x55ed416368b0
struct messaging_dgm_fde contains 8 bytes in 2 blocks (ref 0) 0x55ed41636950
reference to: struct messaging_dgm_fde_ev
we crash when freeing the xconn as the the tevent_req child has a
cleanup function that tries to access a (wrapped) tevent context that
was already destroyed via smb1srv_tcon_disconnect_all() for SMB1 or
smbXsrv_session_logoff_all() for SMB2 a few lines above.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This is handled by using the root_ev_ctx in order to register
the timer event.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This is handled by using the root_ev_ctx in order to register
the signal event.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This makes sure we're doing the correct impersonation for async
requests, which is a requirement to start adding path based
async SMB_VFS calls.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
For now they just add debugging, but that will change shortly.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
As a start these are just wrappers arround
smbd_impersonate_debug_create(), without any real impersonation.
But this will change shortly.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This will be used to implement no-op impersonation
for the create_conn_struct_as_root() case were we
don't really have other unrelated events in the loop
and only need a valid tevent wrapper context to avoid
double free on the raw event context on teardown.
This also adds useful debugging instead of being
a full no-op wrapper.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
If session info was passed down from upstream, then try to use it to get
security token, other then creating token every time.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13521
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
When a session_info passed down to here, the unix_info could be NULL.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13521
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Add session_info arg as optional and pass it down to get_conn_tos.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13521
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Add session_info arg, so caller can pass it in to reuse authentication info
later. This will improve performance a lot while doing ntacl operations
on large amount of files, e.g.: sysvolreset.
Modification for upstream caller will come in following patches.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13521
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This reverts commit c53646bccd.
As mentioned by Andrew, we shouldn't break environments where
"force group" has been configured to use substituted variables.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Jul 10 00:12:19 CEST 2018 on sn-devel-144
This reverts commit c58194e3d2.
As mentioned by Andrew, we shouldn't break environments where
"force user" has been configured to use substituted variables.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
On a cluster filesystem the device numbers may differ on the cluster
nodes. We already verify the file_id in vfs_default_durable_reconnect(),
so we can safely remove the dev/inode checks.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13318
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
As the comment above the if condition says:
/*
* If we are in case insentive mode, we don't need to
* store names that need no translation - else, it
* would be a waste.
*/
Ie if stat_cache_add() is called as
stat_cache_add("foo/bar", "foo/bar", false)
There's no need to cache the path, as a simple stat() on the client
supplied name (full_orig_name) matches the name used in the
filesystem (passed to stat_cache_add() as translated_path).
So fix the if condition to match the comment.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
set_conn_force_user_group() and change_to_user_internal() leak onto
the callers' talloc stackframe. Drop the unnecessary heap allocations.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The majority of these lp_servicename(talloc_tos(), ...) callers leak
onto the talloc stackframe. Drop the unnecessary heap allocations.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The majority of these lp_servicename(talloc_tos(), ...) callers leak
onto the talloc stackframe. Drop the unnecessary heap allocations.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Add create_file function to smbd API.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Extract initialization code from set_nt_acl_conn for reuse.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
