1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-27 03:21:53 +03:00
Commit Graph

122123 Commits

Author SHA1 Message Date
Volker Lendecke
4f2117d687 lib: Allow additional dbufs in g_lock_store()
Preparation for g_lock_writev_data()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
3d0cfb82ec lib: Add g_lock_seqnum()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
c38f37147d lib: Add g_lock_wake_watchers()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
90dcfa4e59 torture: Test g_lock_watch_data_send()/recv()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
46e8e0cd7e lib: Add g_lock_watch_data_send/recv()
Same concept as dbwrap_watched_watch_send/recv: Get informed if the
underlying data of a record changes. This utilizes the watched
database that g_lock is based upon anyway. To avoid spurious wakeups
by pure g_lock operations this patch adds a sequence number for the
data that is stored in the g_lock data field.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
66d62ed797 lib: Allow DBWRAP_LOCK_ORDER_NONE in db_open()
locking.tdb will not have a LOCK_ORDER anymore, this will be done by
the code in g_lock.c. We need to allow opening a database with dbwrap
without having a lock order.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
5ae58cd0db dbwrap: Make dbwrap_lock_order_[un]lock() public
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
38a80a3005 dbwrap: Simplify dbwrap_lock_order_[un]lock()
Directly pass the database name and lock order to the core functions,
avoid passing struct db_context.

In the next steps these functions will become public: locking.tdb will
be based on g_lock.c to avoid holding a tdb-level locking.tdb mutex
while doing complex file system operations like unlink() which can
take ages on FAT for example.

This means that g_lock.c will participate in the dbwrap lock order
protection and needs access to dbwrap_lock_order_[un]lock() without
providing a direct db_context.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
01db877c77 srvsvc: Move brl_get_locks() out of enum_file_fn()
With share_infos.tdb this is a locking order violation:
share_infos.tdb is level 4, brlock.tdb is level 2. Avoid this by first
walking the share_infos.tdb and then fetching all the brlock entries.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu May 14 22:06:32 UTC 2020 on sn-devel-184
2020-05-14 22:06:32 +00:00
Volker Lendecke
8e4583f730 test: Show that netfileenum is broken
Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-14 20:29:38 +00:00
Volker Lendecke
1d40cc01c2 rpcclient: Make netfileenum cmd print the path names
Needed for the next commit testing netfileenum

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-14 20:29:38 +00:00
Volker Lendecke
8c080f28c3 rpcclient: Use struct initializers in cmd_srvsvc_net_file_enum()
Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-14 20:29:38 +00:00
Volker Lendecke
46ab1d478d srvsvc: Collect file ids in enum_file_fn()
Will be used a few patches down

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-14 20:29:38 +00:00
Volker Lendecke
bda0b3875d srvsvc: Use a struct initializer in net_enum_files()
Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-14 20:29:38 +00:00
Volker Lendecke
96d68bb9f2 srvsvc: Directly use "ctr3->count" instead of "i"
To me this was not very transparent, and now that we have "ctr3" a
single indirect looks okay

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-14 20:29:38 +00:00
Volker Lendecke
ff80f68c30 srvsvc: Use a struct assignment in enum_file_fn()
Looks nicer than 5 complex array references...

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-14 20:29:38 +00:00
Volker Lendecke
a9397f8788 srvsvc: Introduce ctx3 helper var in enum_file_fn()
Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-14 20:29:38 +00:00
Ralph Boehme
7215669d29 vfs: remove root_dir_fid arg from SMB_VFS_CREATE_FILE()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu May 14 19:43:27 UTC 2020 on sn-devel-184
2020-05-14 19:43:27 +00:00
Ralph Boehme
54883d4b3a smbd: move processing of root_dir_fid outside of SMB_VFS_CREATE_FILE()
The only two callers that pass a root_dir_fid != 0 are reply_ntcreate_and_X()
and call_nt_transact_create(). By moving root_dir_fid processing outside of
SMB_VFS_CREATE_FILE() we can avoid rewriting get_relative_fid_filename() to work
with *AT semantics and dirfsps.

Bonus points: the previous code in both functions passed fname to
filename_convert() which can't really have worked as it is relative to
root_dir_fid so filename_convert() will not be able to stat() the relative path
and it's components.

The only test I can find that uses a root dir_fid is raw.samba3rootdirfid and
that uses a handle on the share root which will work.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14380

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-14 18:07:39 +00:00
Ralph Boehme
f0df11ce9d s4/torture: add a *real* root_dir_fid test
raw.samba3rootdirfid tests with the share root directory as root_dir_fid handle,
that doesn't cover the case where the relative name has more then one path
component. It only works because in unix_convert() we run into the creating file
optimasation.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14380

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-14 18:07:39 +00:00
Andreas Schneider
ab70153c20 testprogs: Add 'net ads join' test for fips
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Thu May 14 14:27:26 UTC 2020 on sn-devel-184
2020-05-14 14:27:26 +00:00
awalker
84c7eb4787 lib:param: Fix memory leak in lpcfg_load_internal()
Signed-off-by: awalker <awalker@ixsystems.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed May 13 17:34:43 UTC 2020 on sn-devel-184
2020-05-13 17:34:43 +00:00
Ralph Boehme
6e419dda71 ctdb: increase TasksMax limit, the systemd default is just 512
In 2015 systemd introduced a TasksMax which limits the number of processes in a
unit:

https://lists.freedesktop.org/archives/systemd-devel/2015-November/035006.html

The default of 512 may be too low in certain situations leading to vfork()
failing with errno=EAGAIN when trying to spawn lock-helper processes.

With the default for LockProcessesPerDB being 200 the increased TasksMax limit
should cover the problematic scenario.

Additional background: the failing vfork()s have been seen on production
clusters and were tracked down to being logged in the context of ctdb calling
tdb_repack().

Links:

9ded9cd14c
https://www.suse.com/support/kb/doc/?id=000015901
https://success.docker.com/article/how-to-reserve-resource-temporarily-unavailable-errors-due-to-tasksmax-setting
https://www.percona.com/blog/2019/01/02/tasksmax-another-setting-that-can-cause-mysql-error-messages/

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed May 13 13:30:12 UTC 2020 on sn-devel-184
2020-05-13 13:30:12 +00:00
Jeremy Allison
393da520e4 s3: libsmbclient: Finish unifing bad iconv behavior across CORE NT1 SMB2 protocols.
On bad name conversion, exit the directory listing with an error, but leave the
connection intact. We were already checking for finfo->name == NULL here,
but were ignoring it and not reporting an error.

Remove the knownfail.d/bad_iconv file as we now
behave the same across CORE/NT1/SMB2.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14374

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue May 12 21:32:44 UTC 2020 on sn-devel-184
2020-05-12 21:32:44 +00:00
Jeremy Allison
753115a8d1 s3: libsmb: In SMB2 return NT_STATUS_INVALID_NETWORK_RESPONSE if name conversion ended up with a NULL filename.
Can happen if namelen == 0.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14374

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-05-12 19:53:46 +00:00
Jeremy Allison
b10de0bb64 s3: libsmb: In SMB1 old protocol - return NT_STATUS_INVALID_NETWORK_RESPONSE if name conversion ended up with a NULL filename.
Can happen if namelen == 0.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14374

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-05-12 19:53:46 +00:00
Jeremy Allison
e016671d34 s3: selftest: Add test_smbclient_iconv.sh to check client behavior on bad name conversion.
SMB2 and NT1 fail this, CORE already returns NT_STATUS_INVALID_NETWORK_RESPONSE
on bad conversion.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14374

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-05-12 19:53:46 +00:00
Jeremy Allison
a9651d6bc2 s3: selftest: Add share definition [bad_iconv] in fileserver.
Creates a utf8 valid filename within that is invalid in CP850.
Useful to test smbclient list directory character set conversions.

https://bugzilla.samba.org/show_bug.cgi?id=14374

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-05-12 19:53:46 +00:00
Stefan Metzmacher
0f01b10679 vfs_io_uring: retry after a short writes in vfs_io_uring_pwrite_completion()
We need to be prepared for short writes from the kernel depending on
the state of the page cache.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:46 +00:00
Stefan Metzmacher
42e77c4cf2 vfs_io_uring: retry after a short read in vfs_io_uring_pread_completion()
We need to be prepared for short reads from the kernel depending on
the state of the page cache. Windows and Mac clients don't
expect short reads for files, so we need to retry ourself.

For the future we may be able to play with some io_uring flags
in order to avoid the retries in userspace, but for now we just fix
the data corruption bug...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:45 +00:00
Stefan Metzmacher
c57a731c4c vfs_io_uring: protect vfs_io_uring_fsync_completion() against invalid results
We should never get back a value > 0.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:45 +00:00
Stefan Metzmacher
283f968722 vfs_io_uring: protect vfs_io_uring_pwrite_completion() against invalid results
We should never get more acked than we asked for.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:45 +00:00
Stefan Metzmacher
f085dbf8b2 vfs_io_uring: protect vfs_io_uring_pread_completion() against invalid results
We should never get back more than we asked for.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:45 +00:00
Stefan Metzmacher
2f6abb00b0 vfs_io_uring: split out a vfs_io_uring_pwrite_submit() function
This can be reused when we add handling for short writes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:45 +00:00
Stefan Metzmacher
9de4f8be1d vfs_io_uring: split out a vfs_io_uring_pread_submit() function
This can be reused when we add handling for short reads.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:45 +00:00
Stefan Metzmacher
ab89b8e753 vfs_io_uring: split out a vfs_io_uring_request_submit() function
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:45 +00:00
Stefan Metzmacher
f96f45c9ba vfs_io_uring: avoid stack recursion of vfs_io_uring_queue_run()
Instead we remember if recursion was triggered and jump to
the start of the function again from the end.

This should make it safe to be called from the completion_fn().

This is hideously complex stuff, so document the hell
out of it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:45 +00:00
Stefan Metzmacher
388bc2e6e4 vfs_io_uring: make use of sys_valid_io_range() in vfs_io_uring_pwrite_send()
This makes the follow up commits easier as we don't have to care
about overflows.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:45 +00:00
Stefan Metzmacher
5005ae3fb2 vfs_io_uring: make use of sys_valid_io_range() in vfs_io_uring_pread_send()
This makes the follow up commits easier as we don't have to care
about overflows.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:45 +00:00
Stefan Metzmacher
40be2232a4 vfs_io_uring: move error handling out of vfs_io_uring_fsync_recv()
We should do that as early as possible and that's in
vfs_io_uring_fsync_completion().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:45 +00:00
Stefan Metzmacher
a51969b8c7 vfs_io_uring: move error handling out of vfs_io_uring_pwrite_recv()
We should do that as early as possible and that's in
vfs_io_uring_pwrite_completion().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:45 +00:00
Stefan Metzmacher
a1487067a6 vfs_io_uring: move error handling out of vfs_io_uring_pread_recv()
We should do that as early as possible and that's in
vfs_io_uring_pread_completion().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:44 +00:00
Stefan Metzmacher
456533c9cf vfs_io_uring: introduce vfs_io_uring_request->completion_fn()
We'll need to add more logic than a simple _tevent_req_done()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:44 +00:00
Stefan Metzmacher
f78e98e022 vfs_io_uring: replace vfs_io_uring_request->state with _tevent_req_data()
We don't need a direct pointer to the state...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:44 +00:00
Stefan Metzmacher
fadc7043a7 vfs_io_uring: fix the prefix for parametric options from 'vfs_io_uring' to 'io_uring'
This is what the manpage describes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-12 19:53:44 +00:00
Jeremy Allison
801c06f4c9 s3: VFS: default. Change pwrite() -> sys_pwrite_full() in SMB_VFS_PWRITE_SEND() to protect against short writes.
Note that as sys_pwrite_full() deals with the EINTR case
we can remove the do {} while loop here.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-12 19:53:44 +00:00
Jeremy Allison
bf2e546be3 s3: VFS: default. Change pread() -> sys_pread_full() in SMB_VFS_PREAD_SEND() to protect against short reads.
Note that as sys_pread_full() deals with the EINTR case
we can remove the do {} while loop here.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-12 19:53:44 +00:00
Jeremy Allison
ca8c3619f6 s3: VFS: default. Change sys_pwrite() -> sys_pwrite_full() in SMB_VFS_PWRITE() to protect against short writes.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-12 19:53:44 +00:00
Jeremy Allison
7daa79908b s3: VFS: default. Change sys_pread() -> sys_pread_full() in SMB_VFS_PREAD() to protect against short reads.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-12 19:53:44 +00:00
Jeremy Allison
20ee8b03bb s3: VFS: aio_fork: Change sys_pwrite() -> sys_pwrite_full() to protect against short writes.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2020-05-12 19:53:44 +00:00