1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-11 05:18:09 +03:00
Commit Graph

2457 Commits

Author SHA1 Message Date
Ralph Boehme
96320eccc9 vfs_fruit: prepare struct fio for fake-fd and on-demand opening
Not used for now, that comes in the subsequent commits.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:21 +01:00
Ralph Boehme
1e055a7954 vfs_fruit: add fio->created
fio->created tracks whether a create created a stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:21 +01:00
Ralph Boehme
a26032c3f6 vfs_fruit: remove resource fork special casing
Directly unlinking a file with open handles is not good, don't do it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:21 +01:00
Ralph Boehme
993c7c4e71 vfs_fruit: add some debugging of dev/ino
Aids in debugging dev/ino mismatch failures in open_file_ntcreate.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:21 +01:00
Ralph Boehme
ebfcf75e99 vfs_fruit: filter empty streams
First step in achieving macOS compliant behaviour wrt to empty streams:
- hide empty streams in streaminfo
- prevent opens of empty streams

This means that we may carry 0-byte sized streams in our streams
backend, but this shouldn't really hurt.

The previous attempt of deleting the streams when an SMB setinfo eof to
0 request came in, turned out be a road into desaster.

We could set delete-on-close on the stream, but that means we'd have to
check for it for every write on a stream and checking the
delete-on-close bits requires fetching the locking.tdb record, so this
is expensive and I'd like to avoid that overhead.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:20 +01:00
Ralph Boehme
046456fca6 vfs_fruit: use check on global_fruit_config.nego_aapl for macOS specific behaviour
Ensure any non MS compliant protocol behaviour targetted at supporting
macOS clients are only effective if the client negotiated AAPL.

Currently this only guards the resource fork which only macOS client are
going to use, but subsequent commits add more this at this place.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:20 +01:00
Ralph Boehme
287082d739 vfs_fruit: don't unlink 0-byte size truncated streams
This caused all sort of havoc with subsequent SMB request that acted on
the handle of the then deleted backend storage (file or blob, depending
on the used streams module).

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:20 +01:00
Ralph Boehme
055ca448f1 vfs_fruit: update handling of read-only creation of resource fork
macOS SMB server versions supports this since 10.12, so we adapt our
behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:19 +01:00
Ralph Boehme
9f58997d7a vfs_streams_xattr: fix open implementation
Since a long time the modules's open function happily returned success
when opening a non existent stream without O_CREAT.

This change fixes it to return -1 and errno=ENOATTR if

o get_ea_value() returns NT_STATUS_NOT_FOUND (eg mapped from
  getxattr() = -1, errno=ENOATTR) and

o flags doesn't contain O_CREAT

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:18 +01:00
Ralph Boehme
3649f1a41a vfs_fruit: optionally delete AppleDouble files without Resourcefork data
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:18 +01:00
Ralph Boehme
e00e61345c vfs_fruit: add option "delete_empty_adfiles"
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:17 +01:00
Ralph Boehme
7be979f958 vfs_fruit: detect empty resource forks in ad_convert()
For some reason the macOS client often writes AppleDouble files with a
non-zero sized resource fork, but the resource fork data is just
boilerplate data with the following string close to the start

  This resource fork intentionally left blank

A dump with apple_dump looks like this:

Entry ID   : 00000002 : Resource Fork
Offset     : 00000052 : 82
Length     : 0000011E : 286

-RAW DUMP--:  0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F : (ASCII)
00000000   : 00 00 01 00 00 00 01 00 00 00 00 00 00 00 00 1E : ................
00000010   : 54 68 69 73 20 72 65 73 6F 75 72 63 65 20 66 6F : This resource fo
00000020   : 72 6B 20 69 6E 74 65 6E 74 69 6F 6E 61 6C 6C 79 : rk intentionally
00000030   : 20 6C 65 66 74 20 62 6C 61 6E 6B 20 20 20 00 00 :  left blank   ..
00000040   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000050   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000060   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000070   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000080   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000090   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000A0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000B0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000C0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000D0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000E0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000F0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000100   : 00 00 01 00 00 00 01 00 00 00 00 00 00 00 00 1E : ................
00000110   : 00 00 00 00 00 00 00 00 00 1C 00 1E FF FF       : ..............

We can safely discard this Resource Fork data.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:17 +01:00
Ralph Boehme
2dbb2d2873 vfs_fruit: add option "wipe_intentionally_left_blank_rfork"
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-31 21:27:17 +01:00
Ralph Boehme
5d565f636f vfs_fruit: remove check for number of xattrs from ad_convert_xattr
Turns out that there exist AppleDouble files with an extended FinderInfo
entry that includes the xattr marshall buffer, but the count of xattrs
in the buffer is just zero.

We do want to discard this extended FinderInfo entry and convert it to a
simple fixed size FinderInfo entry, so remove the check.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-29 23:36:24 +01:00
Björn Baumbach
ff3e2fa84f vfs_full_audit: ntimes: log a-, m-, c- and creation-time
Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Björn Baumbach <bb@sernet.de>
Autobuild-Date(master): Thu Oct 11 13:40:27 CEST 2018 on sn-devel-144
2018-10-11 13:40:27 +02:00
Ralph Boehme
31daab88e6 vfs_fruit: move check in ad_convert() to ad_convert_*() subfunctions
Currently the whole conversion is skipped if the FinderInfo entry in the
AppleDouble file is of the default size (ie not containing xattrs).

That also means we never converted FinderInfo from the AppleDouble file
to stream format. This change finally fixes this.

Note that this keeps failing with streams_depot, much like the existing
known-fail of "samba3.vfs.fruit streams_depot.OS X AppleDouble file
conversion". Fixing the conversion to work with vfs_streams_depot is a
task for another day.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Thu Oct 11 01:30:13 CEST 2018 on sn-devel-144
2018-10-11 01:30:13 +02:00
Ralph Boehme
9cf087a474 vfs_fruit: make call to ad_convert_truncate() optional
Call ad_convert_truncate() based on whether the previous call
ad_convert_xattr() returned converted_xattr=true.

Upcoming fixes for a different Samba bug (#13642) will hook into calling
ad_convert_truncate() in other cases, this also prepares for that.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:15 +02:00
Ralph Boehme
acb72c1ea7 vfs_fruit: add out arg "converted_xattr" to ad_convert_xattr
Used to let the caller know if a conversion has been done. Currently not
used in the caller, that comes next.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:15 +02:00
Ralph Boehme
5598e6bc35 vfs_fruit: add check for OS X filler in FinderInfo conversion
This ensures that the function only acts on AppleDouble files created by
macOS and not AppleDouble files created by us that are already in the
correct format (only using the Resource Fork).

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:15 +02:00
Ralph Boehme
70d3ae5a89 vfs_fruit: call ad_convert_move_reso() from ad_convert_xattr()
ad_convert_xattr() is the place that triggers the need to move the
resource fork, so it should also call ad_convert_move_reso().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:15 +02:00
Ralph Boehme
1692ca5fd8 vfs_fruit: let the ad_convert_*() subfunction update the on-disk AppleDoube header as needed
Another step in simplifying ad_convert() itself. It means that we may
write to disk twice, but is only ever done once per AppleDouble file.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:15 +02:00
Ralph Boehme
918c6c5990 vfs_fruit: let the ad_convert_*() subfunctions mmap as needed
This may mean that we mmap twice when we convert an AppleDouble file,
but this is the only sane way to cleanly modularize ad_convert().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:14 +02:00
Ralph Boehme
f91e0c857a vfs_fruit: fix error returns in ad_convert_xattr()
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:14 +02:00
Ralph Boehme
93b7e05621 vfs_fruit: use ADEDOFF_RFORK_DOT_UND offset macro in ad_convert_move_reso()
We really want the fixed size offset here, not a calculated one. Note
that "ad_getentryoff(ad, ADEID_FINDERI) + ADEDLEN_FINDERI" is equal to
ADEDOFF_RFORK_DOT_UND.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:14 +02:00
Ralph Boehme
4c7e1de46f vfs_fruit: split out moving of the resource fork
No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:14 +02:00
Ralph Boehme
b948681b2b vfs_fruit: use ADEDOFF_RFORK_DOT_UND offset macro in ad_convert_truncate()
We really want the fixed size offset here, not a calculated one. Note
that "ad_getentryoff(ad, ADEID_RFORK)" is equal to ADEDOFF_RFORK_DOT_UND
in this case.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:14 +02:00
Ralph Boehme
98bd7c0a46 vfs_fruit: split out truncating from ad_convert()
This may look a little ill-advised as this increases line count, but
the goal here is modularizing ad_convert() itself and making it as slick
as possible helps achieving that goal.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:14 +02:00
Ralph Boehme
4f1174b6eb vfs_fruit: move FinderInfo lenght check to ad_convert()
The final step in consolidating all conversion related work in
ad_convert(). No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:14 +02:00
Ralph Boehme
d27d0326c3 vfs_fruit: move FinderInfo conversion to helper function and call it from ad_convert()
No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:13 +02:00
Ralph Boehme
b355a09576 vfs_fruit: move storing of modified struct adouble to ad_convert()
ad_convert() modified it, so let ad_convert() also save it to disk. No
change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:13 +02:00
Ralph Boehme
99cc9ef82b vfs_fruit: remove unneeded fd argument from ad_convert()
Use the struct adouble member ad_fd instead of passing it as an
argument. Who did that in the first place? :)

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:13 +02:00
Ralph Boehme
8bc36d723f vfs_fruit: do direct return from error checks in ad_convert()
Subsequent commits will move the mmap() into the subfunctions. This
change just prepares for that.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:13 +02:00
Ralph Boehme
d161e04771 vfs_fruit: move setting ADEID_FINDERI length to ad_convert_xattr()
ad_convert_xattr() does the conversion of the xattr data in the
AppleDouble file, so we should update it's size there and should not
defer it to the caller.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:13 +02:00
Ralph Boehme
8ee7e6135e vfs_fruit: store filler bytes from AppleDouble file header in struct adouble
This can later be used to distinguish between macOS created AppleDouble
files and AppleDouble files created by Samba or Netatalk.

macOS:    "Mac OS X        "
Samba:    "Netatalk        "
Netatalk: "Netatalk        "

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:13 +02:00
Ralph Boehme
7e010abbaa vfs_fruit: fix two comments
Thanks to the recent addition of ad_convert_xattr() we now correctly
handle this case.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-10-10 22:22:13 +02:00
Christof Schmitt
8453c61789 vfs_gpfs: Check for GPFS file system on connect
The vfs_gpfs modules uses GPFS API calls that only succeed when using
the module with the GPFS file system. Add an explicit statfs check for
the file system type on connect, to make it obvious when the file system
is missing or not mounted. The check can be skipped by setting
gpfs:check_fstype to 'no'.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-09-19 00:54:23 +02:00
Volker Lendecke
aca0f2a18b streams_xattr: Make error handling more obvious
Do the NULL check right after the alloc call

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-09-14 07:49:14 +02:00
Volker Lendecke
36055dbb66 streams_xattr: Make error handling more obvious
Do the NULL check right after the alloc call

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-09-14 07:49:14 +02:00
Volker Lendecke
a5f9b33cb4 vfs_fruit: fix an uninitialized variable error
clang does not recognize "smb_panic" as an "exit()" equivalent

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-09-07 17:26:18 +02:00
Christof Schmitt
ae7db3e543 s3: Rename server_event_context() to global_event_context()
This reflects that the event context is also used outside of the server
processes.

The command used for the rename:
find . -name '*.[hc]' -print0 | xargs -0 sed -i 's/server_event_context/global_event_context/'

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-09-07 17:26:17 +02:00
Thomas Nagy
8077f462c9 build:wafsamba: Build on waf 1.9
Signed-off-by: Thomas Nagy <tnagy@waf.io>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-05 06:37:21 +02:00
Ralph Boehme
44840ba5b3 vfs_delay_inject: adding delay to VFS calls
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-08-31 22:22:22 +02:00
Jeremy Allison
4d72ebb821 s3: VFS: vfs_full_audit: Ensure smb_fname_str_do_log() only returns absolute pathnames.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13565

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Aug 27 20:23:55 CEST 2018 on sn-devel-144
2018-08-27 20:23:55 +02:00
Jeremy Allison
59f1334726 s3: VFS: vfs_full_audit: Add $cwd arg to smb_fname_str_do_log().
Not yet used.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13565

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2018-08-27 17:28:24 +02:00
Volker Lendecke
8c14234871 vfs_fruit: Don't unlink the main file
The original fix for bug 13441 was missing a check that verifies that
fruit_ftruncate() is actually called on a stream.

Follow-up to

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13441

Pair-Programmed-With: Volker Lendecke <vl@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Aug 23 15:28:48 CEST 2018 on sn-devel-144
2018-08-23 15:28:47 +02:00
Volker Lendecke
51d5707379 vfs_fruit: Fix a leak of "br_lck"
Fix a panic if fruit_access_check detects a locking conflict.

do_lock() returns a valid br_lck even in case of a locking conflict.
Not free'ing it leads to a invalid lock order panic later, because
"br_lck" corresponds to a dbwrap lock on brlock.tdb.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13584

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-08-20 23:28:25 +02:00
Volker Lendecke
c2ea100777 lib: Pass mem_ctx to state_path()
Fix a confusing API: Many places TALLOC_FREE the path where it's not
clear you have to do it.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2018-08-17 11:30:11 +02:00
Ralph Wuerthner
4909b96605 s3: vfs: time_audit: fix handling of token_blob in smb_time_audit_offload_read_recv()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13568

Signed-off-by: Ralph Wuerthner <ralph.wuerthner@de.ibm.com>
Reviewed-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-08-13 19:46:08 +02:00
Ralph Boehme
0736fdcdb0 smbd: use async dos_mode_at_send in smbd_smb2_query_directory_send()
Finally: use the new dos_mode_at_send() in the directory enumeration
loop. This means that fetching the DOS attributes for directory entries
is done asynchronously with regard to the enumeration loop.

As the DOS attribute is typically read from an extended attribute in the
filesytem, this avoids sequentially blocking on IO. If the IO subsystem
is slow servicing these request, enabling async processing can result in
performance improvements.

A parametric option

  smbd:async dosmode = true | false (default: false)

can be used to enable the new async processing.

Simulating slow IO with usleep(5000) in the synchronous and asynchronous
versions of SMB_VFS_GET_DOS_ATTRIBUTES(), the results of enumerating a
directory with 10,000 files are:

    smbd:async dosmode = no:

        $ time bin/smbclient -U slow%x //localhost/test -c "ls dir\*" > /dev/null
        real    0m59.597s
        user    0m0.024s
        sys     0m0.012s

    smbd:async dosmode = yes:

        $ time bin/smbclient -U slow%x //localhost/test -c "ls dir\*" > /dev/null
        real    0m0.698s
        user    0m0.038s
        sys     0m0.025s

Performance gains in real world workloads depends on whether the actual
IO requests can be merged and parallelized by the kernel. Without such
wins at the IO layer, the async processing may even be slower then the
sync processing due to the additional overhead.

The following parameters can be used to adapt async processing behaviour
for specific workloads and systems:

        aio max threads = X (default: 100)
        smbd:max async dosmode = Y (default: "aio max threads" * 2)

By default we have at most twice the number of async requests in flight
as threads provided by the underlying threadpool. This ensures a worker
thread that finishes a job can directly pick up a new one without going
to sleep.

It may be advisable to reduce the number of threads to avoid scheduling
overhead while also increasing "smbd:max async dosmode".

Note that we disable async processing for certain VFS modules in the VFS
connect function to avoid the overhead of triggering the sync fallback
in dos_mode_at_send(). This is done for VFS modules that implement the
sync SMB_VFS_GET_DOS_ATTRIBUTES(), but not the async version (gpfs), and
for VFS modules that don't share a real filesystem where fchdir() can be
used (ceph, gluster). It is disabled for catia, because we realized that
the catia name translation macros used on
fsps (CATIA_FETCH_FSP_[PRE|POST]_NEXT) have a bug (#13547).

We use threadpool = smb_vfs_ev_glue_tp_chdir_safe() and then
pthreadpool_tevent_max_threads(threadpool) to get the number of maximum
worker threads which matches the pool used by the low level
SMB_VFS_GETXATTRAT_[SEND|RECV] implementation in vfs_default.

This is a terrible abstraction leak that should be removed in the future
by maybe making it possible to ask a VFS function which threadpool it
uses, internally suporting chaining so VFS function FOO that internally
uses BAR can forward the question to BAR.

On a hyphotetical system that had a getxattrat(dirfd, path, ...)
syscall and at the same time doesn't support per-thread current working
directories (eg FreeBSD doesn't have the latter) but has support for
per-thread-credentials, pthreadpool_tevent_max_threads() on the
tp_chdir_safe threadpool returns 1.

So when hooking the hyphotetical getxattrat() into the async
SMB_VFS_GETXATTRAT_[SEND|RECV] implementation in an VFS module, the
implementation could use the tp_path_safe threadpool, but the SMB2
layer would use the wrong threadpool in the call to
pthreadpool_tevent_max_threads(), resulting in no parallelism.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-27 13:07:15 +02:00
Ralph Boehme
1265b51673 vfs_default: implement SMB_VFS_GET_DOS_ATTRIBUTES_SEND/RECV
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-27 13:07:14 +02:00
Ralph Boehme
47d77432e4 s3: vfs: add SMB_VFS_GET_DOS_ATTRIBUTES_SEND/RECV
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-27 13:07:14 +02:00
Ralph Boehme
8a6013f612 vfs_xattr_tdb: implement SMB_VFS_GETXATTRAT_SEND/RECV
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-27 13:07:14 +02:00
Ralph Boehme
c1c8a1f635 vfs_default: implement SMB_VFS_GETXATTRAT_SEND/RECV
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-27 13:07:14 +02:00
Ralph Boehme
7e1f9c9d83 s3: vfs: add SMB_VFS_GETXATTRAT_SEND/RECV
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-27 13:07:14 +02:00
Stefan Metzmacher
bd79564af1 s3:modules: add vfs_not_implemented module
This provides helper functions, which can be used by other modules,
if they don't implement a specific function.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-25 17:49:07 +02:00
Ralph Boehme
cd37badc02 vfs_aio_pthread: use event context and threadpool from user_vfs_evg
Or the root glue in case we're already root.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
2018-07-25 17:49:07 +02:00
Ralph Boehme
2dd95c1c38 s3: vfs: add user_vfs_evg to connection_struct
This will be used to in order to pass down the
impersonation magic from the SMB layer through
the SMB_VFS layer.

This includes the following options:

smbd:force sync user path safe threadpool
smbd:force sync user chdir safe threadpool
smbd:force sync root path safe threadpool
smbd:force sync root chdir safe threadpool

They can be used in order to test the non linux code
path on linux, once we get code that makes full use
of the new infrastructure.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
2018-07-25 17:49:06 +02:00
Ralph Boehme
2be7518ae5 smbd: rename sconn->pool to sconn->raw_thread_pool
This should in future not be used directly, we'll provide
wrapper pools, which will provide impersonation for
path based async calls.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-25 17:49:06 +02:00
Ralph Boehme
27bb2cbc2e vfs_default: fix async fsync idle/busy time profiling
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-25 00:23:13 +02:00
Stefan Metzmacher
1bc92d1090 vfs_default: call smb_vfs_assert_all_fns()
This module needs to implement every call.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-25 00:23:12 +02:00
Ralph Boehme
0d2eeb9422 lib/util: rename USE_LINUX_THREAD_CREDENTIALS to HAVE_LINUX_THREAD_CREDENTIALS
The define reflects the results of a feature test, not a configure
option.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-24 17:38:28 +02:00
Stefan Metzmacher
0dcaa0707b smbd: make use of smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers
For now they just add debugging, but that will change shortly.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Timur I. Bakeyev
3d0ed62f91 vfs_audit: Extend list of recognized syslog(3) facilities and wrap them into #ifdef's. That list should be comprehensive enough to cover most of the existing OSes.
Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Wed Jul 11 08:44:10 CEST 2018 on sn-devel-144
2018-07-11 08:44:10 +02:00
Pooja Mahadik
36b4b56554 pass 'rdonly' or 'directory' flag to open a directory file.
Signed-off-by: Pooja Mahadik <pooja.mahadik@veritas.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Jul 11 02:22:18 CEST 2018 on sn-devel-144
2018-07-11 02:22:18 +02:00
David Disseldorp
926ae50627 vfs_ceph: don't lie about flock support
Instead, match vfs_gluster behaviour and require that users explicitly
disable "kernel share modes".

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13506

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-07-06 20:37:20 +02:00
Timur I. Bakeyev
d63b24d5ae Make "none" the default setting for the successful and failed operations in the vfs_full_audit, so you don't blow up your server by just adding this module to the configuration.
Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-05 09:40:59 +02:00
Timur I. Bakeyev
32e5b18d49 Make sure that vfs*audit modules recognize and accept all the syslog facilities.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13436

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-05 09:40:59 +02:00
Stefan Metzmacher
f2df8be16b smbd: add an effective connection_struct->user_ev_ctx that holds the event context used for the current user
This will be filled with an impersonation wrapper in the next commits.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-06-18 08:59:21 +02:00
Stefan Metzmacher
b0af5715b0 vfs_glusterfs: explain that/why we use the raw tevent_context in init_gluster_aio()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-06-18 08:59:19 +02:00
Stefan Metzmacher
db83a8f7c8 vfs_default: maintain vfswrap_offload_write_state->{src_ev,dst_ev}
These get filled with impersonation wrappers in the following commits.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-06-18 08:59:18 +02:00
Stefan Metzmacher
506c9b37df vfs_default: make use of change_to_user_by_fsp() in order to switch between src and dst fsp
This may matter if at least one share uses "force user".

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-06-18 08:59:18 +02:00
Stefan Metzmacher
2d75da9f61 vfs_btrfs: make use of become_user_by_fsp() in order to switch between src and dst fsp
We can use become_user_by_fsp()/unbecome_user() as it spans only parts of
the btrfs_offload_write_send() function and never goes async in between.

This may matter if at least one share uses "force user".

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-06-18 08:59:18 +02:00
Stefan Metzmacher
4512b91897 vfs_btrfs: don't keep state->subreq in btrfs_offload_write_send/recv()
This can be a local variable as used in most of our tevent_req based
code.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-06-18 08:59:18 +02:00
Stefan Metzmacher
2317316233 vfs_btrfs: update s/btrfs_cc_state/btrfs_offload_write_state/ s/cc_state/state/
This matches our naming conventions used for tevent_req based functions.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-06-18 08:59:18 +02:00
Stefan Metzmacher
5ac91e487d vfs_btrfs: remove unused checks which are already caught by vfs_offload_token_check_handles()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-06-18 08:59:18 +02:00
Stefan Metzmacher
4af45b8ff6 vfs_default: remove unused checks which are already caught by vfs_offload_token_check_handles()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-06-18 08:59:18 +02:00
Stefan Metzmacher
55b1b5568f vfs_glusterfs: use tevent_req_defer_callback() in order to use the correct event context
The callback and _recv() functions should be called from the same
event context that was passed to the _send() function.

In future the completion pipe should be replaced by
tevent_threaded_context_create()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-06-14 20:52:23 +02:00
Swen Schillig
d33c355103 CID 1416475: possibly dereferencing NULL in fruit_pread_meta
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Reviewed-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Jun 13 16:29:48 CEST 2018 on sn-devel-144
2018-06-13 16:29:48 +02:00
Swen Schillig
9715e55e65 CID 1416476: possibly dereferencing NULL in fruit_ftruncate_rsrc
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Reviewed-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-06-13 13:41:45 +02:00
Swen Schillig
808b7e2b03 CID 1416477: possibly dereferencing NULL in fruit_pwrite_meta
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Reviewed-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-06-13 13:41:45 +02:00
Swen Schillig
c0e4818d68 CID 1416478: de-ref NULL value in fruit_pwrite_rsrc
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Reviewed-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-06-13 13:41:45 +02:00
Jeremy Allison
44f3bf1f63 s3: VFS: Remove unused enum value.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Sat Jun  2 03:03:42 CEST 2018 on sn-devel-144
2018-06-02 03:03:42 +02:00
Ralph Boehme
46d127865f vfs_fruit: delete 0 byte size streams if AAPL is enabled
macOS SMB server uses xattrs as storage backend for streams, directly
exposing xattr get/set characteristics. Setting EOF on a stream to 0
just deletes the xattr as macOS doesn't support 0-byte sized xattrs.

Note that this does not apply to the AFP_AfpInfo and AFP_Resource
streams, they have even stranger semantics and we have other tests
for those.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13441

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed May 30 02:34:29 CEST 2018 on sn-devel-144
2018-05-30 02:34:29 +02:00
Christof Schmitt
1531fcde82 vfs_gpfs: Use full_path_tos instead of talloc_asprintf
full_path_tos avoids the talloc call for most cases; use that instead of
talloc_asprintf.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat May 26 01:43:09 CEST 2018 on sn-devel-144
2018-05-26 01:43:09 +02:00
Christof Schmitt
1843a2d632 vfs_gpfs: Remove get_full_smb_filename from is_offline check
No stream information is required here.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-05-25 22:52:12 +02:00
Christof Schmitt
4a29a949ac vfs_gpfs: Remove wrong get_full_smb_filename from ntimes function
Updating the timestamps requires the path to the file, but no stream
information.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-05-25 22:52:12 +02:00
Christof Schmitt
01a6aef522 vfs_gpfs: Adjust debug level when get_winattrs returns EBADF
This is returned for a get_winattrs call against a non-gpfs file system.
This can happen for the .. entry when listing a share on the file system
root.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-05-25 22:52:12 +02:00
Jeremy Allison
52778afdab s3: vfs_fake_acls: Correctly implement the chmod/fchmod algorithm on fake acls.
We now pass samba3hide(nt4_dc), so remove it from knownfail.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri May 25 21:29:32 CEST 2018 on sn-devel-144
2018-05-25 21:29:32 +02:00
Jeremy Allison
5bbb831041 s3: VFS: Remove SMB_VFS_FCHMOD_ACL().
No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-05-25 18:39:25 +02:00
Jeremy Allison
55f013edbb s3: modules: vfs_ceph: Remove FCHMOD_ACL in cephwrap_fchmod().
Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-05-25 18:39:25 +02:00
Jeremy Allison
0c65f6b833 s3: modules: vfs_default: Remove FCHMOD_ACL in fchmod.
Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-05-25 18:39:24 +02:00
Jeremy Allison
7b8fa17f06 s3: VFS: Remove SMB_VFS_CHMOD_ACL().
No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-05-25 18:39:24 +02:00
Jeremy Allison
b8477abbde s3: modules: vfs_ceph: Remove CHMOD_ACL in cephwrap_chmod().
Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-05-25 18:39:24 +02:00
Jeremy Allison
d50bb46038 s3: modules: vfs_ceph: Remove CHMOD_ACL in cephwrap_mkdir().
Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-05-25 18:39:24 +02:00
Jeremy Allison
c5060e1ad5 s3: modules: vfs_default: Remove CHMOD_ACL in chmod.
Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-05-25 18:39:24 +02:00
Jeremy Allison
90117f25bf s3: modules: vfs_default: Remove CHMOD_ACL in mkdir.
Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed. If the mkdir
succeeded it's already set the correct mode.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-05-25 18:39:24 +02:00
Volker Lendecke
ae196c5063 vfs_time_audit: Fix a log message
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-05-25 13:08:47 +02:00
Ralph Boehme
f93cc23237 vfs_zfsacl: return synthesized ACL when ZFS return ENOTSUP
This allows accessing the ZFS .snapshots directory where ZFS returns
ENOTSUP when calling acl(".snapshots").

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13175

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri May 18 22:03:21 CEST 2018 on sn-devel-144
2018-05-18 22:03:21 +02:00
Andreas Schneider
e7e4362ba2 s3:modules: Initialize pointers in vfs_virusfilter
Found by Coverity.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-05-16 21:30:22 +02:00
Timur I. Bakeyev
d3cbcbd5c0 Remove extra 0x prefix for the "%p" format specifiers, avoiding 0x0x0 strings in the output.
Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2018-05-16 18:06:23 +02:00