1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-26 10:04:02 +03:00

77 Commits

Author SHA1 Message Date
Ralph Boehme
c9b6298249 mdssvc.idl: pass policy_handle as pointer
No change in behaviour, this just changes all functions to take the
policy_handle argument as pointer instead of passing it by value.
This is how all other IDLs pass it.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Oct  9 15:52:55 UTC 2019 on sn-devel-184
2019-10-09 15:52:55 +00:00
Ralph Boehme
63cbeaf0e7 s3:mdssvc: fix service startup in deamon mode
Changes:

* Don't initialize the RPC service by calling setup_rpc_module() in the parent
  mdssd. This is not needed in the parent, only in the worker childs.

* In the worker childs call setup_rpc_module() instead of init_rpc_module()
  which ensures rpc_mdssvc_init() is called with the mdssvc callback which is
  needed to initialize mdssvc via mdssvc_init_cb() -> init_service_mdssvc()

* Finally rpc_setup_mdssvc() is adjusted to be a noop if mdssvc is configured to
  as external and when called by the main parent smbd via dcesrv_ep_setup() ->
  setup_rpc_modules()

I've manually tested all 4 combinations of external=yes|no X module=yes|no with
the new mdfind command.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
2019-10-09 14:35:29 +00:00
Ralph Boehme
78a2d95511 s3:mdssvc: use a helper variable for the service type
No change in behaviour. Simplifies a subsequent logical change.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
2019-10-09 14:35:29 +00:00
Ralph Boehme
f479e01096 s3:mdssvc: add unit tests for the Spotlight to Elasticsearch parser
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
2019-10-09 14:35:29 +00:00
Ralph Boehme
f5510d7db3 s3:mdssvc: add Elasticsearch backend
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
2019-10-09 14:35:29 +00:00
Ralph Boehme
c338bdf5a4 s3:mdssvc: add missing mds_ctx deallocation
The mds_ctx object was created in _mdssvc_open() as a talloc child of the pipe
which means as long as the pipe is connected it's not freed.

To ensure we do proper rundown of all resources including backend connections
and pending queries, we must free the mds_ctx object.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
2019-10-09 14:35:29 +00:00
Ralph Boehme
eb740b571d s3:mdssvc: fix a long line
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
2019-10-09 14:35:29 +00:00
Ralph Boehme
ba899694e4 s3:mdssvc: modernize a few DEBUG macros
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
2019-10-09 14:35:29 +00:00
Ralph Boehme
08cb82d26f s3:mdssvc: remove unused snum from struct sl_query
Looks like this was never used, it's also available via mds_ctx->snum.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
2019-10-09 14:35:28 +00:00
Volker Lendecke
2dd7ebe072 smbd: Fix CID 1452293 Incorrect expression (NO_EFFECT)
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Aug 14 11:48:42 UTC 2019 on sn-devel-184
2019-08-14 11:48:42 +00:00
Volker Lendecke
76bf8a1ac4 mdssvc: Fix the clang build
clang complains about "%lu" not to match size_t on 32-bit FreeBSD

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Aug  9 07:34:05 UTC 2019 on sn-devel-184
2019-08-09 07:34:04 +00:00
Ralph Boehme
64f182412a s3:mdssvc: fix slrpc_fetch_attributes() when CNID is not known
Samba currenlty fails the whole RPC request, macOS returns returns a nil entry
for the requested CNID:

DALLOC_CTX(#1): {
	sl_array_t(#3): {
		uint64_t: 0x0000
		CNIDs: unkn1: 0xfec, unkn2: 0x6b000020
			DALLOC_CTX(#1): {
				uint64_t: 0xe4bbf314c03b1e
			}
		sl_filemeta_t(#1): {
			sl_array_t(#2): {
				nil
				nil
			}
		}
	}
}

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Aug  8 21:43:14 UTC 2019 on sn-devel-184
2019-08-08 21:43:14 +00:00
Ralph Boehme
b2bf13ecf7 s3:mdssvc: close mdssvc rpc command must return in handle
Checked against macOS mdssvc.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:33 +00:00
Ralph Boehme
a5e705504b s3:mdssvc: failing the RPC request if the mdssvc policy handle is not found
Turns out macOS mdssvc doesn't fail the RPC request if the policy handle is all
zero. Also, if it fails with a non-all-zero handle, it returns a different RPC
error, namely DCERPC_NCA_S_PROTO_ERROR, not DCERPC_FAULT_CONTEXT_MISMATCH (or
rather their mapped NT_STATUS codes).

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:33 +00:00
Ralph Boehme
6336699687 s3:mdssvc: the open command must work on shares with Spotlight disabled
Move the implementation of this setting down to the actual search query
processing. macOS has no notion of "spotlight = false" at the DCERPC layer and
the open request will always succeed even on all shares.

When later the client issues search requests on such shares, we ensure we use
the noindex backend.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:33 +00:00
Ralph Boehme
520175e227 s3:mdssvc: use an early return if spotlight is not enabled.
No change in behaviour, best viewed with git show -w.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:33 +00:00
Ralph Boehme
68a9d1205e s3:mdssvc: update a few debug macros to modern style
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:33 +00:00
Ralph Boehme
940c3b31dd s3:mdssvv: don't fail the RPC request if the share name is unknown
Taken from macOS. We have to return an empty share_path and an empty policy
handle, but not fail the RPC request.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:33 +00:00
Ralph Boehme
fef3a4c75f s3:mdssvc: initialize the returned share_path with the empty string
macOS returns the empty path for an unknown share. This paves the way for that
change. Currently we still fail the RPC request if the share is not known with
DCERPC_FAULT_CANT_PERFORM, but this is wrong and is going to be changed in the
next commit.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:33 +00:00
Ralph Boehme
c4b2dc95d5 s3:mdssvc: macOS returns the client values in the open command
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:33 +00:00
Ralph Boehme
a3cb4fca86 s3:mdssvc: macOS returns UINT64_MAX, not 0 for closeQueryForContext mdscmd
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:33 +00:00
Ralph Boehme
5750f38ca7 s3:mdssvc: fix error handling of mdssvc RPC requests
It seems for certain error cases macOS just sends an empty response
blob. So if our mdssvc request processing fails, we should just return an empty
response blob, but not fail the mdssvc request at the DCERPC layer.

Example, passing "xxx" as sharename which does not exist at the server:

  $ bin/rpcclient -U slow%pass macmini -c "fetch_attributes xxx /foo/bar 123" -d 10
  ....
  Got pdu len 56, data_len 32
  rpc_api_pipe: got frag len of 56 at offset 0: NT_STATUS_OK
  rpc_api_pipe: host macmini returned 32 bytes.
  mdssvc_cmd: struct mdssvc_cmd
     out: struct mdssvc_cmd
         fragment                 : *
             fragment                 : 0x00000000 (0)
         response_blob            : *
             response_blob: struct mdssvc_blob
                 length                   : 0x00000000 (0)
                 size                     : 0x00010000 (65536)
                 spotlight_blob           : *
                     spotlight_blob: ARRAY(0)
         unkn9                    : *
             unkn9                    : 0x00000000 (0)
  ...

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:33 +00:00
Ralph Boehme
b34fd5b997 s3:mdssvc: fix unmarshalling of empty CNID array
len=0 is invalid, len=8 is an empty array, len>8 is an array with members, so
for the len=8 case we must add the empty cnid array.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:32 +00:00
Ralph Boehme
29ef030624 s3:mdssvc: marshalling: fix unpacking empty CNID structure
Pass the correct tag member tag.size to sl_unpack_CNID(), not
tag.length. tag.size is the size actually used in a buffer of size tag.length.

Cf other users of tag.size that already do this correctly, this was only wrong
in this place.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:32 +00:00
Ralph Boehme
d67c98184a s3:mdssvc: marshalling: fix unpacking empty filemeta structure
This is how a correct dump of an empty sl_filemeta_t container should look like:

DALLOC_CTX(#1): {
	sl_array_t(#3): {
		uint64_t: 0x0023
		CNIDs: unkn1: 0x0, unkn2: 0x0
			DALLOC_CTX(#0): {
			}
		sl_filemeta_t(#0): {
		}
	}
}

This is basically the response from macOS mdssvc for a query that yields no
results: sl_filemeta_t is empty, the CNIDs array as well.

Looking at the raw packet data, the empty sl_filemeta_t container as a size of 8
bytes which fails the following check in sl_unpack_cpx():

        case SQ_CPX_TYPE_FILEMETA:
                ...
		if (tag.size < 16) {
		        *boom*
                }

Only tag.size=0 is invalid, tag.size=8 denotes an empty container and tag.size>=16
denotes a sl_filemeta_t container with actual content must be unpacked by
calling sl_unpack(). Note that size is always a muliple of 8.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:32 +00:00
Ralph Boehme
ed37f2d3c4 s3:mdssvc: marshalling: add a newline to a DEBUG message
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:32 +00:00
Ralph Boehme
25c5012c53 s3:mdssvc: avoid strncpy when marshalling strings
Avoids failure when at O3 level:

  [2082/4232] Compiling source3/rpc_server/mdssvc/marshalling.c

  ==> /builds/samba-team/devel/samba/samba-o3.stderr <==
  In file included from /usr/include/string.h:494,
                   from /usr/include/bsd/string.h:30,
                   from ../../lib/tevent/../replace/replace.h:164,
                   from ../../source3/include/includes.h:23,
                   from ../../source3/rpc_server/mdssvc/marshalling.c:21:
  In function ‘strncpy’,
      inlined from ‘sl_pack_string’ at ../../source3/rpc_server/mdssvc/marshalling.c:493:2,
      inlined from ‘sl_pack_loop’ at ../../source3/rpc_server/mdssvc/marshalling.c:607:13:
  /usr/include/bits/string_fortified.h:106:10: error: ‘__builtin_strncpy’ output
  truncated before terminating nul copying as many bytes from a string as its
  length [-Werror=stringop-truncation]
    106 |   return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest));
        |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ../../source3/rpc_server/mdssvc/marshalling.c: In function ‘sl_pack_loop’:
  ../../source3/rpc_server/mdssvc/marshalling.c:458:8: note: length computed here
    458 |  len = strlen(s);
        |        ^~~~~~~~~
  cc1: all warnings being treated as errors

Marshalled strings are not 0 terminated.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:32 +00:00
Ralph Boehme
0b1d9d40c2 s3:mdssvc: supposed status field is in fact a fragment indicator
Spotted this in mdssvc response that containied many results for a search
request: if the mdssvc response blob is larger then ~32k, the server fragments
the response in 32k fragments and sets the "fragment" field to 1.

Note that mdssvc implemenets result set "fragmentation" at the result set layer,
not at the marshalled response buffer layer. Therefor mdssvc always sets this
field to 0.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:32 +00:00
Ralph Boehme
c742ab7a4c s3:mdssvc: add noindex backend
Add a new default backend that, while allowing mdsvc RPC and search queries from
clients, always returns no results.

Shares using this backend will behave the same way as shares on a macOS SMB
server where indexing is disabled.

This change will later also allow us to compile the Spotlight RPC service by
default which is a big step in the direction of adding tests to CI.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:32 +00:00
Ralph Boehme
e5a4114bb5 s3:mdssvc: log inode number in ino_path_map_destr_cb()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:31 +00:00
Ralph Boehme
f58026b30b s3:mdssvc: mds_dalloc_dump() -> dalloc_dump()
Move mds_dalloc_dump() to dalloc and rename it to dalloc_dump().

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:31 +00:00
Ralph Boehme
b5731a94bb s3:mdssvc: add Unicode normalisation
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:31 +00:00
Ralph Boehme
1b897df4f7 s3:mdssvc: new option "spotlight backend"
Currently there's only the tracker backend, but subsequent commits will add
other backends.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:31 +00:00
Ralph Boehme
66b8d86d71 s3:mdssvc: pass down sharename and SNUM
Not used for now, will be needed in the upcoming Elasticsearch backend.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:31 +00:00
Ralph Boehme
1ef96e0907 s3-mdssvc: factor out Tracker backend logic
This moves all Tracker backend logic into a modularized component.

This should not result in any change in behaviour, it just paves the way
for adding additional backends. Currently the only available backend is
Gnome Tracker.

slq_destroy_send/recv is not needed anymore as the problem is solved now by
correctly checking if an async Tracker request was cancelled and we got
G_IO_ERROR_CANCELLED in tracker_con_cb() or tracker_query_cb() and avoid using
user_data in that the case.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:31 +00:00
Ralph Boehme
f80f8c5f99 s3:mdssvc: move debug macro to header file
This is going to be used from other parts of the code soon.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:31 +00:00
Ralph Boehme
17f29fa867 s3:mdssvc: README.Coding fixes, long lines, modernize debug macros
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-08-08 20:24:31 +00:00
Samuel Cabrero
d1d37a0a6e s3:rpc_server: Return NTSTATUS in rpc_ep_setup and setup functions
Signed-off-by: Samuel Cabrero <scabrero@suse.de>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-07-22 16:49:14 +00:00
Ralph Boehme
9053391f86 s3:mdssvc: fix flex compilation error
[4440/4495] Compiling bin/default/source3/rpc_server/mdssvc/sparql_lexer.lex.c
../../source3/rpc_server/mdssvc/sparql_lexer.l:26: error: "yyalloc" redefined [-Werror]
26 | #define yyalloc SMB_MALLOC

Looks like the dirty redefine trick doesn't work anymore with newer flex
versions. According to the flex manual the right thing to do is to provide own
functions for yyalloc and yyrealloc when passing the options "noyyalloc
noyyrealloc".

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue May 28 11:49:06 UTC 2019 on sn-devel-184
2019-05-28 11:49:06 +00:00
Ralph Boehme
613acd3ddd s3-mdssvc: add a comment to mds_init()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <npower@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Apr 24 19:32:12 UTC 2019 on sn-devel-184
2019-04-24 19:32:12 +00:00
Ralph Boehme
003b5d3ee4 s3-mdssvc: make mds_ctx_destructor_cb static
This is only used in this compilation unit.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2019-04-24 18:32:15 +00:00
Ralph Boehme
91c9c03435 s3-mdssvc: add missing call to g_cancellable_new()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2019-04-24 18:32:15 +00:00
Ralph Boehme
984c890cd0 s3-mdssvc: use default g_main context
Way back when the module was developed it seemed to be necessary the use
a private context with push/pop as thread default. Maybe there was a bug
in libtracker-sparql dispatching callback in the wrong (global)
context. It's not necessary anymore with a recent libtracker-sparql
version.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2019-04-24 18:32:15 +00:00
Ralph Boehme
55b2cca14a s3-mdssvc: use tevent_glib_glue in mdssvc RPC service
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2019-04-24 18:32:15 +00:00
Ralph Boehme
bc053abdd1 s3-mdssvc: call [un]become_authenticated_pipe_user()
This ensures we're running as the authenticated user int the tevent
callback which might be running in an arbitrary impersonation context.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2019-04-24 18:32:15 +00:00
Ralph Boehme
675902f800 s3-mdssvc: add tevent context arg to mds_init_ctx
This is needed later when adding tevent_glib_glue support, not used for now.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2019-04-24 18:32:15 +00:00
Volker Lendecke
b0077bb059 rpc_server3: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-12-11 00:40:31 +01:00
Volker Lendecke
f8df3f3499 mdssvc: Move a variable declaration closer to its use
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Sep 14 10:48:47 CEST 2018 on sn-devel-144
2018-09-14 10:48:47 +02:00
Volker Lendecke
98cf04c7d1 mdssvc: Move a variable declaration closer to its use
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-09-14 07:49:14 +02:00
Volker Lendecke
71c7060798 mdssvc: Move a variable declaration closer to its use
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-09-14 07:49:14 +02:00