1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-11 05:18:09 +03:00
Commit Graph

120065 Commits

Author SHA1 Message Date
Volker Lendecke
2379dc29bc smbd: Add share_mode_watch_send/recv
For now this is a simple wrapper around dbwrap_watched_watch_send()
that will make the direct db_record reference in struct
share_mode_data unnecessary.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:47 +00:00
Volker Lendecke
341223a005 dbwrap_watch: Don't store in-RAM caches
The history of this file is a mess with lots of bugs. Most of the bugs
I believe are based on the cache of database contents we maintain in
struct dbwrap_watch_rec. This patch removes that cache and does all
modifications directly in the backend database.

This means we have to mess with the database format in a few more
places, but I think the format is simple enough that this does not
really hurt.

I tried for a few days to split this up into small pieces that are
easier to understand, but every time I separated out individual chunks
I found difficult to track down bugs that are all resolved in the
final code presented here. It's more lines of code, but I hope it's more
robust.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:47 +00:00
Volker Lendecke
e5c28fec6f dbwrap_watch: Simplify struct dbwrap_watched_watch_state
With the wakeup messages changed, we don't need to store what used to
be the old message in the struct dbwrap_watched_watch_state
anymore. We still need to store the key that is watched for removal of
our watch entry in the request destructor

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:47 +00:00
Volker Lendecke
a793cc2276 dbwrap_watch: Simplify the wakeup messages
The instance ID per process globally identifies a watch instance
across databases. It's not necessary to send the database ID and the
watched key across, the uin64_t instance ID is sufficient.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:47 +00:00
Volker Lendecke
7349612c59 dbwrap_watch: Add a uin64_t instance to watchers
Initially used for debugging purposes only

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:47 +00:00
Volker Lendecke
053405719a dbwrap_watch: Encapsulate watchers in "struct dbwrap_watcher"
Next patch will extend this structure

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:47 +00:00
Volker Lendecke
7cea5fec29 dbwrap_watch: Prevent two watchers per fetch_locked sequence
This can lead to very confusing bugs, and the code right now does not
deal with it well.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:47 +00:00
Volker Lendecke
79dcabfbab torture3: Test the next patch: No two waiters in one do_locked()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:47 +00:00
Volker Lendecke
e341911dda torture3: Consolidate dbwrap_watch test initialization
More lines, but less error-prone copy&paste

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:47 +00:00
Volker Lendecke
ca8c23bb42 dbwrap: Don't set rec->value in dbwrap_do_locked()
We pass that via the callback now

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:46 +00:00
Volker Lendecke
ffe4933c77 locking: Remove a call to dbwrap_record_get_value()
This makes a copy of the record values TDB_DATA.

In this case, it is okay: We only do a dbwrap_record_storev modifying
the value that we retrieved from the share_mode_lock destructor.

This patch is necessary because the next commit will make the "value"
argument to the dbwrap_do_locked() callback the only source of the
record value. Thus for a nested share_mode_do_locked() call we have to
maintain the record value explicitly.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:46 +00:00
Volker Lendecke
1dc3c16824 dbwrap: Protect against invalid db_record->value
After dbwrap_record_storev()/delete(), dbwrap_record_get_value()
information is stale. Assert on the attempt to re-fetch data after it
became stale. This can't protect against someone copying the result
from dbwrap_record_get_value() somewhere else, but it's better than
nothing.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:46 +00:00
Volker Lendecke
5291208836 vfs: Use dbwrap_do_locked() in vfs_offload_token_db_store_fsp()
Less malloc

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:46 +00:00
Volker Lendecke
dbf24bcb1b vfs: Use dbwrap_parse_record() in vfs_offload_token_db_fetch_fsp()
Less malloc

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:46 +00:00
Volker Lendecke
78c96f6995 g_lock: Pass "data" down to g_lock_trylock()
Avoid a call to dbwrap_record_get_value(), dbwrap_do_locked() already
gave us the value.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:46 +00:00
Volker Lendecke
caef82b1fe dbwrap: Pass "value" to dbwrap_do_locked() callback
I want to reduce dbwrap_record_get_value(). It makes the caller believe it can
make a copy of the TDB_DATA returned and that the value remains constant. It's
not, as you can always do a dbwrap_record_store().

This patch removes one requirement for getting the value out of a
db_record via dbwrap_record_get_value(). You can still make a copy, but from an
API perspective to me it's more obvious that "value" as a parameter to the
callback has a limited lifetime.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:46 +00:00
Volker Lendecke
a58633f64e dbwrap_tdb: Return correct error from db_tdb_storev()
Don't lose information to NT_STATUS_UNSUCCESSFUL

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:46 +00:00
Volker Lendecke
bc2fc51abe dbwrap_tdb: Avoid includes.h
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:46 +00:00
Andreas Schneider
e7f0e858e0 s3:printing: Use httpConnect2 from CUPS
This fixes deprecation warnings.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Nov 22 19:54:55 UTC 2019 on sn-devel-184
2019-11-22 19:54:55 +00:00
Andrew Bartlett
6107c79c90 build: Do not build selftest binaries for builds without --enable-selftest
Add new for_selftest option to SAMBA_BINARY() and SAMBA3_BINARY()

This allows us to be much more consistent (at least in the core Samba)
and documents clearly why the binary should not be installed.

Not modified are
 - test_lp_load
 - notifyd-tests
 - gendrandperf
 - test* from examples/libsmbclient
 - dbwrap_torture
 - split_tokens
 - locktest2
 - msgtest
 - msg_sink
 - msg_source
 - versiontest
 - rpc_open_tcp
 - test_headers

As these are not tested in selftest so any change would also be
untested.  Of course they probably should be added in a different
MR.

Also not modified (because they are not tests, nor part of the
build system) are:
 - smb2mount
 - notifydd
 - log2pacp
 - debug2html
 - smbfilter
 - destroy_netlogon_creds_cli
 - spotlight2*
 - tevent_glib_tracker

These do however appear to be untested.

For now, the source4 forked client tools are left unchanged:
 - smbclient4
 - nmblookup4

Finally, the heimdal binaries are left as install=False as
they are either part of the build system or end-user tools
that we just don't want to install.  These are however tested.

The motivation is commit like c34ec003b7
and da87fa998a, which are both totally
correct but are not needed if the selftest is not run on MacOS.

There are likely other platforms or build environments where building
our test binaries is more pain than valuable, see for example also
https://lists.samba.org/archive/samba/2019-November/227137.html

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Isaac Boukris <iboukris@samba.org>

Autobuild-User(master): Isaac Boukris <iboukris@samba.org>
Autobuild-Date(master): Fri Nov 22 11:48:59 UTC 2019 on sn-devel-184
2019-11-22 11:48:59 +00:00
Andrew Bartlett
f8947538b5 lib/fuzzing: Add oss-fuzz info to README.md
Note that Samba has not been accepted yet, but will be soon once some requirements
are addressed per:

https://github.com/google/oss-fuzz/pull/2993

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Nov 21 00:45:33 UTC 2019 on sn-devel-184
2019-11-21 00:45:33 +00:00
Andrew Bartlett
cc128c7885 lib/fuzzing/oss-fuzz: copy required libraries to the build target
This is an alternative to static linking as we do not have static source
libraries for all the things we depend on.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
2019-11-20 23:19:36 +00:00
Andrew Bartlett
4946811eb6 lib/fuzzing/oss-fuzz: Install chrpath as we use it in the build.sh script to set -rpath
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
2019-11-20 23:19:35 +00:00
Douglas Bagnall
fbb2377d51 lib/fuzzing/oss-fuzz: Add build_image.sh using Samba's bootstrap tools
Google's oss-fuzz environment is Ubuntu 16.04 based so we can
just use the maintained bootstrap system rather than a manual
package list here that will get out of date.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
Pair-programmed-by: Andrew Bartlett <abartlet@samba.org>
2019-11-20 23:19:35 +00:00
Andrew Bartlett
ec4f6f8fd3 lib/fuzzing/oss-fuzz: add stub build.sh that will not change often
This makes local development of build_samba.sh easier as it will remain in the source tree.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
2019-11-20 23:19:35 +00:00
Andrew Bartlett
f57c0238e9 lib/fuzzing/oss-fuzz: Add build_samba.sh for oss-fuzz
We work hard to put the primary logic for oss-fuzz here, and
where possible into waf, so that only a tiny stub needs to
be maintained in the Google oss-fuzz repo.

This will be called by build.sh (not copied directly because
it is too easy to forget to copy in an updated version when
doing development in the docker image).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
2019-11-20 23:19:35 +00:00
Andrew Bartlett
92ee647858 lib/fuzzing: Use --fuzz-target-ldflags if specified
This makes integration with oss-fuzz possible.  Only the fuzzer binaries should be
linked with libFuzzer, not things like asn1_compile, so this can not be done via
the global ADDITIONAL_LDFLAGS.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
2019-11-20 23:19:35 +00:00
Douglas Bagnall
cb24051097 build: Set fuzzer=True on fuzzer binaries
This ensures that the binaries are the only binaries built
when configured for fuzzing.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-11-20 23:19:35 +00:00
Andreas Schneider
9e51bb77f8 s4:rpc_server: Return the status code from dcesrv_transport_session_key()
Normally this returns the right status code, which normally is:

   NT_STATUS_NO_USER_SESSION_KEY

This is consitent with the source3 samr server implmentation.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Isaac Boukris <iboukris@samba.org>

Autobuild-User(master): Isaac Boukris <iboukris@samba.org>
Autobuild-Date(master): Wed Nov 20 22:29:35 UTC 2019 on sn-devel-184
2019-11-20 22:29:35 +00:00
Volker Lendecke
e7cab18c8b vfs: Fix a typo
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Nov 20 15:23:10 UTC 2019 on sn-devel-184
2019-11-20 15:23:10 +00:00
Andrew Bartlett
f7f92803f6 librpc: Check for NULL pointer in value() in ntlmssp_AUTHENTICATE
This allows ndrdump --validate to avoid following a NULL pointer when re-pushing
a valid but unusual input.

It also avoids an issue if the Samba server code were to provide a response
without an EncryptedRandomSessionKey.

At this stage ntlmssp.idl is not used for this, instead the packets are
generated with msrpc_gen().

Found by Douglas Bagnall using Hongfuzz and the new fuzz_ndr_X
fuzzer.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Nov 20 06:06:29 UTC 2019 on sn-devel-184
2019-11-20 06:06:29 +00:00
Andrew Bartlett
33e9021cbe selftest: Test repushing an ntlmssp AUTHENTICATE_MESSAGE
This demonstrates a bug found by Douglas Bagnall using Hongfuzz and the new fuzz_ndr_X
fuzzer where the value() evaluatuion could segfault if it was made to follow a NULL
pointer.

This also demonstrates that the --base64 mode works on file inputs.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:28 +00:00
Andrew Bartlett
ac1be895d2 selftest: Confirm that NDR bugs are fixed in DCOM code
Test input provided by Michael Hanselmann and found using Hongfuzz.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13875
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:28 +00:00
Douglas Bagnall
1aec742575 pidl: check the size of pulled arrays of arrays
We were accidentally checking the memory just past the array instead of
checking each member.

This could have led to the size of some arrays not being checked.

Found by Michael Hanselmann using Honggfuzz and an fuzzer for Samba's
NDR layer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13877

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-11-20 04:41:28 +00:00
Douglas Bagnall
536a84935c ndr_orpc: properly allocate empty DUALSTRINGARRAY
When there is no data we still need to allocate for the terminating NULL.

Found by Michael Hanselmann using Honggfuzz and an fuzzer for Samba's
NDR layer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13875

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-11-20 04:41:28 +00:00
Andrew Bartlett
b1eda993b6 selftest: Confirm that --base64-input and --input work and a PIDL bug is fixed.
The PIDL bug is in the handling of arrays of arrays.

Test input provided by Michael Hanselmann and found using Hongfuzz.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13875

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:28 +00:00
Andrew Bartlett
24fa3374e0 python: Return the stdout when also checking error codes
This will aid in checking that ndrdump behaves as expected when
failing to parse

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:28 +00:00
Andrew Bartlett
2da54d11f1 ndrdump: Still print --dump bytes after parse failure
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:28 +00:00
Andrew Bartlett
0491f17275 ndrdump: Invert sense of --stop-on-parse-failure into --print-after-parse-failure
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:28 +00:00
Andrew Bartlett
c83ad13e58 ndrdump: Return a different error code for ndr_pull() failures
This may assist in distinguishing between "runner" and "pull" failures.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:28 +00:00
Andrew Bartlett
6f0d30fd5c ndrdump: Allow for base64-encoded input in a file and on the command line
It has become customary to provide reproduction steps for fuzzing failures
in terms of an ndrdump command line.  This allows the input to be provided
as a argument or in a file rather than via base64 -d.  This makes reproducing
the issue easier as everything can be put in a plaintext bug report.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:28 +00:00
Andrew Bartlett
d4eabfb763 ndrdump: Check for input decode failures
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:28 +00:00
Andrew Bartlett
3194baaf88 ndrdump: Add const
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:27 +00:00
Andrew Bartlett
c90bc75df8 ndrdump: TALLOC_FREE() on each exit path to allow running with leak detection
This allows us to learn if there are other memory leaks not on the mem_ctx
during the processing of the packet.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:27 +00:00
Andrew Bartlett
8a6f1de67b ndrdump: print structure name when failing to setup
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:27 +00:00
Andrew Bartlett
db6c12f158 ndrdump: Fix one more NTSTATUS rather than friendly ndr message
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-11-20 04:41:27 +00:00
Andrew Bartlett
1af1ebe54a librpc: Unify packet dumping on ndr_pull() failure
There were two duplicate implementations of packet dumping just for the s4 RPC server!

This unifies them and makes them easier to find because they are not triggered
from the generated server stub any more.

The behaviour have unified on setting "dcesrv:stubs directory" and
being compiled with --enable-developer.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Samuel Cabrero <scabrero@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Nov 20 02:14:56 UTC 2019 on sn-devel-184
2019-11-20 02:14:56 +00:00
Christof Schmitt
7ae46019df vfs_gpfs: Use update_stat_ex_create_time
Make the code a bit easier to read by avoiding direct access to the
statex struct.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Tue Nov 19 22:44:53 UTC 2019 on sn-devel-184
2019-11-19 22:44:53 +00:00
Christof Schmitt
f2339fe0dd vfs_gpfs: Create fileid from filesystem metadata
MacOS SMB clients require that file ids are not quickly reused when
files are deleted and new files are created with the same name. Inode
numbers do not satisfy that requirement, as they will be quickly reused.

To address this problem, create a unique id from the available file
system specific metadata. As that id is larger than the available 64bit,
use a hash to generate a 64bit id for usage as fileid.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2019-11-19 21:19:36 +00:00
Christof Schmitt
ba5d4803ba vfs_gpfs: Use gpfs_fstat_x in vfs_gpfs_fget_dos_attributes
This is no functional change, but allows to use additional metadata
later on.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2019-11-19 21:19:36 +00:00