IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
We may want to add additional defines in the future in order to deal with
NTTIME(-1) and NTTIME(-2) coming in over the wire. They have special semantics
attached to them, -1 requests "no automatic write time updates" on a filehandle
and -2 reenables them.
We could use something like
#define SAMBA_UTIME_FREEZE (SAMBA_UTIME_OMIT - 1)
#define SAMBA_UTIME_THAW (SAMBA_UTIME_FREEZE - 1)
in the future.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=7771
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Isaac Boukris <iboukris@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Dec 5 17:13:36 UTC 2019 on sn-devel-184
We are setting ldb_debug_string() as the ldb debug function. The context
we give it is only valid as long as we're in that function.
On failure, we jump to the teardown function. The pointer for
debug_string isn't valid anymore, but the ldb debug system still points
to that address, and when we store the location of the allocated string,
we overwrite memory, in that case something from talloc and a
talloc_free() then jumps into the nirvana.
Thanks to Florian Weimer who helped debugging this.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Isaac Boukris <iboukris@samba.org>
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Isaac Boukris <iboukris@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Nov 27 12:24:16 UTC 2019 on sn-devel-184
After dbwrap_record_storev()/delete(), dbwrap_record_get_value()
information is stale. Assert on the attempt to re-fetch data after it
became stale. This can't protect against someone copying the result
from dbwrap_record_get_value() somewhere else, but it's better than
nothing.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
I want to reduce dbwrap_record_get_value(). It makes the caller believe it can
make a copy of the TDB_DATA returned and that the value remains constant. It's
not, as you can always do a dbwrap_record_store().
This patch removes one requirement for getting the value out of a
db_record via dbwrap_record_get_value(). You can still make a copy, but from an
API perspective to me it's more obvious that "value" as a parameter to the
callback has a limited lifetime.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Add new for_selftest option to SAMBA_BINARY() and SAMBA3_BINARY()
This allows us to be much more consistent (at least in the core Samba)
and documents clearly why the binary should not be installed.
Not modified are
- test_lp_load
- notifyd-tests
- gendrandperf
- test* from examples/libsmbclient
- dbwrap_torture
- split_tokens
- locktest2
- msgtest
- msg_sink
- msg_source
- versiontest
- rpc_open_tcp
- test_headers
As these are not tested in selftest so any change would also be
untested. Of course they probably should be added in a different
MR.
Also not modified (because they are not tests, nor part of the
build system) are:
- smb2mount
- notifydd
- log2pacp
- debug2html
- smbfilter
- destroy_netlogon_creds_cli
- spotlight2*
- tevent_glib_tracker
These do however appear to be untested.
For now, the source4 forked client tools are left unchanged:
- smbclient4
- nmblookup4
Finally, the heimdal binaries are left as install=False as
they are either part of the build system or end-user tools
that we just don't want to install. These are however tested.
The motivation is commit like c34ec003b7
and da87fa998a, which are both totally
correct but are not needed if the selftest is not run on MacOS.
There are likely other platforms or build environments where building
our test binaries is more pain than valuable, see for example also
https://lists.samba.org/archive/samba/2019-November/227137.html
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Isaac Boukris <iboukris@samba.org>
Autobuild-User(master): Isaac Boukris <iboukris@samba.org>
Autobuild-Date(master): Fri Nov 22 11:48:59 UTC 2019 on sn-devel-184
Note that Samba has not been accepted yet, but will be soon once some requirements
are addressed per:
https://github.com/google/oss-fuzz/pull/2993
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Nov 21 00:45:33 UTC 2019 on sn-devel-184
This is an alternative to static linking as we do not have static source
libraries for all the things we depend on.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
Google's oss-fuzz environment is Ubuntu 16.04 based so we can
just use the maintained bootstrap system rather than a manual
package list here that will get out of date.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
Pair-programmed-by: Andrew Bartlett <abartlet@samba.org>
This makes local development of build_samba.sh easier as it will remain in the source tree.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
We work hard to put the primary logic for oss-fuzz here, and
where possible into waf, so that only a tiny stub needs to
be maintained in the Google oss-fuzz repo.
This will be called by build.sh (not copied directly because
it is too easy to forget to copy in an updated version when
doing development in the docker image).
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
This makes integration with oss-fuzz possible. Only the fuzzer binaries should be
linked with libFuzzer, not things like asn1_compile, so this can not be done via
the global ADDITIONAL_LDFLAGS.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
This ensures that the binaries are the only binaries built
when configured for fuzzing.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Otherwise we have a memory leak and so fail the Google oss-fuzz check_build test.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Nov 18 21:02:52 UTC 2019 on sn-devel-184
* Upgrade waf to version 2.0.18 to fix a cross-compilation issue
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13846
Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Mon Nov 18 11:44:08 UTC 2019 on sn-devel-184
* Upgrade waf to version 2.0.18 to fix a cross-compilation issue.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13846
Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Upgrade waf to 2.0.18 to fix a cross-compilation issue
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13846
Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Almost all the callers are debug tools or developer debugging aids
and these callers clearly expect to create a new file.
Unchanged in behaviour is:
- TLS certificate creation. This already confirms the files do no exist
prior to generation.
These will now no longer overwrite the given filename
- net ads pac save
- net eventlog export
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Due to the increased default minimum protocol
level to SMB2, some users notice that
specifying smbclient -m NT1 fails with
NT_STATUS_CONNECTION_DISCONNECTED, with no SMB
traffic on the wire. Report when the max protocol
is set less than the min protocol.
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Tue Nov 12 17:52:28 UTC 2019 on sn-devel-184
For really large keys (that probably don't exist), use dump_data()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14168
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Oct 23 08:23:13 UTC 2019 on sn-devel-184
Checksums are better ignored during fuzzing, hence a flag is added to
the regfio parser to disable checksums.
Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
A temporary file is used to store the fuzzing input.
Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
This gets only registered by our tests.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Oct 14 07:41:52 UTC 2019 on sn-devel-184
The autobuild cleanup script fails with:
The tree has 3 new uncommitted files!!!
git clean -n
Would remove MEMORY:tmp_smb_creds_SK98Lv
Would remove MEMORY:tmp_smb_creds_kornU6
Would remove MEMORY:tmp_smb_creds_ljR828
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Oct 7 11:48:24 UTC 2019 on sn-devel-184
io_uring is the way to go these days, libaio was never really useful
for Samba
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Oct 4 18:18:41 UTC 2019 on sn-devel-184
Fixes:
ib/talloc/pytalloc_util.c:245:8: warning: Value stored to 'type_obj' during its initialization is never read <--[clang]
void *type_obj = talloc_check_name(ptr, type_name);
^~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
On Linux we should avoid linking everything against libpthread. Symbols
used my most application are provided by glibc and code which deals with
threads has to explicitly link against libpthread. This avoids setting
LDFLAGS=-pthread globally.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14140
Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Pair-Programmed-With: Isaac Boukris <iboukris@gmail.com>
Reviewed-by: Matthias Dieter Wallnöfer <mdw@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
This calls clock_gettime() which is available in glibc on Linux. If the
wscript in libreplace detected that librt is needed for clock_gettime()
we have to link against it.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14140
Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Pair-Programmed-With: Isaac Boukris <iboukris@gmail.com>
Reviewed-by: Matthias Dieter Wallnöfer <mdw@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
fdatasync() and clock_gettime() are provided by glibc on Linux, so there
is no need to link against librt. Checks have been added so if there are
platforms which require it are still functional.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14140
Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Pair-Programmed-With: Isaac Boukris <iboukris@gmail.com>
Reviewed-by: Matthias Dieter Wallnöfer <mdw@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
We should always use krb5_get_init_creds_opt_set_canonicalize()
and krb5_get_init_creds_opt_set_win2k() for heimdal
and expect the client principal to be changed.
There's no reason to have a different logic between MIT and Heimdal.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14124
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Use of Samba with plaintext authenticaiton is incredibly rare, even more
rare is plaintext authentication on systems without a crypt() call and
where DES based crypt() would be the right thing to do.
Remove this additional cryptographic code per our current efforts
to rely entirely on external libraries instead.
Similar to the arguments in this thread about zlib discussed on
samba-technical here:
https://lists.samba.org/archive/samba-technical/2019-May/133476.html
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Sep 19 09:28:21 UTC 2019 on sn-devel-184
Print the talloc full report into a FILE*. talloc itself provides a
very similar function, talloc_report_full(). However, that has a
slightly different output, in particular it does not print the
contents of strings, which is very handy for debugging.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
These date back to 3a9beef2b7 in 2003 and
829e72fe9c in 1998 and appear to be related
to smbwrapper.
More of these should be removed but the getdirents() test caused a timeout
on an ARM builder in Debian. It might just be a fluke but the tests are
pointless regardless.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Sep 17 13:48:18 UTC 2019 on sn-devel-184
clang is not happy when it sees another main nested inside the main
function and fails the test for prctl syscall, therefore avoid adding
implicit main() here
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Reviewed-by: Matthias Dieter Wallnöfer <mdw@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Fri Sep 13 12:33:32 UTC 2019 on sn-devel-184
Originally this parameter only set two other parameters, but never set
the original string. By setting the string as well, we can make it be
emitted by testparm -v correctly (instead of ''), and set it back as the
value for the parameter.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Probably not required, but looks safer and gives static checkers less
reason to complain about potentially uninitialized variable reads
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
At least as of 2ac9d0afa6 ctdb does not care about db prios anymore
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This hopefully fixes a lot of coverity defects like:
Control flow issues (DEADCODE)
Execution cannot reach this statement: ";".
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
* Update waf to version 2.0.17 in order to
fix the build with python 3.8.
* Other minor build fixes.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Sep 4 11:47:04 UTC 2019 on sn-devel-184
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Björn Jacke <bjacke@samba.org>
Autobuild-Date(master): Thu Aug 29 17:10:32 UTC 2019 on sn-devel-184
AIX requires the flags to be 0, we need to do those checks manually.
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
FreeBSD and AIX already set errno to ENAMETOOLONG, this is what we should map
other platforms also to to finally map to the correct NT error code also.
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The print functions used in Samba NULL terminate, but do not assume they will
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14049
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This is beyond the normal level of clarity we expect in Samba, and is of course
rudundent, but this is a complex routine that has confusing tests, some of
pointers and some of boolean state values.
This tries to make the code as clear as possible pending a more comprehensive
rewrite.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14049
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14049
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14049
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Aug 27 01:16:33 UTC 2019 on sn-devel-184
This is a macro that sets the pointer to NULL after the talloc_free()
and is part of our standard coding practices.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Thankfully this only fails if the DB is corrupt and has a duplicate record.
The test was at the wrong end of the loop, and was for the
wrong boundary condition. A write after the end of the array would
occour before the condition was hit.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13695
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fix the lmdb size test which ensures that databases > 4GiB can be
written by the lmdb backend. This test is not run as part of the normal
CI run as it exhausts the available disk on the test runners.
It was broken by changes to LDB allowing the lmdb map size to be
specified, and requiring GUID indexing by default.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Commit e45d33e9 removed the sync fallback path for the case that no
thread could be created. Remove the comment for that fallback as it does
not make sense anymore.
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Fri Aug 23 23:09:12 UTC 2019 on sn-devel-184
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Aug 22 21:24:00 UTC 2019 on sn-devel-184
If done with popt context it should be free'd.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Aug 22 01:41:37 UTC 2019 on sn-devel-184
This requires GnuTLS >= 3.6.5.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Aug 21 11:14:11 UTC 2019 on sn-devel-184
Samba will soon require GnuTLS >= 3.4.7.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Adjusted by Andrew Bartlett from an earlier more comprehensive patch by Andreas
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Direct leak of 1152 byte(s) in 1 object(s) allocated from:
#0 0x7f06393dfc08 in __interceptor_malloc (/lib64/libasan.so.5+0xefc08)
#1 0x7f06392cfd59 in __talloc_with_prefix ../../talloc.c:782
#2 0x7f06392cfd59 in _talloc_pool ../../talloc.c:837
#3 0x7f06392cfd59 in talloc_pool ../../talloc.c:859
#4 0x40b83c in test_magic_protection ../../testsuite.c:1960
#5 0x40b83c in torture_local_talloc ../../testsuite.c:2164
#6 0x402603 in main ../../testsuite_main.c:32
#7 0x7f063908a412 in __libc_start_main (/lib64/libc.so.6+0x24412)
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Matthias Dieter Wallnöfer <mdw@samba.org>
Direct leak of 100 byte(s) in 1 object(s) allocated from:
#0 0x7f06393dfc08 in __interceptor_malloc (/lib64/libasan.so.5+0xefc08)
#1 0x7f06392d1af3 in __talloc_with_prefix ../../talloc.c:782
#2 0x7f06392d1af3 in __talloc ../../talloc.c:824
#3 0x7f06392d1af3 in __talloc_strlendup ../../talloc.c:2455
#4 0x7f06392d1af3 in talloc_strdup ../../talloc.c:2471
#5 0x40b4f0 in test_rusty ../../testsuite.c:1543
#6 0x40b4f0 in torture_local_talloc ../../testsuite.c:2146
#7 0x402603 in main ../../testsuite_main.c:32
#8 0x7f063908a412 in __libc_start_main (/lib64/libc.so.6+0x24412)
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Matthias Dieter Wallnöfer <mdw@samba.org>
Direct leak of 96 byte(s) in 1 object(s) allocated from:
#0 0x7f06393dfc08 in __interceptor_malloc (/lib64/libasan.so.5+0xefc08)
#1 0x7f06392d0c45 in __talloc_with_prefix ../../talloc.c:782
#2 0x7f06392d0c45 in __talloc ../../talloc.c:824
#3 0x7f06392d0c45 in _talloc_named_const ../../talloc.c:981
#4 0x7f06392d0c45 in talloc_named_const ../../talloc.c:1748
#5 0x40901e in test_pool_nest ../../testsuite.c:1451
#6 0x40901e in torture_local_talloc ../../testsuite.c:2096
#7 0x402603 in main ../../testsuite_main.c:32
#8 0x7f063908a412 in __libc_start_main (/lib64/libc.so.6+0x24412)
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Matthias Dieter Wallnöfer <mdw@samba.org>
Direct leak of 96 byte(s) in 1 object(s) allocated from:
#0 0x7fd52c00dc08 in __interceptor_malloc (/lib64/libasan.so.5+0xefc08)
#1 0x7fd52befec45 in __talloc_with_prefix ../../talloc.c:782
#2 0x7fd52befec45 in __talloc ../../talloc.c:824
#3 0x7fd52befec45 in _talloc_named_const ../../talloc.c:981
#4 0x7fd52befec45 in talloc_named_const ../../talloc.c:1748
#5 0x4099bd in test_realloc_on_destructor_parent ../../testsuite.c:1000
#6 0x4099bd in torture_local_talloc ../../testsuite.c:2129
#7 0x402603 in main ../../testsuite_main.c:32
#8 0x7fd52bcb8412 in __libc_start_main (/lib64/libc.so.6+0x24412)
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Matthias Dieter Wallnöfer <mdw@samba.org>
* Build fixes
* Improve the performance by inlining the tdb_oob() checks
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Aug 20 14:45:41 UTC 2019 on sn-devel-184
The msg for each database record is allocated on the module context, but
never freed. The module seems like it could be a long-running context (as
the database would normally get repacked by the samba executable).
Even if it's not a proper leak, it shouldn't hurt to cleanup the memory.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Tue Aug 20 04:57:10 UTC 2019 on sn-devel-184
Firstly, with Samba AD this looks a little weird because we log the same
message 5 times (once for every partition). If we log that we're doing
this to records in different partitions, hopefully someone with a little
Samba knowledge can figure out what's going on.
Secondly, the info about what partitions are actually changing might be
useful. E.g. if we hit a fatal error repacking the 3rd partition, and
the transaction doesn't abort properly, then it would be useful to know
what partitions were repacked and which ones weren't.
There doesn't appear to be a useful name for the partition
(ldb_kv->kv_ops->name() doesn't seem any more intelligible to a user),
so just log the first record that we update. We can use that to infer
the partition database).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
The "format 0x26011968" log confused me (and I'm a developer).
We can subtract the base offset from the pack format to get a more
user-friendly number, e.g. v0 (not actually used), v1, v2, etc.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
The main reason is so that any future pack formats will continue
incrementing this number in a sequential fashion.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Store it on the repack context so that we can log a more informative
message "Repacking from format x to format y".
While this is not really a big deal currently, it could be worth
recording for potential future scenarios (i.e. supporting three or more
pack versions), where upgrades could potentially skip an intermediary
pack format version.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
LDB_DEBUG_WARNING gets logged by Samba as level 2, whereas the default
log level for Samba is 0. It's not really fair to the user to change the
format of their database on disk and potentially not tell them.
This patch adds a log with level zero (using a alias define, as this
technically isn't a fatal problem).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
In case of a failing talloc_realloc(), the only reference
to the originally allocated memory is overwritten.
Instead use a temp var until success is verified.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Matthias Dieter Wallnöfer <mdw@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add a new default backend that, while allowing mdsvc RPC and search queries from
clients, always returns no results.
Shares using this backend will behave the same way as shares on a macOS SMB
server where indexing is disabled.
This change will later also allow us to compile the Spotlight RPC service by
default which is a big step in the direction of adding tests to CI.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Currently there's only the tracker backend, but subsequent commits will add
other backends.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Aug 7 07:25:39 UTC 2019 on sn-devel-184
This adds a direct conversion hook using libicu to perform NFC <-> NFD
conversion on UTF8 strings. The defined charset strings are "UTF8-NFC" and
"UTF8-NFD", to convert from one to the other the caller calls smb_iconv_open()
with the desired source and target charsets, eg
smb_iconv_open("UTF8-NFD", "UTF8-NFC");
for converting from NFC to NFD.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Use the oLschema2ldif library functions introduced in commit
0c7c44a284 to implement a fuzzing utility.
Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
The "tiniparser_load" function is made into a wrapper for the newly
added "tiniparser_load_stream" function which accepts a FILE pointer.
This way no actual files have to be opened for fuzzing (memfd_create(2)
isn't readily available on all systems yet).
Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
LibFuzzer, Honggfuzz and other programs implement simple interfaces for
fuzzing appropriately prepared code. Samba contains quite a lot of
parsing code, often a good target for fuzzing.
With this change the build system is amended to support building fuzzing
binaries (added in later changes).
Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
When you set
in tdbtorture.c to make it more similar to locking.tdb use,
bin/tdbtorture -m -n 1 -l 100000 -s
becomes twice as fast. This is a pretty extreme case, but all other
tests that I did improve significantly as well.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This is common between both implementations of tdb_oob(). It's
faster if we don't have to dereference function pointers.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
tdb_oob() will become a public function encapsulating the pointer
dereferences.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Use iterate_range kv op to skip the index section of the database when
running a full search. Quick local testing showed 18% improved throughput
on a full search with no results on a 50k database. With more results,
improvement is smaller but still noticeable.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Aug 2 02:29:42 UTC 2019 on sn-devel-184
We have a GnuTLS DCEPRC backupkey implementation for the server and the
test. However this is only working with GnuTLS >= 3.4.7. So we need to
keep this around till we can require at least GnuTLS in a newer version.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14031
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Fixes:
lib/addns/dnsquery.c:222:10: warning: Assigned value is garbage or undefined <--[clang]
*numdcs = num_srvs; /* size_t->int */
^
1 warning generated.
Also fixes the out param being modified even on failure
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fixes:
/home/samba/samba/lib/addns/dnsmarshall.c:406:2: warning: Value stored to 'err' is never read <--[clang]
err = ERROR_DNS_NO_MEMORY;
^ ~~~~~~~~~~~~~~~~~~~
/home/samba/samba/lib/addns/dnsmarshall.c:447:3: warning: Value stored to 'err' is never read <--[clang]
err = buf->error;
^ ~~~~~~~~~~
2 warnings generated.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fixes:
lib/addns/dnssock.c:143:3: warning: The left operand of '!=' is a garbage value <--[clang]
TALLOC_FREE(conn);
^
1 warning generated.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* add pytalloc_get_name() to safely access te talloc name in Python bindings
* Use a new minor version to allow talloc updates in the 4.11 release stream
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz
Being careful is good and all, but if we don't trust the
static PyTypeObject *type = NULL;
two lines up, we need to reconsider our entire software universe.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
If the python object is not a talloc object, we will end up
with a NULL pointer. We weren't checking for that properly
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
In several places we go
talloc_get_name(pytalloc_get_ptr(py_obj))
which is a certain NULL derefernce if py_obj is not a talloc object.
This is a helper function that chooses to say "non-talloc object"
rather than crash.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
FIPS requires that a random number generator from a certified crypto
library is used.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Jul 18 01:30:20 UTC 2019 on sn-devel-184
Triggered by two coverity false positives. Loading both files into
talloc'ed memory seems inefficient to me. Rely on stdio to do proper
buffering. This removes the restriction from ae95d611: "It is meant for
small files".
This is more lines, but to me it has less implicit complexity.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Wed Jul 17 12:45:51 UTC 2019 on sn-devel-184
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Jul 10 05:48:52 UTC 2019 on sn-devel-184
There was no way to call ldb.open without evoking signal 11, so it is
unlikely anyone was using it.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Following the python/C convention for checking vs non-checking
convertors.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
In the Python/C API, conversion functions which check the types of their arguments
have names like:
double PyFloat_AsDouble(PyObject *pyfloat);
while conversion macros that don't check have names like:
PyFloat_AS_DOUBLE(pyfloat)
The pyldb_Ldb_AsLdbContext() macro looks like one of the checking functions
but it actually isn't. This has fooled us more than once. Here we fork
the macro into two -- one which performs checks and keeps the camel
case, and one with a shouty name that keeps the check-free behaviour.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This will be used by pyldb_Ldb_AsLdbContext().
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
If you try to add a dn to itself, it expands as it goes. The resulting
loop cannot end well.
It looks like this in Python:
dn = ldb.Dn(ldb.Ldb(), 'CN=y,DC=x')
dn.add_base(dn)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
SMB2_02 was available with Windows Vista.
It's time to turn SMB1 off by default.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Fixes:
lib/param/loadparm.c:2164:2: warning: Value stored to 'bRetval' is never read <--[clang]
bRetval = false;
^ ~~~~~
1 warning generated.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fixes lib/util/iov_buf.c:50:4: warning: Null pointer passed as an argument to a 'nonnull' parameter <--[clang]
memcpy(p, iov[i].iov_base, thislen);
^
1 warning generated.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
len includes space for the NUL character, so the calculation needs to
take the NUL character into account.
While touching this, drop unnecessary casts by updating format string
and update to modern debug macro.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jul 5 02:24:52 UTC 2019 on sn-devel-184
Avoid parenthesising an unsigned subtraction that can be negative and,
therefore, underflow. There is no need for the parentheses and
removing them results in an expression that is evaluated left-to-right
and can not underflow.
It isn't clear that the underflow matters. lp <= ls, so if (li - lp)
underflows then ls + (li - lp) will always overflow. This should
produce the correct answer. However, depending on this seems wrong.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* add ldb_options_get
add a function to get the options passed in ldb connect.
* add "batch_mode" option.
This options stops sub transactions being started for key value
operations. It is intended to improve the performance in batch
operations. As it bypasses the protections on operations if an
operation fails, the entire transaction will be aborted by a commit.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The previous code would override the caller with the DB size
estimate rather than allowing the caller to force the bigger size.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Test the the ldb "batch_mode" option sets batch mode operation.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
When performing a join the overhead of the sub transactions protecting
key value operations becomes significant. This commit adds a new
"batch_mode" option that disables the sub transactions around key value
operations.
The operation level index cache is also disabled, which means the
overall transaction level index cache can become inconsistent if an
operation fails. To protect against this and other possible on disk
inconsistencies, if any operation fails during a batch_mode
transaction the commit will fail and transaction will be rolled back.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is needed for modules to access the ldb->options array, as this in in ldb_private.h
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
ldb_tdb can be a module, but the test is actually looking for ltdb_err_map() in
ldb_tdb_err_map.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Jul 4 03:51:58 UTC 2019 on sn-devel-184
Flag is used to enforce binary encoded attribute values per attribute.
Signed-off-by: Björn Baumbach <bb@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The tdb routines return 0/-1 and return the specific error via
tdb_error(). server_id_db_prune_name() is expected to return an errno,
not 0/-1.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Wed Jul 3 10:51:32 UTC 2019 on sn-devel-184
commit: 8b92539997 introduced a leak
by replacing mktemp with mkstemp (mkstemp returns an open fd)
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Allow the lmdb map size to be specified in the ldb option
"lmdb_env_size".
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* copy the ldb_options passed to ldb_connect onto the ldb_context,
making them more generally available.
* fix index buffering.
As a performance enhancement the indexes are cached in memory during a
transaction, and written to disk as part of the prepare commit. The
indexes could become corrupt in the event of a failed operation.
* fix read beyond buffer
Calling the "ldb_parse_tree" function with a filter consisting of
exactly a single space (" ") would trigger a read beyond the input
buffer.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13900
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Copy the options supplied to to ldb_connect, and place them on the
ldb_context. This allows backend options i.e. lmbd map size to be passed
cleanly from the callers.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Fix the ubsan warning
lib/tdb/common/tdb.c:184:9: runtime error: null pointer passed as
argument 2, which is declared to never be null"
memcmp call now guarded by a length check.
memcmp returns zero when called with a zero length parameter.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Mon Jul 1 14:50:54 UTC 2019 on sn-devel-184
Fixes:
lib/krb5_wrap/krb5_samba.c:3241:3: warning: Null pointer passed as an argument to a 'nonnull' parameter <--[clang]
memcpy(gss_cksum + 28, in_data->data, orig_length);
^
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fixes:
lib/krb5_wrap/krb5_samba.c:2012:2: warning: Call to function 'mktemp' is insecure as it always creates or uses insecure temporary file. Use 'mkstemp' instead <--[clang]
mktemp(tmp_name);
^~~~~~
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fixes:
lib/util/server_id_db.c:181:3: warning: Value stored to 'ret' is never read <--[clang]
ret = tdb_store(tdb, key, talloc_tdb_data(ids), TDB_MODIFY);
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fixes:
lib/ldb/common/ldb.c:1091:3: warning: Value stored to 'ret' is never read <--[clang]
ret = 0;
^ ~
1 warning generated.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fixes:
lib/tdb/common/rescue.c:299:2: warning: Null pointer passed as an argument to a 'nonnull' parameter <--[clang]
qsort(found.arr, found.num, sizeof(found.arr[0]), cmp_key);
^ ~~~~~~~~~
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fixes:
lib/util/tfork.c:260:3: warning: Value stored to 'ret' is never read <--[clang]
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
One case needs a variable declared, so it can be compared to -1 and
then cast to size_t for comparison.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Jul 1 08:00:29 UTC 2019 on sn-devel-184
I may be missing something subtle but I can't see a reason for
declaring these as ssize_t.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andreas Schneider <asn@samba.org>
The internal string conversion routines smb_strtoul(l) return
an error if the provided string could not be converted to an integer.
This can be the case if the string is empty or if it starts with non-numeric
characters which cannot be converted.
The standard C library, however, does allow this and simply returns 0 as the
converted value.
If this behaviour is wanted, it can be enabled by using
the "SMB_STR_ALLOW_NO_CONVERSION" flag.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Sun Jun 30 12:47:24 UTC 2019 on sn-devel-184
The standard string to integer conversion routines stop at the first
character which cannot be converted to a number.
However, if such a character is found, it is not considered an error.
With the flag "SMB_STR_FULL_STR_CONV" enabled, an error will be returned
if the string could not be converted entirely.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
The standard string to integer conversion routines allow strings
with a leading "-" to indicate a negative number.
However, the returned value is always an unsigned value representing
the bit-pattern of this negative value.
Typically, this behaviour is NOT wanted and therefore the standard
behavior of the internal smb_strtoul(l) return an erros in such situations.
It can be enabled though by using the flag SMB_STR_ALLOW_NEGATIVE.
This test verifies the correct processing.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
Adding two addtl. flags SAMBA_STR_ALLOW_NO_CONVERSION and SAMBA_STR_GLIBC_STANDARD
for the wrappers strtoul_err() and strtoull_err() providing the possibility
to get standard glibc behaviour for string to integer conversion.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
Some callers want to have the entire string being used for a
string to integer conversion, otherwise flag an error.
This is possible by providing the SAMBA_STR_FULL_STR_CONV flag.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
In order to still be bisectable when changing the API for the wrappers
strtoul_err() and strtoull_err() some preparations need to be performed.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
The following flags are defined intially
SMB_STR_STANDARD # raise error if negative or non-numeric
SMB_STR_ALLOW_NEGATIVE # allow strings with a leading "-"
SMB_STR_FULL_STR_CONV # entire string must be converted
SMB_STR_ALLOW_NO_CONVERSION # allow empty strings or non-numeric
SMB_STR_GLIBC_STANDARD # act exactly as the standard glibc strtoul
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
The standard string to integer conversion routines return zero
if a string was to be converted which did not reflect a number.
It is not flag'ed as an error.
The wrapper functions strtoul_err() and strtoull_err() are expected
to exactly do this.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
Verify that a string representing a negative number is throwing an error.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
The wrapper functions strtoul_err() and strtoull_err() trigger
other functions/routines which modify errno.
However, callers of those wrapper functions expect errno to be unchanged.
This test verifies the expectation.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
This allows Samba to use GnuTLS for drsuapi_{en,de}crypt_attribute_value()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
This will avoid duplicated code as we convert arcfour_crypt_blob() into
direct GnuTLS calls
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Fixes:
lib/param/loadparm.c:3325:36: warning: Access to field 'szService' results in a dereference of a null pointer (loaded from variable 'service') <--[clang]
return lpcfg_string((const char *)service->szService)
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Wed Jun 26 11:53:08 UTC 2019 on sn-devel-184
Fixes:
lib/dbwrap/dbwrap.c:645:4: warning: Null pointer passed as an argument to a 'nonnull' parameter <--[clang]
memcpy(p, dbufs[i].dptr, thislen);
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fixes:
lib/dbwrap/dbwrap.c:533:4: warning: 2nd function call argument is an uninitialized value <--[clang]
dbwrap_lock_order_unlock(db, lockptr);
^
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fixes:
lib/util/util_tdb.c:385:11: warning: Value stored to 'result' during its initialization is never read <--[clang]
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fixes:
lib/tdb/common/transaction.c:354:2: warning: Null pointer passed as an argument to a 'nonnull' parameter <--[clang]
memcpy(tdb->transaction->blocks[blk] + off, buf, len);
&
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Fixes:
lib/tdb/common/lock.c:933:6: warning: Dereference of null pointer <--[clang]
if (tdb->allrecord_lock.count) {
^~~~~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This came about because in py2 we had to check for strings and unicode.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Mon Jun 24 18:48:53 UTC 2019 on sn-devel-184
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jun 21 05:45:02 UTC 2019 on sn-devel-184
As a performance enhancement the key value layer maintains a cache of
the index records, which is written to disk as part of a prepare commit.
This patch adds an extra cache at the operation layer to ensure that the
cached indexes remain consistent in the event of an operation failing.
Add test to test for index corruption in a failed modify.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
The callers will soon have two possible parents for this pointer, so
we need to remove this check, which was added out of caution given
the rather strange pattern of putting an active memory pointer into a
TDB (as a hash map).
That is, the only callers that did call this with "true" would
have to call this with "false", so just remove the complexity.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add test exercising the sub/nested transactions wrapping the key value
operations.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Use the nested transaction support added to the key value back ends to
make key value operations atomic. This will ensure that rename
operation failures, which delete the original record and add a new
record, leave the database in a consistent state.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add limited nested transaction support to the back ends to make the key value
operations atomic (for those back ends that support nested transactions).
Note: that only the lmdb backend currently supports nested transactions.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Remove the "DeprecationWarning: Please use assertEqual instead."
warnings from api.py
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add a test to document that ldb does not currently support nested
transactions.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
A duplicate define warning is better than not defining this at all.
Similar to a patch Torsten Werner submitted to the Samba Bugzilla in
2005.
Not tested on HP-UX
reverts fc84e916f6 that was still present in
the new copy of nis.h created in c29d087e1e
BUG: https://bugzilla.samba.org/show_bug.cgi?id=2406
BUG: https://bugzilla.samba.org/show_bug.cgi?id=2140
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Jun 18 06:47:05 UTC 2019 on sn-devel-184
Fixes csbuild error.
Error: COMPILER_WARNING:
lib/ldb/ldb_key_value/ldb_kv_search.c: scope_hint: In
function ‘search_func’ lib/ldb/ldb_key_value/ldb_kv_search.c:308:47:
warning: unused parameter ‘ldb_kv’ [-Wunused-parameter]
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Jun 13 08:27:12 UTC 2019 on sn-devel-184
Fixes csbuild error.
Error: COMPILER_WARNING:
lib/ldb/ldb_key_value/ldb_kv_index.c: scope_hint: In function
‘ldb_kv_index_add1’
lib/ldb/ldb_key_value/ldb_kv_index.c:2601:15: warning: comparison of
integer expressions of different signedness: ‘int’ and ‘unsigned int’
[-Wsign-compare]
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Fixes csbuild error.
Error: COMPILER_WARNING:
lib/ldb/ldb_key_value/ldb_kv_index.c: scope_hint: In function
‘ldb_kv_index_traverse_store’
lib/ldb/ldb_key_value/ldb_kv_index.c:779:60: warning: unused parameter
‘tdb’ [-Wunused-parameter]
By tagging the tdb_context parameter in ldb_kv_index_traverse_store as
_UNUSED_
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Fixes csbuild error.
Error: COMPILER_WARNING:
lib/ldb/ldb_key_value/ldb_kv_index.c: scope_hint: In function
‘ldb_kv_key_dn_from_idx’
lib/ldb/ldb_key_value/ldb_kv_index.c:545:15: warning: comparison of
integer expressions of different signedness: ‘int’ and ‘unsigned int’
[-Wsign-compare]
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Add lib/util/attr.h to the ldb distribution, to make the _UNUSED_ macro
available.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Fixes
lib/tdb/common/freelistcheck.c:96:3: warning: Value stored to 'last_ptr' is never read <--[clang]
last_ptr = rec_ptr;
^ ~~~~~~~
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer gary@catalyst.net.nz
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Tue Jun 11 13:31:01 UTC 2019 on sn-devel-184
Fixes:
lib/util/rbtree.c:170:8: warning: Access to field 'rb_parent_color' results in a dereference of a null pointer (loaded from variable 'other') <--[clang]
We could avoid accessing the NULL pointer but previously the code would
have crashed here. Given this is a rbtree probably better to preserve the
fatal nature of encountering a NULL pointer here while satisfying the static
checker.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer gary@catalyst.net.nz
Fixes:
lib/util/ms_fnmatch.c:75:8: warning: Access to field 'predot' results in a dereference of a null pointer (loaded from variable 'max_n') <--[clang]
if (max_n->predot && max_n->predot <= n) {
^
lib/util/ms_fnmatch.c:91:8: warning: Access to field 'predot' results in a dereference of a null pointer (loaded from variable 'max_n') <--[clang]
if (max_n->predot && max_n->predot <= n) {
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer gary@catalyst.net.nz
Fixes:
lib/util/charset/convert_string.c:301:5: warning: Value stored to 'reason' is never read <--[clang]
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer gary@catalyst.net.nz
Fixes:
lib/util/debug.c:705:7: warning: Null pointer passed as an argument to a 'nonnull' parameter <--[clang]
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer gary@catalyst.net.nz
Fixes:
lib/tevent/tevent_wrapper.c:295:3: warning: Access to field 'next' results in a dereference of a null pointer (loaded from field 'prev') <--[clang]
Additionally fix similar instance of the same macro
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer gary@catalyst.net.nz
Fixes:
lib/tevent/tevent_wrapper.c:213:6: warning: Access to field 'destroyed' results in a dereference of a null pointer (loaded from variable 'glue') <--[clang]
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer gary@catalyst.net.nz
Fixes
lib/util/tests/file.c:153:2: warning: Value stored to 'lines' is never read <--[clang]
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Calling the "ldb_parse_tree" function with a filter consisting of
exactly a single space (" ") would trigger a read beyond the input
buffer. A unittest is included.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13900
Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>
* Default pack format changed to version 2 for GUID Indexed databases
(essentially Samba's sam.ldb). This means faster unpacking for records
with lots of attributes.
* Enforce link between all new database features, following toggling of
GUID indexing. If user toggles GUID indexing on/off, target pack format
is changed. Likewise the special ORDERED_INTEGER index format is used
(when GUID indexing on) or behaves like INTEGER (when GUID indexing off).
* Database repacked if packing format not as expected.
* In the case of MDB, since GUID indexing is mandatory, feature toggling
provided by pack_format_override LDB option.
* Check for errors from ldb_unpack_data() in ldb_tdb
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 29 05:56:59 UTC 2019 on sn-devel-184
For TDB databases, toggling GUID indexing mode will also toggle
pack format version 2. This provides a convenient downgrade path for
Samba databases, but the process doesn't work for MDB databases because
GUID indexing cannot be disabled when the MDB backend is used. This patch
addresses that corner case by providing support for a pack_format_override
option which will force the database to use pack format version 2.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Python test to confirm that after enabling or disabling GUID indexing,
the database is repacked on the next transaction with V1 if GUID
indexing was disabled, or V2 if it was enabled.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
VERY IMPORTANT PATCH
Now that we have a new packing format, we need to enable it by repacking
the database. We've decided to link all new database features together,
so once GUID indexing is enabled, the database will be repacked with
version 2 format. Repacking is done following the same iterate pattern as
reindexing.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Only enable ORDERED_INTEGER and index_format_fn functionality if GUID
indexing is enabled. This is in line with the rest of ldb_kv which binds
the new ORDERED_INTEGER to GUID indexed databases, and allows a practical
way to create the old index format (by disabling the GUID index).
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
We need to check for the errors given by ldb_unpack() et al by preserving
the error code from kv_ctx->parser() called by tdb_parse_record() in
ltdb_parse_record().
Otherwise we will silently accept corrupt records and segfault later.
Likewise new pack formats will confuse the parser but not be
detected except by the incomplete struct ldb_message.
With this patch, the user will see a message like:
Invalid data for index DN=@BASEINFO
Failed to connect to 'st/ad_dc/private/sam.ldb' with backend 'tdb': Unable to load ltdb cache records for backend 'ldb_tdb backend'
Failed to connect to st/ad_dc/private/sam.ldb - Unable to load ltdb cache records for backend 'ldb_tdb backend'
This can be refined in the future by a specific check for
pack format versions in a higher caller, but this much is
needed regardless to detect corrupt records.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13959
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Parse errors aren't passed up correctly by the tdb backend. This
patch modifies a test to expose the issue, next patch will fix it.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13959
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
In some situations we use DEBUGADDC() in order to print out content
without a related debug header line.
This is important with the new per class logfile with:
log level = 1 dsdb_json_audit:10@/var/log/samba/log.dsdb_json_audit
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13915
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
dbghdrclass() sets the global 'current_msg_class' and for that
DEBUGC() should pass the given dbgc_class instead of the per file
DBGC_CLASS.
This is important with the new per class logfile with:
log level = 1 dsdb_audit:10@/var/log/samba/log.dsdb_audit
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13915
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Fix ubsan warning null pointer passed as argument 2 when the source
pointer is NULL. The calls to memcpy are now guarded by an
if (len > 0)
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Mon May 27 01:29:48 UTC 2019 on sn-devel-184
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13961
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat May 25 00:05:25 UTC 2019 on sn-devel-184
All code now uses sys_popenv() which is much
harder to use incorrectly.
Remove the extract_args() function that was the
cause of possible issues.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13964
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Not yet used.
Duplicate code to file_pload() except uses vectored
argument list. file_pload() will be removed once all
callers are converted.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13964
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Add sys_popenv(char * const argl[]) that uses a NULL
terminated vector array of args. Change sys_popen() to
split up its command string and call sys_popenv().
Once all callers are converted to sys_popenv() we
can remove sys_popen().
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13964
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Ensure that the referrals returned in a search request use the same
scheme as the request, i.e. referrals recieved via ldap are prefixed
with "ldap://" and those over ldaps are prefixed with "ldaps://"
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12478
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri May 24 05:12:14 UTC 2019 on sn-devel-184
README is far from a complete, good and accurate document, but what's
in there should at least not have obvious errors.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
For testing we need to know the actual KV level key of records and each
record's pack format version. This patch makes ldbdump add comments with
that info. We will parse it out in python tests.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 22 05:58:17 UTC 2019 on sn-devel-184
In future commits we'll be adding more logging to LDB, which breaks the
ldb_key_value_test suite. By removing the debug handler, a bug
involving an expired debug_string variable being written to is avoided.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
We want to reuse the reindex context struct for repacking, but it has an
unnecessary module pointer on it. Turns out the existing code doesn't
need it either, so this patch deletes the pointer.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* Checking pack format is version 1 and erroring if not (will change soon)
* Pack format routines for unpack and pack version 2 (but not used)
* Test fixes for issues caused by upcoming repack functionality for upgrade
* Making ldbdump print out pack format info and keys so we have low level visibility for testing in python
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Pack function for new pack format with values separated from other data
so that while unpacking, the value section (which is probably large)
doesn't have to be loaded into cache/memory.
The new format is disabled for now.
Two tests are added that operate on a detailed binary breakdown of the
new format.
NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Unpack function for new pack format with values separated from other
data so that while unpacking, the value section (which is probably large)
doesn't have to be loaded into cache/memory.
Additionally, width of length field can now vary per-element to save space.
The old unpack routine is still present and is called if the old pack
format version number is found.
LDB torture suite is modified to run relevant tests on both old and new
pack format.
NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Since we're about to introduce a new packing format, it's a good time to
improve our code style and change some magic numbers into explicit
constants.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Replacing push and pull functions (which may cause issues with Undefined
Sanitizer) with Andreas Schneider's excellent macros which are a work in
progress and not yet merged into master. Once his work is upstream, I'll
rebase and change this code to import his headers.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
We will be adding a new packing format in forthcoming commits and there
may be more versions in the future. We need to make sure the database
contains records in a format we know how to read and write.
Done by fetching the @BASEINFO record and reading the first 4
bytes which contain the packing format version.
NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue May 21 01:18:08 UTC 2019 on sn-devel-184
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri May 17 20:44:36 UTC 2019 on sn-devel-184
* Do not propogate any errors from tdb_repack() to to tdb_transaction_commit()
The repack may fail due to lock ordering or memory limits, but the original data has
been written so the call must succeed. (bug 13952)
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri May 17 08:21:52 UTC 2019 on sn-devel-184
The call to tdb_repack() inside tdb_transaction_commit()
is an optimization, not part of the transaction itself,
so failing due to lock or other errors isn't a fatal error
that should cause the caller to think the transaction was
a failure by returning -1.
The tdb transaction itself has finished and been committed
onto stable storage via fsync and all locks released at the
point tdb_repack() is called.
tdb_repack() is only called here as it's a convenient point
to attempt to reduce tdb fragmentation without having to add
a timer call to repack in all users of tdb.
This causes lock ordering issues in Samba, showing up as:
ldb: ltdb: tdb(../private/sam.ldb.d/DC=SAMBA2008R2,DC=EXAMPLE,DC=COM.ldb): tdb_transaction_prepare_commit: failed to upgrade hash locks: Locking error
This is because Samba has multiple tdb databases open, and the lock order between them
is important.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13952
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Returning a non-zero value from a function with bool as return value is
the same as returning true. Change the return value to false if
sigprocmask or pthread_sigmask fails to indicate failure.
Detected with the help of cppcheck.
Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu May 16 19:08:29 UTC 2019 on sn-devel-184
To avoid warning above produced by using
-Wcast-function-type we;
+ ensure PyCFunctions of type METH_VARARGS do not declare
unused and problematic kargs param.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
To avoid warning above produced by using
-Wcast-function-type we;
+ ensure PyCFunctions of type METH_NOARGS defined dummy arg
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
To avoid warning above produced by using
-Wcast-function-type we;
+ ensure PyCFunctions of type METH_NOARGS defined dummy arg
+ ensure PyCFunctions of type METH_KEYWORDS use PY_DISCARD_FUNC_SIG
macro
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
To avoid warning above produced by using
-Wcast-function-type we;
+ ensure PyCFunctions of type METH_NOARGS defined dummy arg
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
squash 'cast between incompatible function types' warning
To avoid warning above produced by using
-Wcast-function-type we;
+ ensure PyCFunctions of type METH_NOARGS defined dummy arg
+ ensure PyCFunctions of type METH_KEYWORDS use PY_DISCARD_FUNC_SIG
macro
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
squash 'cast between incompatible function types' warning
To avoid warning above produced by using
-Wcast-function-type we;
+ ensure PyCFunctions of type METH_NOARGS defined dummy arg
+ ensure PyCFunctions of type METH_KEYWORDS use PY_DISCARD_FUNC_SIG
macro
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
It seems very likely that our clever attempts to dynamically realloc
the output buffer were never triggered. Two lines of reasoning lead to
this conclusion:
1. We allocate 3 * srclen to start with, but no conversion we use will
more than that. To be precise, from 8-bit charsets we will only deal
with codepoints in the Unicode basic multilingual plane (up to 0xFFFF).
These can all be expressed as 3 or fewer utf-8 bytes. In UTF16 they
are naturally 2 bytes, while in the DOS codes they are 1 byte.
We have checked the code tables, and can not find a plausible
(e.g. not EBCDIC) DOS code page or unix charset that is outside
this range. Clients cannot chose the code page, the only code
pages we will use come from 'unix charset' and 'dos charset'
smb.conf parameters.
Therefore the worst that can possibly happen is we expand 1 byte into 3
(specifically, when converting some e.g. CP850 codepoints to UTF-8).
2. If the reallocation was ever used, the results would have been
catastrophically wrong, as the input pointer was not reset.
Therefore we skip the complication of the goto loop and let E2BIG be
just another impossible error to report.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
iconv() advances the inbuf pointer; if we decide to realloc and re-iconv,
we need to reset inbuf to the source string
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Now, if destlen were SIZE_MAX - 1, destlen * 2 would wrap to SIZE_MAX - 3,
which makes (destlen * 2 + 2) == SIZE_MAX - 1, the same number again.
So we need the <= comparison in this case.
As things stand, it is not actually possible for destlen to be
SIZE_MAX (because it is always an even number after the first round,
and the first round is constrained to be < SIZE_MAX / 2, but *if*
destlen was SIZE_MAX, destlen * 2 + 2 would be 0, so that case is OK.
Similarly the SIZE_MAX - 2 and smaller cases were covered by the
original formula.
We add the comment for people who are wondering WTF is going on with
all this destlen manipulation.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
In failure cases the destination string pointer is set to NULL, but
the size is not changed. Some callers have not been checking the
return value and passing the destination pointer and uninitialised
length onto other functions. We can curse and blame those callers, but
let's also keep them safe.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Rework pack routines to remove unused features:
- ldb_unpack_data_only_attr_list_flags() is removed
- LDB_UNPACK_DATA_FLAG_NO_DATA_ALLOC is now always implied
This improves the unpack performace significantly.
* Improve search performance via new internal flag
LDB_UNPACK_DATA_FLAG_READ_LOCKED which removes a memdup
* Improve search performance during full scan by ignoring
index records early.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Unpack functions currently take an attribute list to restrict the set of
attributes to be returned in the constructed message. This
functionality is never used and complicates implementation of
forthcoming new pack format. This patch removes that functionality.
Using the unpack function then filtering the result turns
out not to be any slower.
NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This patch moves ldb_kv's filter code into the pack code to replace
'only attr list' functionality which will be removed in forthcoming
commit. Unpacking data then filtering the result is not any slower
than the removed 'only attr list' approach.
'only attr list' test repurposed to test unpack -> filter flow.
NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Optimising filter_attrs by removing msg and dn allocation/copying. The
caller can construct the msg and possibly steal the dn.
Also giving the function an ldb for future use.
NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Making unpack flag LDB_UNPACK_DATA_FLAG_NO_DATA_ALLOC required
behaviour, since allocating data during unpack is slow and unnecessary
in all current usages. In any future unpack usage, if editing of
returned memory is required, some function that duplicates the message
should be used, such as one of the filter_attrs functions, or msg_copy.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This more modern routine allocates a nice talloc tree.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
The DN is now exploded so as to improve other aspects of the search handling.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Previously we half-heartedly checked one end.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
ldb_msg_new() is currently the same as talloc_zero(), but it might
not always be.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
It is unlikely to be NULL, since we're in the callback.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This is the way we do it right now, avoid confusion why "tmp" might be
needed
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
IIRC there are platforms that don't like memcpy() with len=0.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Found by Joe Guo during preperation for automated code coverage output.
In order to allow the Makefile wrapper to work we need to rename the
test directory to tests.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This is built close to the code it tests in lib/replace/wscript_build as
replace_testsuite.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Increase the debug line buffer to 4KiB, the existing size of 1KiB is too
small for dsdbChange JSON audit messages. These messages were then
split across multipe lines causing issues in log ingestion tools
expecting single line messages.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13902
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Mon May 6 01:18:22 UTC 2019 on sn-devel-184
The code has a default of one week (10080 minutes) if the parameter is
set to 0. Make this the public default of the parameter, instead of
hiding it in the code. This change also has the code match the
documentation that setting this parameter to 0 disables the check.
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We don't want to see this:
python3 -c "import sys
sys.path.insert(0, 'bin/python')
import ldb
m = ldb.Message()
e = ldb.MessageElement('q')
try:
m.add(e)
except ldb.LdbError:
pass
print(m)
"
Segmentation fault (core dumped)
instead we want this:
Traceback (most recent call last):
File "<string>", line 7, in <module>
ValueError: The element has no name
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Tue Apr 23 19:03:35 UTC 2019 on sn-devel-144
Commit c89a33a07a 'debug: Use backends instead of explicitly logging to
syslog or file' introduced a regression where early startup failures (e.g.
unable to connect to CTDB) are no longer logged because the debug subsystem
is not yet fully initialized. Enable logging again with reasonable defaults
when reopen_logs() is called and the parameter file is not yet parsed.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13904
Signed-off-by: Ralph Wuerthner <ralph.wuerthner@de.ibm.com>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Thu Apr 18 22:21:15 UTC 2019 on sn-devel-144
This allows the vfs_glusterfs_fuse build to complete on AIX.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13872
Guenther
Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Tue Apr 16 15:14:50 UTC 2019 on sn-devel-144
We have the ldb_kv in the caller, just fill it into the context and
so avoid the cost of the talloc_get_type().
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
The new string conversion wrappers detect and flag errors
which occured during the string to integer conversion.
Those modifications required an update of the callees
error checks.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
The standard string conversion routines convert a "non-number string"
to zero and do not flag an error.
This is changed now by returning EINVAL if no conversion occured.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Apr 11 05:25:02 UTC 2019 on sn-devel-144
This makes LDAP bind significantly faster in the case of having many
members, due to large size of these records (with tens of thousands of
member links). During the nested group calculation, you are only
interested in memberOf not the member links.
(We add a bit-field to determine whether or not the backend actually
supports pointing into database memory. For some reason TDB pointers
aren't stable, so for now we set this option just on LMDB backends.)
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
@ records like @IDXLIST are only available via a base search on the specific name
but the method by which they were excluded was expensive, after the unpack the
DN is exploded and ldb_match_msg_error() would reject it for failing to match the
scope.
This uses the fact that @ records have the DN=@ prefix on their TDB/LMDB key
to quickly exclude them from consideration.
Based on analysis by Garming Sam.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13893
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Apr 10 06:23:39 UTC 2019 on sn-devel-144
* Version bump for adding index_format_fn to the schema syntax structure.
* Range index support added, allowing <= and >= operations to be indexed
* Improved reindex performance by setting the in-memory TDB hash size correctly
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
We need to make sure that duplicates are correctly returned (uSNChanged
for instance is UNIQUE but, we should be able to index on attributes
which are not unique).
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Testing max, min and negative values for indexed 32 and 64 bit types.
This has to be done in two different files because the 64 bit type is
LDB_SYNTAX_INTEGER which is implemented at the ldb level, while the 32
bit is added in the ldb-samba module. Schema syntax binding added for
ldb-samba.
We also need to make sure that full scans are not invoked for LMDB.
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Adding ordered integer proto schema handling in kv index cache. This
allows ordered 64 bit integers to be used in cached fields like
@ATTRIBUTES
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We had to devise our own scheme for writing integers in a human readable
format which also sorted correctly numerically. This might look a bit
confusing to outsiders, so here's a large comment as a peace offering.
Pair-programmed-with: Tim Beale <timbeale@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Activating <= and >= mdb indexing in samba for int32 and int64 attributes by:
1. Adding index_format_fn to LDB_SYNTAX_SAMBA_INT32 in ldb_samba
2. Cloning the 64bit LDB_SYNTAX_INTEGER type as LDB_SYNTAX_ORDERED_INTEGER
3. Adding index_format_fn to the new type
4. Modifying LargeInteger use the new type in samba schema
5. Bumping the index version to trigger reindexing
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The reason we needed it in the first place was that the original
canonicalize is being used for non-index functions and it never produced
the right order originally (at least for integers).
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
It makes no difference in our standard case because \0 will always go
before any value for our index_format_fn, but this is better for
correctness (in case we do mess up our NUL terminations elsewhere).
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Full implementation of <= and >= indexed searching using iterate_range
backend operation. Adds index_format_fn to ldb_schema_syntax so
requires an ABI version bump. The function must be provided for any
type for which <= and >= indexing is required, and must return a
lexicographically ordered canonicalization of a value. This causes
index entries to be written in correct order to the database, so
iterate_range on the index DNs can be used.
ldb_kv_index_key is modified to return an index DN with attribute name
but without value if an empty value is provided. This is needed for
constructing keys that match the beginning or end of an index DN range.
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Adding iterate_range to LDB API and implementing in LMDB. This
operation takes a start_key and end_key and returns all records between
the two, inclusive of both. This will be used to implementing indexing
for <= and >= expressions.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
With the fixed accounting of talloc objects, the default cache size
needs to increase. The exact increase required depends on the workloads,
going form 256k to 512k seems like a reasonable guess.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13865
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
With memcache_add_talloc, the talloc object becomes part of the pool and
the memcache_element stores a pointer to the talloc object. The
size of the the talloc object was not used when tracking the used space,
allowing the cache to grow larger than defined in the memcache_init
call.
Fix this by adding the size of the talloc object to the used space.
Also record the initial size of the talloc object for proper adjustment
of the used space in the cache later. This is in case the size of the
talloc object is modified while being owned by the cache (e.g.
allocating talloc child objects). This should never happen, but better
be safe than ending up with a broken cache usage counter.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13865
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This allows extending the additional data stored for talloced objects
later.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13865
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
If ldb_kv_filter_attrs() fails, we don't know that the dn of filtered_msg
is OK.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Apr 5 05:46:55 UTC 2019 on sn-devel-144
Set the default index cache from the passed option
"transaction_index_cache_size" on open. This allows the default cache
size to be overridden when processing large transactions i.e. joining a
large domain.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Set the index cache size to the number of records in the databse when
reindexing.
This significantly improves reindex performance. For a domain with
100,000 users the reindex times are reduced from 17 minutes to 45
seconds.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add the get_size method to the ldb_key_value layer, this will allow the
reindexing code to get an estimate of the number of records in the
database.
The lmdb backend returns an accurate count of the number of records in
the database withe the mdb_env_stat call.
The tdb backend does not provide a low cost method to determine the
number of records on the database. It does provide a tdb_summary call
however this this walks the entire database.
So for tdb we use the map size divided by 500, this over estimates the counts
for small domains, but the extra memory allocated for the cache should
not be significant.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pass the index cache size to ldb_kv_index_transaction_start. This will
allow it to be set for reindex and join operations, where the current
defaults result in a significant performance penalty on large databases.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
