IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
We had to devise our own scheme for writing integers in a human readable
format which also sorted correctly numerically. This might look a bit
confusing to outsiders, so here's a large comment as a peace offering.
Pair-programmed-with: Tim Beale <timbeale@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Activating <= and >= mdb indexing in samba for int32 and int64 attributes by:
1. Adding index_format_fn to LDB_SYNTAX_SAMBA_INT32 in ldb_samba
2. Cloning the 64bit LDB_SYNTAX_INTEGER type as LDB_SYNTAX_ORDERED_INTEGER
3. Adding index_format_fn to the new type
4. Modifying LargeInteger use the new type in samba schema
5. Bumping the index version to trigger reindexing
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The reason we needed it in the first place was that the original
canonicalize is being used for non-index functions and it never produced
the right order originally (at least for integers).
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
It makes no difference in our standard case because \0 will always go
before any value for our index_format_fn, but this is better for
correctness (in case we do mess up our NUL terminations elsewhere).
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Full implementation of <= and >= indexed searching using iterate_range
backend operation. Adds index_format_fn to ldb_schema_syntax so
requires an ABI version bump. The function must be provided for any
type for which <= and >= indexing is required, and must return a
lexicographically ordered canonicalization of a value. This causes
index entries to be written in correct order to the database, so
iterate_range on the index DNs can be used.
ldb_kv_index_key is modified to return an index DN with attribute name
but without value if an empty value is provided. This is needed for
constructing keys that match the beginning or end of an index DN range.
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Adding iterate_range to LDB API and implementing in LMDB. This
operation takes a start_key and end_key and returns all records between
the two, inclusive of both. This will be used to implementing indexing
for <= and >= expressions.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
With the fixed accounting of talloc objects, the default cache size
needs to increase. The exact increase required depends on the workloads,
going form 256k to 512k seems like a reasonable guess.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13865
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
With memcache_add_talloc, the talloc object becomes part of the pool and
the memcache_element stores a pointer to the talloc object. The
size of the the talloc object was not used when tracking the used space,
allowing the cache to grow larger than defined in the memcache_init
call.
Fix this by adding the size of the talloc object to the used space.
Also record the initial size of the talloc object for proper adjustment
of the used space in the cache later. This is in case the size of the
talloc object is modified while being owned by the cache (e.g.
allocating talloc child objects). This should never happen, but better
be safe than ending up with a broken cache usage counter.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13865
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This allows extending the additional data stored for talloced objects
later.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13865
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
If ldb_kv_filter_attrs() fails, we don't know that the dn of filtered_msg
is OK.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Apr 5 05:46:55 UTC 2019 on sn-devel-144
Set the default index cache from the passed option
"transaction_index_cache_size" on open. This allows the default cache
size to be overridden when processing large transactions i.e. joining a
large domain.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Set the index cache size to the number of records in the databse when
reindexing.
This significantly improves reindex performance. For a domain with
100,000 users the reindex times are reduced from 17 minutes to 45
seconds.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add the get_size method to the ldb_key_value layer, this will allow the
reindexing code to get an estimate of the number of records in the
database.
The lmdb backend returns an accurate count of the number of records in
the database withe the mdb_env_stat call.
The tdb backend does not provide a low cost method to determine the
number of records on the database. It does provide a tdb_summary call
however this this walks the entire database.
So for tdb we use the map size divided by 500, this over estimates the counts
for small domains, but the extra memory allocated for the cache should
not be significant.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pass the index cache size to ldb_kv_index_transaction_start. This will
allow it to be set for reindex and join operations, where the current
defaults result in a significant performance penalty on large databases.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Remove the lazy initialisation of the index cache. This make setting
the size of the cache for re-indexing easier, which will be done in
later commits.
Performance testing shows that the removal of lazy initialisation makes
no appreciable difference to performance.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Apr 3 07:50:03 UTC 2019 on sn-devel-144
Follow-up to
https://lists.samba.org/archive/samba/2018-September/217992.html
and following. This also fixes a small and very theoretical race: Between the
fstat and the read call the file size might change. This would make us fail on
potentially legitimate files.
This is more complex and probably slower, but looking at the use cases I don't
think the speed matters.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13859
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Mar 26 04:43:40 UTC 2019 on sn-devel-144
* Remove Python 2.x support except to build just the bare C library
* Remove --extra-python (a build time mode to produce Python2 and Python3
bindings at the same time)
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Mar 21 05:08:49 UTC 2019 on sn-devel-144
* Remove Python 2.x support except to build just the bare C library
* Remove --extra-python (a build time mode to produce Python2 and Python3
bindings at the same time)
* New minor version to allow Samba 4.10 to release a tevent if
required from that branch
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* Remove Python 2.x support except to build just the bare C library
* Remove --extra-python (a build time mode to produce Python2 and Python3
bindings at the same time)
* New minor version to allow Samba 4.10 to release a tdb if
required from that branch
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* Remove pytalloc_CObject_FromTallocPtr()
* Remove --extra-python (a build time mode to produce Python2 and Python3
bindings at the same time)
* New minor version to allow Samba 4.10 to release a talloc if
required from that branch
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
We do this by removing the confusing mandatory option to
conf.SAMBA_CHECK_PYTHON{,_HEADERS}(), instead just use the value of
--disable-python internally
This follows the default minimum of Python 3.4 and keeps things consistent
with the main Samba build where --disable-python is required to skip building
python bindings.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
We now used the default of 3.4 from conf.SAMBA_CHECK_PYTHON()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
These are no longer used by the build system so avoid
confusion by removing them from the tree.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This option is quite invasive in waf and was mainly for the python3 transition.
Testing with multiple python versions can be done by testing a full compile against
multiple versions, likewise multiple different binding versions can be created
the same way.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This function makes it harder to remove the --extra-python handlers and is only
provided for Python 2.x, support for which Samba is removing.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13844
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Mar 20 05:25:48 UTC 2019 on sn-devel-144
warning: cast from 'char *' to 'struct talloc_chunk *' increases required
alignment from 1 to 8
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Mar 19 12:38:50 UTC 2019 on sn-devel-144
If you connnect to a host with smbclient this gets always printed.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13823
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Mar 8 01:41:27 UTC 2019 on sn-devel-144
samba-o3 test failed in ubuntu:1804 image with:
../../lib/mscat/mscat_pkcs7.c: In function ‘mscat_pkcs7_import_catfile’:
../../lib/mscat/mscat_pkcs7.c:143:18: error: ‘blob.length’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
mscat_data.size = blob.length;
~~~~~~~~~~~~~~~~^~~~~~~~~~~~~
../../lib/mscat/mscat_pkcs7.c:142:18: error: ‘blob.data’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
mscat_data.data = blob.data;
~~~~~~~~~~~~~~~~^~~~~~~~~~~
../../lib/mscat/mscat_pkcs7.c: In function ‘mscat_pkcs7_verify’:
../../lib/mscat/mscat_pkcs7.c:225:16: error: ‘blob.length’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
ca_data.size = blob.length;
~~~~~~~~~~~~~^~~~~~~~~~~~~
../../lib/mscat/mscat_pkcs7.c:224:16: error: ‘blob.data’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
ca_data.data = blob.data;
~~~~~~~~~~~~~^~~~~~~~~~~
cc1: all warnings being treated as errors
Since in `mscat_read_file`, it may still return rc = 0 while goto error,
ends up with blob uninitialized.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
samba-o3 test failed in ubuntu:16.04 docker container:
==> /home/samba/samba/samba-o3.stderr <==
../../lib/ldb-samba/ldb_ildap.c: In function ‘ildb_handle_request’:
../../lib/ldb-samba/ldb_ildap.c:535:2: error: assuming signed overflow does not occur when simplifying conditional to constant [-Werror=strict-overflow]
for (i = 0; i < n; i++) {
^
../../lib/ldb-samba/ldb_ildap.c:579:2: error: assuming signed overflow does not occur when simplifying conditional to constant [-Werror=strict-overflow]
for (i = 0; i < n; i++) {
^
cc1: all warnings being treated as errors
Change type to mute errors.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
With the removal of the web server, there are not any users of this
parameter and so should just be removed.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
The standard string conversion routines convert a "signed string"
into the positive representation of the resulting value.
This is not wanted and therefore now detected and flag'ed as an error.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Böhme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
In order to detect an value overflow error during
the string to integer conversion with strtoul/strtoull,
the errno variable must be set to zero before the execution and
checked after the conversion is performed. This is achieved by
using the wrapper function strtoul_err and strtoull_err.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Böhme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Adding wrapper strtoull_err and strtoul_err to handle
error conditions of the conversion process.
Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Ralph Böhme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
