1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-06 13:18:07 +03:00
Commit Graph

138726 Commits

Author SHA1 Message Date
Shachar Sharon
69b671dcbf vfs_ceph_new: use low-level APIs for symlink/readlink
Implement unlinkat using libcephfs low-level APIs. For readlink
operation need to resolve child inode by-lookup and then used the inode
reference for the actual low-level readlink.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 53c9269b21)
2024-08-13 15:37:12 +00:00
Shachar Sharon
9278e66114 vfs_ceph_new: use low-level APIs for unlinkat
Implement unlinkat using libcephfs low-level APIs. Operate using parent
directory's open file-handle. When flags has AT_REMOVEDIR bit set call
low-level rmdir; otherwise, do normal unlink.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 362a7cf866)
2024-08-13 15:37:12 +00:00
Shachar Sharon
2b660aaf9e vfs_ceph_new: use low-level APIs for fntimes
Implement fntimes hook using libcephfs' low-level APIs. Convert
smb_file_time to ceph_statx plus proper field mask on-the-fly upon
issuing low-level call to libcephfs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 20b7d2bfe0)
2024-08-13 15:37:12 +00:00
Shachar Sharon
7780114ab0 vfs_ceph_new: use low-level APIs for fchown/fchmod
Use libcephfs' low-level APIs to implement 'fchown' and 'fchmod' using
open file-handle. If fsp does not have an open cephfs Fh reference,
set errno to EBADF and return -1 to VFS.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit cb14d3630d)
2024-08-13 15:37:12 +00:00
Shachar Sharon
26851f16fc vfs_ceph_new: proper error handling to readdir
Error handling in the case of 'ceph_readdir' is done by setting 'errno'
deep within libcephfs code. In case of error, emit proper debug message
and re-update errno to avoid possible over-write by logging mechanism.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 24a3423949)
2024-08-13 15:37:12 +00:00
Shachar Sharon
9fd6c5b41f vfs_ceph_new: use low-level APIs for readdir ops
Implement readdir and rewinddir operations using libcephfs' low-level
APIs. Casts the opaque DIR pointer into struct vfs_ceph_dirp (the first
member of struct vfs_ceph_fh) to resolve the ceph_dir_result pointer
which libcephfs expects for readdir operations.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 99c7179e5d)
2024-08-13 15:37:12 +00:00
Shachar Sharon
5eaa874946 vfs_ceph_new: use low-level APIs for mkdirat
Implement 'mkdirat' hook using libcephfs' low-level APIs, via the open
file-handle reference to parent directory.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit bd955af86e)
2024-08-13 15:37:12 +00:00
Shachar Sharon
34c848251c vfs_ceph_new: use low-level APIs for fdopendir
Implement fdopendir using libcephfs low-level API and cached (via fsp)
open file-handle. Embed the result within cached vfs_ceph_fh so it may
be used properly by closedir.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit a8a7339c6b)
2024-08-13 15:37:12 +00:00
Shachar Sharon
655e7a1508 vfs_ceph_new: use low-level APIs for fstatat
Use libcephfs' low-level APIs to do lookup-by-name via parent's open
reference followed by getattr on the inode itself.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 13671cefff)
2024-08-13 15:37:12 +00:00
Shachar Sharon
6fdf1922a9 vfs_ceph_new: use low-level APIs for fstat
Use libcephfs' low-level APIs and apply the same logic as stat, but
via explicit inode-reference.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit f16183f90a)
2024-08-13 15:37:12 +00:00
Shachar Sharon
62a43faa11 vfs_ceph_new: use low-level APIs for open/close
Implement openat, close and closedir and hooks using libcephfs'
low-level APIs. Cache the open Fh* from libcephfs and its related
meta-data using VFS fsp-extension mechanism.

Upon open-create of new vfs_ceph_fh store the caller credentials
(ceph's UserPerm*) within the same context object for subsequent calls.
In addition, provide a "pseudo" fd numbering which is reported back to
VFS layer and used as debugging hints.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 4916768464)
2024-08-13 15:37:12 +00:00
Shachar Sharon
f3e5d7237b vfs_ceph_new: ref cephmount_cached entry in handle->data
Allow direct access to ceph-mount cached-entry via 'handle->data'
private pointer. Required in order to allow more complex cached-state
with each cephfs mount. Users should now use the local-helper function
'cmount_of' to access the underlying ceph_mount_info.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 31085c7efc)
2024-08-13 15:37:12 +00:00
Shachar Sharon
7bb3f5f251 vfs_ceph_new: use low-level APIs for lchown
Use libcephfs' low-level API ceph_ll_setattr to implement VFS lchown_fn
hook. Use to standard pattern of iget/iput to allow operation by Inode
reference.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit beb21324c9)
2024-08-13 15:37:12 +00:00
Shachar Sharon
be427077ce vfs_ceph_new: use low-level APIs for statfs
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 47224fbdeb)
2024-08-13 15:37:12 +00:00
Shachar Sharon
8e28065f56 vfs_ceph_new: use low-level APIs for lstat
Use libcephfs' low-level APIs and apply the same logic as stat, but
using AT_SYMLINK_NOFOLLOW flags.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 93d786b143)
2024-08-13 15:37:12 +00:00
Shachar Sharon
9f68daaccc vfs_ceph_new: use low-level APIs for stat
Start migrating to libcephfs' low-level APIs, using explicit Inode*
reference. Implement the VFS 'stat' hook using a ceph_ll_getattr
function, encapsulated with a pair of iget/iput to hold a
pinned-to-cache Inode* instance.

Upon calling to libcephfs this new code crates and destroys on-the-fly
a Ceph UserPerm instance based on the uig, gid and groups from
'handle->conn->session_info->unix_token'. This logic ensures that the
correct caller-credentials are passed-on to cephfs (instead of those
set upon connection-creation in legacy 'vfs_ceph.c').

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 1b78d79663)
2024-08-13 15:37:12 +00:00
Shachar Sharon
5a7f6e4610 vfs_ceph_new: use low-level APIs for disk_free
Start using libcephfs low-level APIs: get reference to root inode and
use it to query statfs. Requires an explicit put-inode to avoid resource
leakage by libcephfs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 192b0cf871)
2024-08-13 15:37:12 +00:00
Shachar Sharon
74524c438b vfs_ceph_new: next iteration of samba-to-cephfs bridge
Defined new module 'vfs_ceph_new.c' which serves as a place holder for
the next development phase of the bridge between samba's VFS layer and
libcephfs. Begin with a module which is almost identical to existing
'vfs_ceph.c', except for hooks-names prefix which is 'vfs_ceph_' in
order to make clear distinction from existing code base. Following
commits will also switch to low-level APIs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
(cherry picked from commit 3720452720)
2024-08-13 15:37:12 +00:00
Stefan Metzmacher
b375043d62 script/autobuild.py: do some basic testing using --without-winbind
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15687

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Aug  7 07:38:35 UTC 2024 on atb-devel-224

(cherry picked from commit c0d0767a3a)

Autobuild-User(v4-21-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-21-test): Wed Aug  7 11:15:19 UTC 2024 on atb-devel-224
2024-08-07 11:15:19 +00:00
Stefan Metzmacher
ba14164f72 s3:lib: add winbind_lookup_name_ex() fallback for --without-winbind
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15687

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 5b31b723c0)
2024-08-07 10:00:12 +00:00
Stefan Metzmacher
0ca6cd90b6 tdb: version 1.4.12
* Regression fix for ABI problem
  TDB_1_4_11 vs. TDB_1.4.11

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15673

Signed-off-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Aug  6 16:16:27 UTC 2024 on atb-devel-224

(cherry picked from commit 6b10cfbaf2)
2024-08-07 10:00:12 +00:00
Andreas Schneider
0cf9c6efd7 autobuild: Add ABI checks for libtalloc, libtevent and libtdb
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15673

Signed-off-by: Andreas Schneider <asn@samba.org>
(cherry picked from commit bcd49e3000)
2024-08-07 10:00:12 +00:00
Pavel Filipenský
725907587b WHATSNEW: update the "Automatic keytab update after machine password change" section
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15689

Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>

Autobuild-User(v4-21-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-21-test): Tue Aug  6 12:49:02 UTC 2024 on atb-devel-224
2024-08-06 12:49:02 +00:00
Pavel Filipenský
6f9a9394cf docs-xml: Delete descriptions for removed commands "net ads keytab add" and "net ads keytab add_update_ads"
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15689

Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Pavel Filipensky <pfilipensky@samba.org>
Autobuild-Date(master): Mon Aug  5 13:29:25 UTC 2024 on atb-devel-224

(cherry picked from commit a5f47f6efe)
2024-08-06 11:46:16 +00:00
Pavel Filipenský
ba6c2f68ec docs-xml: Fix trailing whitespace in net.8.xml
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15689

Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Reviewed-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit 374680010d)
2024-08-06 11:46:16 +00:00
Pavel Filipenský
ff9d9677bb docs:smbdotconf: Improve formatting of 'sync machine password to keytab'
Hint: review this commit with ignoring white space changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15689

Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Reviewed-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit 6c627903ee)
2024-08-06 11:46:16 +00:00
Jo Sutton
de85c86c48 ldb: Fix ldb public library header files being unusable
An accidental negation means that ldb_version.h is not installed when
ldb is built as a public library.

This is a regression introduced by commit
625fb48326.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15690

Signed-off-by: Jo Sutton <josutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Sun Aug  4 01:35:55 UTC 2024 on atb-devel-224

(cherry picked from commit 5851ae5554)
2024-08-06 11:46:16 +00:00
Andreas Schneider
6d69562e27 wafsamba: Fix ABI symbol name generation
Commit 0bc5b6f293 changed the script
for generating the ABI symbol version. It broke the ABI by changing all
dots to underscores.

This reverts the commit partially to preserve the dots in the version
part.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15673

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Günther Deschner <gd@samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Tue Aug  6 00:42:56 UTC 2024 on atb-devel-224

(cherry picked from commit 46215ab1b3)
2024-08-06 11:46:16 +00:00
Stefan Metzmacher
5ba371e09a WHATSNEW: update the Per-user and group "veto files" and "hide files" section
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15688

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(v4-21-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-21-test): Wed Jul 31 16:07:37 UTC 2024 on atb-devel-224
2024-07-31 16:07:37 +00:00
Volker Lendecke
bffa9349d4 docs: Document parametric form of hide and veto files
Bug: https://bugzilla.samba.org/show_bug.cgi?id=15688
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Jul 31 09:41:54 UTC 2024 on atb-devel-224

(cherry picked from commit 10e9b858a3)
2024-07-31 15:00:11 +00:00
Volker Lendecke
d5be00ab53 lib: Remove "token" parameter from set_namearray
Not needed anymore

Bug: https://bugzilla.samba.org/show_bug.cgi?id=15688
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit a150714cc6)
2024-07-31 15:00:11 +00:00
Volker Lendecke
13dbaf5556 lib: Remove per-user support from append_to_namearray
This is done in check_user_ok now

Bug: https://bugzilla.samba.org/show_bug.cgi?id=15688
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit b5169dd717)
2024-07-31 15:00:11 +00:00
Volker Lendecke
244ade4f12 tests: Test parametric per-user syntax for hide/veto files
Bug: https://bugzilla.samba.org/show_bug.cgi?id=15688
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit b5a128685e)
2024-07-31 15:00:11 +00:00
Volker Lendecke
fd73c865ee smbd: Respect per-user hide and veto files with parametric options
For my taste this is a nicer configuration syntax than

/../username1/file1/../username2/file2/

Is this too expensive? I don't think so. The scanning only happens an
tcon time, and it only walks the parametric options. If this turns out
to be a performance problem, we should think about smarter data
structures for parametric options instead of just a linked list of
string triples for everything.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=15688
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 17becb5f52)
2024-07-31 15:00:11 +00:00
Volker Lendecke
af0085aced lib: Factor out append_namearray from set_namearray
We'll have to add to an existing namearray soon.

This turns one talloc_array() into a set of reallocs. This is slower,
but set_namearray is only used for smb.conf entries where we don't
expect hundreds or more entries to add. I've done this to avoid array
length calculations, but if it turns out to be too slow we can get
smarter again.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=15688
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit fcd595a464)
2024-07-31 15:00:11 +00:00
Volker Lendecke
0b9371aa0c loadparm: Add lp_wi_scan_share_parametrics
Bug: https://bugzilla.samba.org/show_bug.cgi?id=15688
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 89da15756d)
2024-07-31 15:00:11 +00:00
Volker Lendecke
5148ff9706 loadparm: Factor out lp_wi_scan_parametrics
We'll scan share parametrics soon as well.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=15688
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 0536ac96e9)
2024-07-31 15:00:11 +00:00
Jule Anger
13fc70f5e0 VERSION: Bump version up to Samba 4.21.0rc2...
and re-enable GIT_SNAPSHOT.

Signed-off-by: Jule Anger <janger@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2024-07-29 13:38:39 +02:00
Jule Anger
729078d20c VERSION: Disable GIT_SNAPSHOT for the Samba 4.21.0rc1 release.
Signed-off-by: Jule Anger <janger@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2024-07-29 09:01:32 +00:00
Jule Anger
33f3cd3f0b WHATSNEW: Up to Samba 4.21.0rc1.
Signed-off-by: Jule Anger <janger@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2024-07-29 09:01:32 +00:00
Jule Anger
93a6656c13 tdb: version 1.4.11
* Add tdbdump -x option to output all data as hex values
* Add missing overflow check for num_values in pytdb.c
* Remove Py2 related tests
* Update times in tdb_transaction_commit per fd, not per name
* Fix compilation with TDB_TRACE=1
* Allow tracing of internal tdb

Signed-off-by: Jule Anger <janger@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2024-07-29 09:01:32 +00:00
Douglas Bagnall
e58e4a5aa9 ldb:kv_index: use subtransaction_cancel in transaction_cancel
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Sat Jul 27 23:51:44 UTC 2024 on atb-devel-224
2024-07-27 23:51:44 +00:00
Douglas Bagnall
18131aebf8 ldb:kv_index: subtransaction_cancel: check for nested tdb
Just in case, but also so ldb_kv_index_transaction_cancel() can use
this and retain the same logic.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2024-07-27 22:47:39 +00:00
Douglas Bagnall
9ecf96859f ldb:kv_index: don't recalculate a length
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2024-07-27 22:47:39 +00:00
Douglas Bagnall
1bf9ede94f ldb:kv_index: realloc away old dn list
We can't just free it, because has the GUID index list as a child, and
these are shared by the new dn list (from the subtransaction we are
committing). But if the dn list is long and the main transaction is
long-lived, we can save a lot of memory by turning this dn list into
an almost empty node in the talloc tree. This returns us to roughly
the situation we had prior to the last commit.

For example, with the repro.sh script on bug 15590 in indexes mode
with 10000 rules, The last 3 commits use this much memory at the end
of an unusually large transaction:

full talloc report on 'struct ldb_context' (total 4012222 bytes in 90058 blocks)
full talloc report on 'struct ldb_context' (total 2405482219 bytes in 90058 blocks)
full talloc report on 'struct ldb_context' (total 4282195 bytes in 90058 blocks)

That is, the last commit increased usage 500 fold, and this commit
brings it back to normal.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15590

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2024-07-27 22:47:39 +00:00
Douglas Bagnall
5f0198d698 ldb_kv_index: dn_list load sub transaction can re-use keys
We don't want to modify the original list, but we can reuse the keys
if we treat them as immutable and don't free them. That makes it a lot
quicker if there are many keys (i.e. where an index is useful) and may
sub-transactions. In particular, it avoids O(n²) talloc_memdups.

A removed comment that says "We have to free the top level index
memory otherwise we would leak", and this will be addressed in the
next commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15590

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2024-07-27 22:47:39 +00:00
Douglas Bagnall
ed7bc50b00 ldb:ldb_kv_dn_list_find_val: check for int overflow
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2024-07-27 22:47:39 +00:00
Douglas Bagnall
d8c7768fed ldb_kv_cache: always initialise dn_list.strict
The strict flag is only read in list intersection, so most of the time
it doesn't matter whether it is set because that path is not used.
Nevertheless seeing it set to all kinds of values is distracting.

The undefined behaviour has likely been hidden from static analysis
because the structure is passed through the in-memory tdb before use.

Incorrect true values will have disabled an optimisation but not
caused the wrong result.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2024-07-27 22:47:39 +00:00
Douglas Bagnall
351636efa7 tdb: allow tracing of internal tdb
This will trace internal databases to files like this:

tdb_0x5da896b51870.trace.267290

We avoid strlen(name) because name could be NULL in this case (which
works fine with glibc but feels bad).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2024-07-27 22:47:39 +00:00
Douglas Bagnall
e653b0870f tdb: fix compilation with TDB_TRACE=1
../../lib/tdb/common/tdb.c: In function ‘tdb_trace_record’:
../../lib/tdb/common/tdb.c:1224:22: error: ‘snprintf’ output truncated before the last format character [-Werror=format-truncation=]
 1224 |                 p += snprintf(p, 2, %02x, rec.dptr[i]);
      |                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../lib/tdb/common/tdb.c:1224:22: note: ‘snprintf’ output 3 bytes into a destination of size 2
cc1: all warnings being treated as errors

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2024-07-27 22:47:39 +00:00