1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-24 21:34:56 +03:00
Commit Graph

99 Commits

Author SHA1 Message Date
Volker Lendecke
1cd2f38b54 profiling: Factor out functions to read smbprofile.tdb
We don't need all of Samba just to dump contents of this tdb, make
exporting profile information cheaper.

No direct use yet, but it's a good cleanup IMHO

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2023-06-16 16:14:30 +00:00
Volker Lendecke
a4972336f1 lib: Add a few required #includes
You find them if you try to #include these files directly

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2023-06-16 16:14:30 +00:00
Volker Lendecke
cac38aa387 vfs: Remove vfs telldir/seekdir functions
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Jun 14 00:26:55 UTC 2023 on atb-devel-224
2023-06-14 00:26:55 +00:00
Stefan Metzmacher
ff259bd1b7 smbprofile: add smbprofile_active() helper
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2023-06-01 07:20:31 +00:00
Volker Lendecke
812cb602e3 vfs: Add SMB_VFS_FSTATAT
Useful if you want to stat/fstat/lstat relative to a directory without
doing chdir first.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2022-03-10 18:23:35 +00:00
Christof Schmitt
f3b5733df7 profile: Remove syscall_kernel_flock profiling
This no longer calls flock, so it should not be part of the system call
profiling.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2021-09-21 18:47:38 +00:00
Samuel Cabrero
aa2ab7feb1 s3: VFS: Remove SMB_VFS_NTIMES(), no longer used
---------------
                        /               \
                       /      REST       \
                      /        IN         \
                     /        PEACE        \
                    /                       \
                    |                       |
                    |     SMB_VFS_NTIMES    |
                    |                       |
                    |                       |
                    |       13 April        |
                    |         2021          |
                    |                       |
                    |                       |
                   *|     *  *  *           | *
          _________)/\\_//(\/(/\)/\//\/\////|_)_______

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Samuel Cabrero <scabrero@samba.org>
Autobuild-Date(master): Mon Apr 19 13:19:35 UTC 2021 on sn-devel-184
2021-04-19 13:19:35 +00:00
Samuel Cabrero
ee3ea1bc74 VFS: Add SMB_VFS_FNTIMES
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2021-04-19 12:28:30 +00:00
Ralph Boehme
e4a38916e6 vfs: add SMB_VFS_OPENAT()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-21 20:38:31 +00:00
Jeremy Allison
773b6e1740 s3: VFS: Add SMB_VFS_GET_NT_ACL_AT().
Currently identical to SMB_VFS_GET_NT_ACL().

Next, add to all VFS modules that implement
get_nt_acl and eventually remove get_nt_acl.

NB. Modules that use smb_vfs_assert_all_fns()
have SMB_VFS_GET_NT_ACL_AT() will not build
until they have this function added.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-05-07 19:27:34 +00:00
Volker Lendecke
3fea05e01f smbd: Remove write cache
Since this was written, our write path has changed significantly. In
particular we have gained very flexible support for async I/O, with the
linux io_uring in the pipeline. Caching stuff in main memory and then
doing a blocking pwrite nowadays does not belong into the core smbd
code. If someone wants it back, it should be doable in a VFS module.

Removes: "write cache size" parameter.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Nov 13 00:20:55 UTC 2019 on sn-devel-184
2019-11-13 00:20:55 +00:00
Jeremy Allison
b01169ff32 s3: VFS: Complete the removal of SMB_VFS_CHOWN()
No longer used anywhere.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Oct 15 20:07:51 UTC 2019 on sn-devel-184
2019-10-15 20:07:51 +00:00
Jeremy Allison
81fbba8f64 s3: VFS: Complete the removal of SMB_VFS_RMDIR()
All users now use SMB_VFS_UNLINKAT(..., AT_REMOVEDIR).

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Thu Oct 10 07:26:43 UTC 2019 on sn-devel-184
2019-10-10 07:26:43 +00:00
Anoop C S
5084a69de1 s3: VFS: Add SMB_VFS_FCNTL
Signed-off-by: Anoop C S <anoopcs@redhat.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-10-08 08:38:32 +00:00
Jeremy Allison
1ec030398b s3: VFS: Complete the replacement of SMB_VFS_UNLINK() -> SMB_VFS_UNLINKAT().
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Thu Sep 26 18:40:53 UTC 2019 on sn-devel-184
2019-09-26 18:40:53 +00:00
Jeremy Allison
c58692cee9 s3: VFS: Add SMB_VFS_UNLINKAT().
Currently identical to SMB_VFS_UNLINK().

Next, add to all VFS modules that implement
unlink and eventually remove unlink.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2019-09-26 17:20:44 +00:00
Jeremy Allison
c7167e7a82 s3: VFS: Complete the replacement of SMB_VFS_MKDIR() -> SMB_VFS_MKDIRAT().
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Sep 11 19:44:44 UTC 2019 on sn-devel-184
2019-09-11 19:44:44 +00:00
Jeremy Allison
ca3bc78099 s3: VFS: Add SMB_VFS_MKDIRAT().
Currently identical to SMB_VFS_MKDIR().

Next, add to all VFS modules that implement
mkdir and eventually remove mkdir.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2019-09-11 18:24:28 +00:00
Jeremy Allison
0c0ca0dbd7 s3: smbd: profile: Remove syscall_symlink entry now syscall is no longer used.
Missed as part of the previous replacement of symlink -> symlinkat.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2019-09-11 18:24:28 +00:00
Jeremy Allison
42414b5bfc s3: VFS: Add SMB_VFS_SYMLINKAT().
Currently identical to SMB_VFS_SYMLINK().

Next, add to all VFS modules that implement
symlink and eventually remove symlink.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
2019-09-03 21:15:42 +00:00
Jeremy Allison
9b4685242f s3: VFS: Complete the replacement of SMB_VFS_READLINK() -> SMB_VFS_READLINKAT().
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Aug 23 20:06:22 UTC 2019 on sn-devel-184
2019-08-23 20:06:22 +00:00
Jeremy Allison
515c062b9e s3: VFS: Add SMB_VFS_READLINKAT().
Currently identical to SMB_VFS_READLINK().

Next, add to all VFS modules that implement
readlink and eventually remove readlink.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
2019-08-23 18:49:35 +00:00
Jeremy Allison
6a8bc6b549 s3: VFS: Complete the replacement of SMB_VFS_MKNOD() -> SMB_VFS_MKNODAT().
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Aug 22 19:17:07 UTC 2019 on sn-devel-184
2019-08-22 19:17:07 +00:00
Jeremy Allison
e2246876d1 s3: VFS: Add SMB_VFS_MKNODAT().
Currently identical to SMB_VFS_MKNOD().

Next, add to all VFS modules that implement
mknod and eventually remove mknod.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
2019-08-22 18:00:33 +00:00
Jeremy Allison
f42642e54e s3: VFS: Complete the replacement of SMB_VFS_LINK() -> SMB_VFS_LINKAT().
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Aug 20 22:26:17 UTC 2019 on sn-devel-184
2019-08-20 22:26:17 +00:00
Jeremy Allison
6c3928e4e5 s3: VFS: Add SMB_VFS_LINKAT().
Currently identical to SMB_VFS_LINK().

Next, move add to all VFS modules that implement
link and eventually remove link.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
2019-08-20 21:09:27 +00:00
Jeremy Allison
6acf4c35b4 s3: VFS: Complete the replacement of SMB_VFS_RENAME() -> SMB_VFS_RENAMEAT().
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2019-08-16 19:52:37 +00:00
Jeremy Allison
606398a3a6 s3: VFS: Add SMB_VFS_RENAMEAT().
Currently identical to SMB_VFS_RENAME() - uses
AT_FDCWD for both src and dst directories.

Next, move add to all VFS modules that implement
rename and eventually remove rename.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2019-08-16 19:52:32 +00:00
Ralph Boehme
7e1f9c9d83 s3: vfs: add SMB_VFS_GETXATTRAT_SEND/RECV
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-27 13:07:14 +02:00
Ralph Boehme
27bb2cbc2e vfs_default: fix async fsync idle/busy time profiling
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-25 00:23:13 +02:00
Jeremy Allison
5bbb831041 s3: VFS: Remove SMB_VFS_FCHMOD_ACL().
No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-05-25 18:39:25 +02:00
Jeremy Allison
7b8fa17f06 s3: VFS: Remove SMB_VFS_CHMOD_ACL().
No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-05-25 18:39:24 +02:00
Jeremy Allison
48f72803f0 s3: profile: Cleanup - we no longer use read/write/fsync syscalls.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-05-17 23:41:10 +02:00
Uri Simchoni
b2ae298526 s3-profile: add PROFILE_TIMESTAMP macro
This is a get-timestamp macro which works only when
profiling is enabled in the build. The underlying
clock_gettime_mono() function can be costly on some
architectures and we want to avoid it if it's not
necessary.

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2016-03-28 20:45:16 +02:00
Uri Simchoni
f1c84e49db s3-profile: reduce dependencies of smbprofile.h
Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2016-03-28 20:45:16 +02:00
Volker Lendecke
b4b4fd0ba0 smbprofile: Add dst pid to smbprofile_cleanup
The consolidation will soon be done by a separate process. We need to
avoid the getpid() call in smbprofile_cleanup().

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2015-11-16 14:51:33 +01:00
Stefan Metzmacher
b9f9869d1b s3:smbprofile: profile the system and user space cpu time
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2015-03-06 12:31:10 +01:00
Volker Lendecke
74a16a1094 s3:smbprofile: Replace sysv shmem with tdb
What?

This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.

Why?

At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.

The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.

But isn't it slow?

Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.

There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.

The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2015-03-06 12:31:10 +01:00
Stefan Metzmacher
5fa692b4aa s3:smbprofile: specify SMBPROFILE_STATS_SECTION_START() with name vs. display[name]
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2015-03-06 12:31:10 +01:00
Stefan Metzmacher
553b83c4d5 s3:smbprofile: profile async pread/pwrite/fsync syscalls
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Nov 19 23:13:10 CET 2014 on sn-devel-104
2014-11-19 23:13:10 +01:00
Stefan Metzmacher
185c6feab7 s3:smbprofile: track connect_count and disconnect_count
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-11-19 20:51:37 +01:00
Stefan Metzmacher
970b00e6ac s3:smbprofile: remove unused {START,END}_PROFILE_STAMP()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-11-19 20:51:37 +01:00
Stefan Metzmacher
487556e68e s3:smbprofile: report idle state of 'idle_count' and 'idle_time'
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-11-19 20:51:37 +01:00
Stefan Metzmacher
49f84f0719 s3:smbprofile: improve profiling for the security context switching.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-11-19 20:51:37 +01:00
Stefan Metzmacher
e007c60a4f s3:smbprofile: count all SMB1 and SMB2 requests as 'request_count'
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-11-19 20:51:37 +01:00
Stefan Metzmacher
0b0ac41aea s3:smb2_server: use async smbprofile macros
This improves profiling and corrently counts the total and idle time for async
requests.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-11-19 20:51:37 +01:00
Stefan Metzmacher
cee1b4b053 s3:smbprofile: rewrite the internal macros
We now autogenerate a lot of code using
SMBPROFILE_STATS_ALL_SECTIONS macro which expands to
different SMBPROFILE_STATS_{COUNT,BASIC,BYTES,IOBYTES} macros.

This also allows async profiling using:

   struct mystate {
       ...

       SMBPROFILE_BASIC_ASYNC_STATE(profile_state);
       ...
   };

   ...

   SMBPROFILE_BASIC_ASYNC_START(SMB2_negotiate, profile_p, mystate->profile_state);

   ...

   SMBPROFILE_BYTES_ASYNC_SET_IDLE(mystate->profile_state);

   ...

   SMBPROFILE_BYTES_ASYNC_SET_BUSY(mystate->profile_state);

   ...

   SMBPROFILE_BASIC_ASYNC_END(mystate->profile_state);

The current START_PROFILE*()/END_PROFILE*() are implemented as legacy wrappers.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2014-11-19 20:51:37 +01:00
Volker Lendecke
f82bb9fffd s3:smbprofile: Make smbprofile.h includable on its own
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-11-19 20:51:37 +01:00
Stefan Metzmacher
86a951b4ff s3:smbd: improve writecache profiling
In order to have useful profiling counters should never be decremented.
We need a separate counter for deallocation events.

The current value can be calculated by allocations - deallocations.

We also use better names and avoid having an array for the flush reasons.
This will simplify further profiling improvements a lot.

The value writecache_num_write_caches (this was similar to writecache_allocations)
is replaced by writecache_cached_writes, which counts the amount of writes which
were completely handled by the cache.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-11-19 20:51:37 +01:00
Stefan Metzmacher
255ff0a972 s3:smbprofile: remove unused nmbd related counters
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-11-19 20:51:36 +01:00