2014-09-11 18:46:39 +04:00
/*
2002-01-30 09:08:46 +03:00
Unix SMB / CIFS implementation .
1999-04-01 09:35:22 +04:00
store smbd profiling information in shared memory
Copyright ( C ) Andrew Tridgell 1999
2006-07-11 22:01:26 +04:00
Copyright ( C ) James Peach 2006
1999-04-01 09:35:22 +04:00
This program is free software ; you can redistribute it and / or modify
it under the terms of the GNU General Public License as published by
2007-07-09 23:25:36 +04:00
the Free Software Foundation ; either version 3 of the License , or
1999-04-01 09:35:22 +04:00
( at your option ) any later version .
2014-09-11 18:46:39 +04:00
1999-04-01 09:35:22 +04:00
This program is distributed in the hope that it will be useful ,
but WITHOUT ANY WARRANTY ; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE . See the
GNU General Public License for more details .
2014-09-11 18:46:39 +04:00
1999-04-01 09:35:22 +04:00
You should have received a copy of the GNU General Public License
2007-07-10 04:52:41 +04:00
along with this program . If not , see < http : //www.gnu.org/licenses/>.
1999-04-01 09:35:22 +04:00
*/
# include "includes.h"
2011-04-15 14:39:15 +04:00
# include "system/filesys.h"
2014-11-14 14:52:33 +03:00
# include "system/time.h"
2011-03-24 17:31:06 +03:00
# include "messages.h"
2011-04-14 02:36:23 +04:00
# include "smbprofile.h"
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
# include "lib/tdb_wrap/tdb_wrap.h"
# include <tevent.h>
# include "../lib/crypto/crypto.h"
1999-04-01 09:35:22 +04:00
2014-11-14 14:52:33 +03:00
# ifdef HAVE_SYS_RESOURCE_H
# include <sys/resource.h>
# endif
2000-10-11 09:31:39 +04:00
struct profile_stats * profile_p ;
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
struct smbprofile_global_state smbprofile_state ;
2000-10-06 22:13:52 +04:00
2000-10-07 03:01:47 +04:00
/****************************************************************************
2007-04-20 05:52:44 +04:00
Set a profiling level .
2000-10-07 03:01:47 +04:00
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
2016-03-28 10:11:33 +03:00
void set_profile_level ( int level , const struct server_id * src )
2000-10-07 03:01:47 +04:00
{
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
SMB_ASSERT ( smbprofile_state . internal . db ! = NULL ) ;
2000-10-07 03:01:47 +04:00
switch ( level ) {
2000-10-11 09:31:39 +04:00
case 0 : /* turn off profiling */
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
smbprofile_state . config . do_count = false ;
smbprofile_state . config . do_times = false ;
2005-12-25 12:49:10 +03:00
DEBUG ( 1 , ( " INFO: Profiling turned OFF from pid %d \n " ,
2016-03-28 10:11:33 +03:00
( int ) procid_to_pid ( src ) ) ) ;
2000-10-07 03:01:47 +04:00
break ;
2000-10-11 09:31:39 +04:00
case 1 : /* turn on counter profiling only */
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
smbprofile_state . config . do_count = true ;
smbprofile_state . config . do_times = false ;
2005-12-25 12:49:10 +03:00
DEBUG ( 1 , ( " INFO: Profiling counts turned ON from pid %d \n " ,
2016-03-28 10:11:33 +03:00
( int ) procid_to_pid ( src ) ) ) ;
2000-10-07 03:01:47 +04:00
break ;
2000-10-11 09:31:39 +04:00
case 2 : /* turn on complete profiling */
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
smbprofile_state . config . do_count = true ;
smbprofile_state . config . do_times = true ;
2005-12-25 12:49:10 +03:00
DEBUG ( 1 , ( " INFO: Full profiling turned ON from pid %d \n " ,
2016-03-28 10:11:33 +03:00
( int ) procid_to_pid ( src ) ) ) ;
2000-10-07 03:01:47 +04:00
break ;
2000-10-11 09:31:39 +04:00
case 3 : /* reset profile values */
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
ZERO_STRUCT ( profile_p - > values ) ;
tdb_wipe_all ( smbprofile_state . internal . db - > tdb ) ;
2005-12-25 12:49:10 +03:00
DEBUG ( 1 , ( " INFO: Profiling values cleared from pid %d \n " ,
2016-03-28 10:11:33 +03:00
( int ) procid_to_pid ( src ) ) ) ;
2000-10-11 09:31:39 +04:00
break ;
2000-10-07 03:01:47 +04:00
}
}
2007-04-20 05:52:44 +04:00
/****************************************************************************
receive a set profile level message
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
2007-05-15 16:18:17 +04:00
static void profile_message ( struct messaging_context * msg_ctx ,
void * private_data ,
uint32_t msg_type ,
struct server_id src ,
DATA_BLOB * data )
2007-04-20 05:52:44 +04:00
{
int level ;
2007-05-15 16:18:17 +04:00
if ( data - > length ! = sizeof ( level ) ) {
DEBUG ( 0 , ( " got invalid profile message \n " ) ) ;
return ;
}
memcpy ( & level , data - > data , sizeof ( level ) ) ;
2016-03-28 10:11:33 +03:00
set_profile_level ( level , & src ) ;
2007-04-20 05:52:44 +04:00
}
2000-11-11 03:33:33 +03:00
/****************************************************************************
receive a request profile level message
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
2007-05-15 16:18:17 +04:00
static void reqprofile_message ( struct messaging_context * msg_ctx ,
2014-09-11 18:46:39 +04:00
void * private_data ,
uint32_t msg_type ,
2007-05-15 16:18:17 +04:00
struct server_id src ,
DATA_BLOB * data )
2000-11-11 03:33:33 +03:00
{
int level ;
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
level = 1 ;
if ( smbprofile_state . config . do_count ) {
level + = 2 ;
}
if ( smbprofile_state . config . do_times ) {
level + = 4 ;
}
2014-09-30 19:08:20 +04:00
2005-09-30 21:13:37 +04:00
DEBUG ( 1 , ( " INFO: Received REQ_PROFILELEVEL message from PID %u \n " ,
( unsigned int ) procid_to_pid ( & src ) ) ) ;
2007-05-15 16:18:17 +04:00
messaging_send_buf ( msg_ctx , src , MSG_PROFILELEVEL ,
2015-05-14 03:26:01 +03:00
( uint8_t * ) & level , sizeof ( level ) ) ;
2000-11-11 03:33:33 +03:00
}
1999-04-01 09:35:22 +04:00
/*******************************************************************
open the profiling shared memory area
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
2007-10-19 04:40:25 +04:00
bool profile_setup ( struct messaging_context * msg_ctx , bool rdonly )
1999-04-01 09:35:22 +04:00
{
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
char * db_name ;
2018-11-05 20:03:51 +03:00
bool ok = false ;
int rc ;
1999-04-01 09:35:22 +04:00
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
if ( smbprofile_state . internal . db ! = NULL ) {
return true ;
}
1999-04-01 09:35:22 +04:00
2018-08-16 11:51:44 +03:00
db_name = cache_path ( talloc_tos ( ) , " smbprofile.tdb " ) ;
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
if ( db_name = = NULL ) {
return false ;
}
2014-09-11 18:46:39 +04:00
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
smbprofile_state . internal . db = tdb_wrap_open (
NULL , db_name , 0 ,
rdonly ? 0 : TDB_CLEAR_IF_FIRST | TDB_MUTEX_LOCKING ,
O_CREAT | ( rdonly ? O_RDONLY : O_RDWR ) , 0644 ) ;
if ( smbprofile_state . internal . db = = NULL ) {
return false ;
1999-04-01 09:35:22 +04:00
}
2014-09-11 18:46:39 +04:00
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
if ( msg_ctx ! = NULL ) {
messaging_register ( msg_ctx , NULL , MSG_PROFILE ,
profile_message ) ;
messaging_register ( msg_ctx , NULL , MSG_REQ_PROFILELEVEL ,
reqprofile_message ) ;
2014-09-11 18:46:39 +04:00
}
2023-05-31 10:48:58 +03:00
profile_p = & smbprofile_state . stats . global ;
2019-05-15 09:41:12 +03:00
2023-05-31 10:48:58 +03:00
rc = smbprofile_magic ( profile_p , & profile_p - > magic ) ;
2018-11-05 20:03:51 +03:00
if ( rc ! = 0 ) {
goto out ;
}
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
2018-11-05 20:03:51 +03:00
ok = true ;
out :
2019-05-15 09:41:12 +03:00
2018-11-05 20:03:51 +03:00
return ok ;
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
}
void smbprofile_dump_setup ( struct tevent_context * ev )
{
TALLOC_FREE ( smbprofile_state . internal . te ) ;
smbprofile_state . internal . ev = ev ;
}
static void smbprofile_dump_timer ( struct tevent_context * ev ,
struct tevent_timer * te ,
struct timeval current_time ,
void * private_data )
{
smbprofile_dump ( ) ;
}
void smbprofile_dump_schedule_timer ( void )
{
struct timeval tv ;
GetTimeOfDay ( & tv ) ;
tv . tv_sec + = 1 ;
smbprofile_state . internal . te = tevent_add_timer (
smbprofile_state . internal . ev ,
smbprofile_state . internal . ev ,
tv ,
smbprofile_dump_timer ,
NULL ) ;
}
static int profile_stats_parser ( TDB_DATA key , TDB_DATA value ,
void * private_data )
{
struct profile_stats * s = private_data ;
if ( value . dsize ! = sizeof ( struct profile_stats ) ) {
* s = ( struct profile_stats ) { } ;
return 0 ;
1999-04-01 09:35:22 +04:00
}
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
memcpy ( s , value . dptr , value . dsize ) ;
if ( s - > magic ! = profile_p - > magic ) {
* s = ( struct profile_stats ) { } ;
return 0 ;
1999-04-01 09:35:22 +04:00
}
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
return 0 ;
}
void smbprofile_dump ( void )
{
2022-07-25 15:29:35 +03:00
pid_t pid = 0 ;
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
TDB_DATA key = { . dptr = ( uint8_t * ) & pid , . dsize = sizeof ( pid ) } ;
struct profile_stats s = { } ;
int ret ;
2014-11-14 14:52:33 +03:00
# ifdef HAVE_GETRUSAGE
struct rusage rself ;
# endif /* HAVE_GETRUSAGE */
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
TALLOC_FREE ( smbprofile_state . internal . te ) ;
2019-01-14 21:42:03 +03:00
if ( ! ( smbprofile_state . config . do_count | |
smbprofile_state . config . do_times ) ) {
return ;
}
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
if ( smbprofile_state . internal . db = = NULL ) {
return ;
1999-04-01 09:35:22 +04:00
}
2022-07-25 15:29:35 +03:00
pid = tevent_cached_getpid ( ) ;
2014-11-14 14:52:33 +03:00
# ifdef HAVE_GETRUSAGE
ret = getrusage ( RUSAGE_SELF , & rself ) ;
if ( ret ! = 0 ) {
ZERO_STRUCT ( rself ) ;
}
profile_p - > values . cpu_user_stats . time =
( rself . ru_utime . tv_sec * 1000000 ) +
rself . ru_utime . tv_usec ;
profile_p - > values . cpu_system_stats . time =
( rself . ru_stime . tv_sec * 1000000 ) +
rself . ru_stime . tv_usec ;
# endif /* HAVE_GETRUSAGE */
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
ret = tdb_chainlock ( smbprofile_state . internal . db - > tdb , key ) ;
if ( ret ! = 0 ) {
return ;
1999-04-01 09:35:22 +04:00
}
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
tdb_parse_record ( smbprofile_state . internal . db - > tdb ,
key , profile_stats_parser , & s ) ;
smbprofile_stats_accumulate ( profile_p , & s ) ;
tdb_store ( smbprofile_state . internal . db - > tdb , key ,
( TDB_DATA ) {
. dptr = ( uint8_t * ) profile_p ,
. dsize = sizeof ( * profile_p )
} ,
0 ) ;
tdb_chainunlock ( smbprofile_state . internal . db - > tdb , key ) ;
ZERO_STRUCT ( profile_p - > values ) ;
return ;
}
2015-11-06 16:55:35 +03:00
void smbprofile_cleanup ( pid_t pid , pid_t dst )
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
{
TDB_DATA key = { . dptr = ( uint8_t * ) & pid , . dsize = sizeof ( pid ) } ;
struct profile_stats s = { } ;
struct profile_stats acc = { } ;
int ret ;
if ( smbprofile_state . internal . db = = NULL ) {
return ;
2007-05-15 16:18:17 +04:00
}
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
ret = tdb_chainlock ( smbprofile_state . internal . db - > tdb , key ) ;
if ( ret ! = 0 ) {
return ;
}
ret = tdb_parse_record ( smbprofile_state . internal . db - > tdb ,
key , profile_stats_parser , & s ) ;
if ( ret = = - 1 ) {
tdb_chainunlock ( smbprofile_state . internal . db - > tdb , key ) ;
return ;
}
tdb_delete ( smbprofile_state . internal . db - > tdb , key ) ;
tdb_chainunlock ( smbprofile_state . internal . db - > tdb , key ) ;
2015-11-06 16:55:35 +03:00
pid = dst ;
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2014-09-29 20:08:17 +04:00
ret = tdb_chainlock ( smbprofile_state . internal . db - > tdb , key ) ;
if ( ret ! = 0 ) {
return ;
}
tdb_parse_record ( smbprofile_state . internal . db - > tdb ,
key , profile_stats_parser , & acc ) ;
/*
* We may have to fix the disconnect count
* in case the process died
*/
s . values . disconnect_stats . count = s . values . connect_stats . count ;
smbprofile_stats_accumulate ( & acc , & s ) ;
acc . magic = profile_p - > magic ;
tdb_store ( smbprofile_state . internal . db - > tdb , key ,
( TDB_DATA ) {
. dptr = ( uint8_t * ) & acc ,
. dsize = sizeof ( acc )
} ,
0 ) ;
tdb_chainunlock ( smbprofile_state . internal . db - > tdb , key ) ;
}
void smbprofile_collect ( struct profile_stats * stats )
{
if ( smbprofile_state . internal . db = = NULL ) {
return ;
}
2023-05-31 10:48:58 +03:00
smbprofile_collect_tdb ( smbprofile_state . internal . db - > tdb ,
profile_p - > magic ,
stats ) ;
1999-04-01 09:35:22 +04:00
}