eca0d916f6
Add argument name assignment support and remove "alias" lines from format. This allows user to assign unique name to each argument. For example, $ echo p do_sys_open dfd=a0 filename=a1 flags=a2 mode=a3 > kprobe_events This assigns dfd, filename, flags, and mode to 1st - 4th arguments respectively. Trace buffer shows those names too. <...>-1439 [000] 1200885.933147: do_sys_open+0x0/0xdf: dfd=ffffff9c filename=bfa898ac flags=8000 mode=0 This helps users to know what each value means. Users can filter each events by these names too. Note that you can not filter by argN anymore. Signed-off-by: Masami Hiramatsu <mhiramat@redhat.com> Cc: Jim Keniston <jkenisto@us.ibm.com> Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Christoph Hellwig <hch@infradead.org> Cc: Frank Ch. Eigler <fche@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Jason Baron <jbaron@redhat.com> Cc: K.Prasad <prasad@linux.vnet.ibm.com> Cc: Lai Jiangshan <laijs@cn.fujitsu.com> Cc: Li Zefan <lizf@cn.fujitsu.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Tom Zanussi <tzanussi@gmail.com> LKML-Reference: <20090910235337.22412.77383.stgit@dhcp-100-2-132.bos.redhat.com> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
145 lines
5.6 KiB
Plaintext
145 lines
5.6 KiB
Plaintext
Kprobe-based Event Tracer
|
|
=========================
|
|
|
|
Documentation is written by Masami Hiramatsu
|
|
|
|
|
|
Overview
|
|
--------
|
|
This tracer is similar to the events tracer which is based on Tracepoint
|
|
infrastructure. Instead of Tracepoint, this tracer is based on kprobes(kprobe
|
|
and kretprobe). It probes anywhere where kprobes can probe(this means, all
|
|
functions body except for __kprobes functions).
|
|
|
|
Unlike the function tracer, this tracer can probe instructions inside of
|
|
kernel functions. It allows you to check which instruction has been executed.
|
|
|
|
Unlike the Tracepoint based events tracer, this tracer can add and remove
|
|
probe points on the fly.
|
|
|
|
Similar to the events tracer, this tracer doesn't need to be activated via
|
|
current_tracer, instead of that, just set probe points via
|
|
/sys/kernel/debug/tracing/kprobe_events. And you can set filters on each
|
|
probe events via /sys/kernel/debug/tracing/events/kprobes/<EVENT>/filter.
|
|
|
|
|
|
Synopsis of kprobe_events
|
|
-------------------------
|
|
p[:EVENT] SYMBOL[+offs]|MEMADDR [FETCHARGS] : Set a probe
|
|
r[:EVENT] SYMBOL[+0] [FETCHARGS] : Set a return probe
|
|
|
|
EVENT : Event name. If omitted, the event name is generated
|
|
based on SYMBOL+offs or MEMADDR.
|
|
SYMBOL[+offs] : Symbol+offset where the probe is inserted.
|
|
MEMADDR : Address where the probe is inserted.
|
|
|
|
FETCHARGS : Arguments. Each probe can have up to 128 args.
|
|
%REG : Fetch register REG
|
|
sN : Fetch Nth entry of stack (N >= 0)
|
|
sa : Fetch stack address.
|
|
@ADDR : Fetch memory at ADDR (ADDR should be in kernel)
|
|
@SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symbol)
|
|
aN : Fetch function argument. (N >= 0)(*)
|
|
rv : Fetch return value.(**)
|
|
ra : Fetch return address.(**)
|
|
+|-offs(FETCHARG) : Fetch memory at FETCHARG +|- offs address.(***)
|
|
NAME=FETCHARG: Set NAME as the argument name of FETCHARG.
|
|
|
|
(*) aN may not correct on asmlinkaged functions and at the middle of
|
|
function body.
|
|
(**) only for return probe.
|
|
(***) this is useful for fetching a field of data structures.
|
|
|
|
|
|
Per-Probe Event Filtering
|
|
-------------------------
|
|
Per-probe event filtering feature allows you to set different filter on each
|
|
probe and gives you what arguments will be shown in trace buffer. If an event
|
|
name is specified right after 'p:' or 'r:' in kprobe_events, the tracer adds
|
|
an event under tracing/events/kprobes/<EVENT>, at the directory you can see
|
|
'id', 'enabled', 'format' and 'filter'.
|
|
|
|
enabled:
|
|
You can enable/disable the probe by writing 1 or 0 on it.
|
|
|
|
format:
|
|
This shows the format of this probe event.
|
|
|
|
filter:
|
|
You can write filtering rules of this event.
|
|
|
|
id:
|
|
This shows the id of this probe event.
|
|
|
|
Event Profiling
|
|
---------------
|
|
You can check the total number of probe hits and probe miss-hits via
|
|
/sys/kernel/debug/tracing/kprobe_profile.
|
|
The first column is event name, the second is the number of probe hits,
|
|
the third is the number of probe miss-hits.
|
|
|
|
|
|
Usage examples
|
|
--------------
|
|
To add a probe as a new event, write a new definition to kprobe_events
|
|
as below.
|
|
|
|
echo p:myprobe do_sys_open dfd=a0 filename=a1 flags=a2 mode=a3 > /sys/kernel/debug/tracing/kprobe_events
|
|
|
|
This sets a kprobe on the top of do_sys_open() function with recording
|
|
1st to 4th arguments as "myprobe" event. As this example shows, users can
|
|
choose more familiar names for each arguments.
|
|
|
|
echo r:myretprobe do_sys_open rv ra >> /sys/kernel/debug/tracing/kprobe_events
|
|
|
|
This sets a kretprobe on the return point of do_sys_open() function with
|
|
recording return value and return address as "myretprobe" event.
|
|
You can see the format of these events via
|
|
/sys/kernel/debug/tracing/events/kprobes/<EVENT>/format.
|
|
|
|
cat /sys/kernel/debug/tracing/events/kprobes/myprobe/format
|
|
name: myprobe
|
|
ID: 75
|
|
format:
|
|
field:unsigned short common_type; offset:0; size:2;
|
|
field:unsigned char common_flags; offset:2; size:1;
|
|
field:unsigned char common_preempt_count; offset:3; size:1;
|
|
field:int common_pid; offset:4; size:4;
|
|
field:int common_tgid; offset:8; size:4;
|
|
|
|
field: unsigned long ip; offset:16;tsize:8;
|
|
field: int nargs; offset:24;tsize:4;
|
|
field: unsigned long dfd; offset:32;tsize:8;
|
|
field: unsigned long filename; offset:40;tsize:8;
|
|
field: unsigned long flags; offset:48;tsize:8;
|
|
field: unsigned long mode; offset:56;tsize:8;
|
|
|
|
print fmt: "%lx: dfd=%lx filename=%lx flags=%lx mode=%lx", ip, REC->dfd, REC->filename, REC->flags, REC->mode
|
|
|
|
|
|
You can see that the event has 4 arguments as in the expressions you specified.
|
|
|
|
echo > /sys/kernel/debug/tracing/kprobe_events
|
|
|
|
This clears all probe points. and you can see the traced information via
|
|
/sys/kernel/debug/tracing/trace.
|
|
|
|
cat /sys/kernel/debug/tracing/trace
|
|
# tracer: nop
|
|
#
|
|
# TASK-PID CPU# TIMESTAMP FUNCTION
|
|
# | | | | |
|
|
<...>-1447 [001] 1038282.286875: do_sys_open+0x0/0xd6: dfd=3 filename=7fffd1ec4440 flags=8000 mode=0
|
|
<...>-1447 [001] 1038282.286878: sys_openat+0xc/0xe <- do_sys_open: rv=fffffffffffffffe ra=ffffffff81367a3a
|
|
<...>-1447 [001] 1038282.286885: do_sys_open+0x0/0xd6: dfd=ffffff9c filename=40413c flags=8000 mode=1b6
|
|
<...>-1447 [001] 1038282.286915: sys_open+0x1b/0x1d <- do_sys_open: rv=3 ra=ffffffff81367a3a
|
|
<...>-1447 [001] 1038282.286969: do_sys_open+0x0/0xd6: dfd=ffffff9c filename=4041c6 flags=98800 mode=10
|
|
<...>-1447 [001] 1038282.286976: sys_open+0x1b/0x1d <- do_sys_open: rv=3 ra=ffffffff81367a3a
|
|
|
|
|
|
Each line shows when the kernel hits a probe, and <- SYMBOL means kernel
|
|
returns from SYMBOL(e.g. "sys_open+0x1b/0x1d <- do_sys_open" means kernel
|
|
returns from do_sys_open to sys_open+0x1b).
|
|
|
|
|