linux/kernel/trace
Steven Rostedt 77ae365eca ring-buffer: make lockless
This patch converts the ring buffers into a completely lockless
buffer recording system. The read side still takes locks since
we still serialize readers. But the writers are the ones that
must be lockless (those can happen in NMIs).

The main change is to the "head_page" pointer. We write to the
tail, and read from the head. The "head_page" pointer in the cpu
buffer is now just a reference to where to look. The real head
page is now kept in the head_page->list->prev->next pointer.
That is, in the list head of the previous page we set flags.

The list pages are allocated to be aligned such that the lowest
significant bits are always zero pointing to the list. This gives
us play to put in flags to their pointers.

bit 0: set when the page is a head page
bit 1: set when the writer is moving the page (for overwrite mode)

cmpxchg is used to update the pointer.

When the writer wraps the buffer and the tail meets the head,
in overwrite mode, the writer must move the head page forward.
It first uses cmpxchg to change the pointer flag from 1 to 2.
Once this is done, the reader on another CPU will not take the
page from the buffer.

The writers need to protect against interrupts (we don't bother with
disabling interrupts because NMIs are allowed to write too).

After the writer sets the pointer flag to 2, it takes care to
manage interrupts coming in. This is discribed in detail within the
comments of the code.

 Changes in version 2:
  - Let reader reset entries value of header page.
  - Fix tail page passing commit page on reader page test.
  - Always increment entries and write counter in rb_tail_page_update
  - Add safety check in rb_set_commit_to_write to break out of infinite loop
  - add mask in rb_is_reader_page

[ Impact: lock free writing to the ring buffer ]

Signed-off-by: Steven Rostedt <srostedt@redhat.com>
2009-07-07 18:36:12 -04:00
..
blktrace.c Merge branch 'for-2.6.31' of git://git.kernel.dk/linux-2.6-block 2009-06-11 11:10:35 -07:00
ftrace.c tracing: Fix stack tracer sysctl handling 2009-06-26 20:48:39 +02:00
Kconfig tracing/fastboot: Document the need of initcall_debug 2009-06-29 10:22:10 +02:00
kmemtrace.c kmemtrace: Print binary output only if 'bin' option is set 2009-07-03 11:39:46 +02:00
Makefile tracing/events: convert block trace points to TRACE_EVENT() 2009-06-09 12:34:23 -04:00
ring_buffer_benchmark.c ring-buffer: have benchmark test print to trace buffer 2009-06-17 17:01:09 -04:00
ring_buffer.c ring-buffer: make lockless 2009-07-07 18:36:12 -04:00
trace_boot.c tracing: use macros to denote usec and nsec per second 2009-04-07 14:43:06 +02:00
trace_branch.c Merge branch 'linus' into tracing/core 2009-05-07 11:17:34 +02:00
trace_clock.c tracing: fix four sparse warnings 2009-03-22 18:16:54 +01:00
trace_event_profile.c tracing/events: fix concurrent access to ftrace_events list 2009-05-06 10:38:19 +02:00
trace_event_types.h trace_export: Repair missed fields 2009-06-26 20:48:40 +02:00
trace_events_filter.c tracing/filters: fix race between filter setting and module unload 2009-06-16 16:25:37 -04:00
trace_events.c tracing/events: Add trace_event boot option 2009-07-01 15:44:24 +02:00
trace_export.c tracing/filters: distinguish between signed and unsigned fields 2009-04-29 14:06:03 +02:00
trace_functions_graph.c function-graph: add stack frame test 2009-06-18 18:40:18 -04:00
trace_functions.c ftrace: Remove duplicate newline 2009-06-25 10:28:36 +02:00
trace_hw_branches.c Merge branch 'tracing/hw-branch-tracing' into tracing/core 2009-05-07 13:36:22 +02:00
trace_irqsoff.c tracing: have latency tracers set the latency format 2009-03-04 22:15:30 -05:00
trace_mmiotrace.c tracing: use macros to denote usec and nsec per second 2009-04-07 14:43:06 +02:00
trace_nop.c tracing/ftrace: make nop-tracer use polling wait for events on pipe 2009-03-23 09:22:15 +01:00
trace_output.c tracing: add protection around module events unload 2009-06-09 17:29:07 -04:00
trace_output.h tracing: add protection around module events unload 2009-06-09 17:29:07 -04:00
trace_power.c Merge branch 'linus' into tracing/core 2009-05-07 11:17:34 +02:00
trace_printk.c tracing_bprintk: Don't increment @pos in t_start() 2009-06-24 11:02:50 +02:00
trace_sched_switch.c tracing/events: move trace point headers into include/trace/events 2009-04-14 22:05:43 -04:00
trace_sched_wakeup.c tracing/wakeup: move access to wakeup_cpu into spinlock 2009-04-23 23:01:36 -04:00
trace_selftest_dynamic.c ftrace: fix dynamic ftrace selftest 2008-05-23 21:13:23 +02:00
trace_selftest.c x86, hw-branch-tracer: allocate selftest iterator on heap 2009-04-07 13:36:21 +02:00
trace_stack.c tracing: Fix stack tracer sysctl handling 2009-06-26 20:48:39 +02:00
trace_stat.c trace_stat: Don't increment @pos in seq start() 2009-06-24 11:02:51 +02:00
trace_stat.h tracing: add handler to trace_stat 2009-03-24 23:22:58 -04:00
trace_syscalls.c tracing/syscalls: use a dedicated file header 2009-04-09 05:43:32 +02:00
trace_sysprof.c Merge branch 'timers-for-linus-migration' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-06-15 10:06:19 -07:00
trace_workqueue.c trace_workqueue: remove blank line between each cpu 2009-06-02 01:14:26 +02:00
trace.c ring-buffer: make lockless 2009-07-07 18:36:12 -04:00
trace.h tracing/events: Add trace_event boot option 2009-07-01 15:44:24 +02:00