linux/Documentation
Andrey Konovalov eec028c938 kcov: remote coverage support
Patch series " kcov: collect coverage from usb and vhost", v3.

This patchset extends kcov to allow collecting coverage from backgound
kernel threads.  This extension requires custom annotations for each of
the places where coverage collection is desired.  This patchset
implements this for hub events in the USB subsystem and for vhost
workers.  See the first patch description for details about the kcov
extension.  The other two patches apply this kcov extension to USB and
vhost.

Examples of other subsystems that might potentially benefit from this
when custom annotations are added (the list is based on
process_one_work() callers for bugs recently reported by syzbot):

1. fs: writeback wb_workfn() worker,
2. net: addrconf_dad_work()/addrconf_verify_work() workers,
3. net: neigh_periodic_work() worker,
4. net/p9: p9_write_work()/p9_read_work() workers,
5. block: blk_mq_run_work_fn() worker.

These patches have been used to enable coverage-guided USB fuzzing with
syzkaller for the last few years, see the details here:

  https://github.com/google/syzkaller/blob/master/docs/linux/external_fuzzing_usb.md

This patchset has been pushed to the public Linux kernel Gerrit
instance:

  https://linux-review.googlesource.com/c/linux/kernel/git/torvalds/linux/+/1524

This patch (of 3):

Add background thread coverage collection ability to kcov.

With KCOV_ENABLE coverage is collected only for syscalls that are issued
from the current process.  With KCOV_REMOTE_ENABLE it's possible to
collect coverage for arbitrary parts of the kernel code, provided that
those parts are annotated with kcov_remote_start()/kcov_remote_stop().

This allows to collect coverage from two types of kernel background
threads: the global ones, that are spawned during kernel boot in a
limited number of instances (e.g.  one USB hub_event() worker thread is
spawned per USB HCD); and the local ones, that are spawned when a user
interacts with some kernel interface (e.g.  vhost workers).

To enable collecting coverage from a global background thread, a unique
global handle must be assigned and passed to the corresponding
kcov_remote_start() call.  Then a userspace process can pass a list of
such handles to the KCOV_REMOTE_ENABLE ioctl in the handles array field
of the kcov_remote_arg struct.  This will attach the used kcov device to
the code sections, that are referenced by those handles.

Since there might be many local background threads spawned from
different userspace processes, we can't use a single global handle per
annotation.  Instead, the userspace process passes a non-zero handle
through the common_handle field of the kcov_remote_arg struct.  This
common handle gets saved to the kcov_handle field in the current
task_struct and needs to be passed to the newly spawned threads via
custom annotations.  Those threads should in turn be annotated with
kcov_remote_start()/kcov_remote_stop().

Internally kcov stores handles as u64 integers.  The top byte of a
handle is used to denote the id of a subsystem that this handle belongs
to, and the lower 4 bytes are used to denote the id of a thread instance
within that subsystem.  A reserved value 0 is used as a subsystem id for
common handles as they don't belong to a particular subsystem.  The
bytes 4-7 are currently reserved and must be zero.  In the future the
number of bytes used for the subsystem or handle ids might be increased.

When a particular userspace process collects coverage by via a common
handle, kcov will collect coverage for each code section that is
annotated to use the common handle obtained as kcov_handle from the
current task_struct.  However non common handles allow to collect
coverage selectively from different subsystems.

Link: http://lkml.kernel.org/r/e90e315426a384207edbec1d6aa89e43008e4caf.1572366574.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Jason Wang <jasowang@redhat.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: David Windsor <dwindsor@gmail.com>
Cc: Elena Reshetova <elena.reshetova@intel.com>
Cc: Anders Roxell <anders.roxell@linaro.org>
Cc: Alexander Potapenko <glider@google.com>
Cc: Marco Elver <elver@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-12-04 19:44:14 -08:00
..
ABI chrome platform changes for v5.5 2019-12-03 14:37:12 -08:00
accounting docs: add some documentation dirs to the driver-api book 2019-07-15 11:03:02 -03:00
admin-guide TTY/Serial patches for 5.5-rc1 2019-12-03 14:09:14 -08:00
arm Documentation/arm/samsung-s3c24xx: Remove stray U+FEFF character to fix title 2019-08-12 15:25:32 -06:00
arm64 arm64 updates for 5.5: 2019-11-25 15:39:19 -08:00
block block: add iostat counters for flush requests 2019-11-21 09:06:47 -07:00
bpf bpf: Add s390 testing documentation 2019-10-30 16:25:31 +01:00
cdrom docs: add some directories to the main documentation index 2019-07-15 11:03:03 -03:00
core-api lib/genalloc.c: rename addr_in_gen_pool to gen_pool_has_addr 2019-12-04 19:44:13 -08:00
cpu-freq Documentation: cpufreq: Update policy notifier documentation 2019-09-02 22:44:05 +02:00
crypto crypto: ablkcipher - remove deprecated and unused ablkcipher support 2019-11-17 09:02:49 +08:00
dev-tools kcov: remote coverage support 2019-12-04 19:44:14 -08:00
devicetree TTY/Serial patches for 5.5-rc1 2019-12-03 14:09:14 -08:00
doc-guide kernel-doc: rename the kernel-doc directive 'functions' to 'identifiers' 2019-11-07 13:17:25 -07:00
driver-api Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
EDID docs: driver-api: add a series of orphaned documents 2019-07-15 11:03:02 -03:00
fault-injection docs: add some directories to the main documentation index 2019-07-15 11:03:03 -03:00
fb docs conversion for v5.3-rc1 2019-07-16 12:21:41 -07:00
features xtensa: update arch features 2019-10-20 23:48:28 -07:00
filesystems Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
firmware_class
firmware-guide ACPI: Documentation: Minor spelling fix in namespace.rst 2019-10-14 11:03:44 +02:00
fpga Documentation: fpga: dfl: add descriptions for thermal/power management interfaces 2019-10-16 19:18:26 -07:00
gpu drm/amdgpu: Improve RAS documentation (v2) 2019-11-06 16:27:48 -05:00
hid docs: add some documentation dirs to the driver-api book 2019-07-15 11:03:02 -03:00
hwmon Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
i2c Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
ia64 docs: add SPDX tags to new index files 2019-07-15 11:03:03 -03:00
ide docs: add some directories to the main documentation index 2019-07-15 11:03:03 -03:00
iio docs: add some documentation dirs to the driver-api book 2019-07-15 11:03:02 -03:00
infiniband Documentation/infiniband: update name of some functions 2019-09-13 16:55:55 -03:00
input Input: docs: fix spelling mistake "potocol" -> "protocol" 2019-08-06 11:24:49 -06:00
isdn docs: isdn: convert to ReST and add to kAPI bookset 2019-07-31 13:30:25 -06:00
kbuild kbuild: remove header compile test 2019-11-15 00:22:35 +09:00
kernel-hacking docs: Add documentation for Symbol Namespaces 2019-09-10 10:30:49 +02:00
leds leds: core: Add support for composing LED class device names 2019-07-25 20:07:52 +02:00
livepatch livepatch: Documentation of the new API for tracking system state changes 2019-11-01 13:08:24 +01:00
locking doc🔒 remove reference to clever use of read-write lock 2019-09-14 01:53:27 -06:00
m68k docs: README.buddha: convert to ReST and add to m68k book 2019-07-31 13:30:10 -06:00
maintainer docs: fix up the maintainer profile document 2019-11-25 08:42:12 -07:00
media media: Documentation: media: *_DEFAULT targets for subdevs 2019-11-10 17:49:27 +01:00
mic docs: driver-api: add remaining converted dirs to it 2019-07-15 11:03:03 -03:00
mips docs: fix some broken references 2019-10-10 11:25:39 -06:00
misc-devices docs: misc: xilinx_sdfec: Actually add documentation 2019-10-11 10:01:27 -06:00
netlabel docs: add some directories to the main documentation index 2019-07-15 11:03:03 -03:00
networking Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
nios2 docs: nios2: add it to the main Documentation body 2019-07-31 13:31:51 -06:00
nvdimm libnvdimm, MAINTAINERS: Maintainer Entry Profile 2019-11-25 08:34:54 -07:00
openrisc docs: openrisc: convert to ReST and add to documentation body 2019-07-31 13:30:20 -06:00
parisc docs: parisc: convert to ReST and add to documentation body 2019-07-31 13:30:15 -06:00
PCI Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2019-08-27 14:23:31 -07:00
pcmcia docs: add some directories to the main documentation index 2019-07-15 11:03:03 -03:00
power pci-v5.5-changes 2019-12-03 13:58:22 -08:00
powerpc powerpc/fsl_booke/32: Document KASLR implementation 2019-11-13 19:27:58 +11:00
process TTY/Serial patches for 5.5-rc1 2019-12-03 14:09:14 -08:00
RCU Documentation: Rename rcu_node_context_switch() to rcu_note_context_switch() 2019-10-29 02:48:29 -07:00
riscv riscv-docs: correct the sequence of the magic number 2 since it's little endian 2019-10-03 10:51:42 -06:00
s390 Documentation/s390: remove outdated debugging390 documentation 2019-08-21 12:41:43 +02:00
scheduler Documentation/scheduler: fix links in sched-stats 2019-10-29 04:35:41 -06:00
scsi scsi: core: Clean up SG_NONE 2019-11-12 22:21:35 -05:00
security Documentation: security: core.rst: fix warnings 2019-11-22 10:01:47 -07:00
sh docs: remove extra conf.py files 2019-07-17 06:57:52 -03:00
sound ALSA: docs: Update about the new PCM sync_stop ops 2019-11-20 19:39:54 +01:00
sparc docs: add arch doc directories to the index 2019-07-15 11:03:01 -03:00
sphinx docs, parallelism: Rearrange how jobserver reservations are made 2019-11-22 10:35:18 -07:00
sphinx-static doc-rst: Reduce CSS padding around Field 2019-10-02 10:03:06 -06:00
spi spi: docs: convert to ReST and add it to the kABI bookset 2019-07-31 14:13:13 -06:00
target docs: add some directories to the main documentation index 2019-07-15 11:03:03 -03:00
timers docs: add some directories to the main documentation index 2019-07-15 11:03:03 -03:00
trace Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
translations TTY/Serial patches for 5.5-rc1 2019-12-03 14:09:14 -08:00
usb USB: rio500: Remove Rio 500 kernel driver 2019-10-04 10:53:36 +02:00
userspace-api Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
virt KVM/arm updates for Linux 5.5: 2019-11-21 09:58:35 +01:00
virtual cpuidle: add haltpoll governor 2019-07-30 17:27:37 +02:00
vm mm/hmm: remove hmm_mirror and related 2019-11-23 19:56:45 -04:00
w1 docs: w1: Fix SPDX-License-Identifier syntax 2019-10-18 09:45:56 -06:00
watchdog linux-watchdog 5.4-rc1 tag 2019-09-27 11:17:38 -07:00
x86 Merge branch 'x86-boot-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-11-26 08:40:20 -08:00
xtensa docs: add arch doc directories to the index 2019-07-15 11:03:01 -03:00
.gitignore
asm-annotations.rst x86/ftrace: Get rid of function_hook 2019-10-25 10:52:22 +02:00
atomic_bitops.txt
atomic_t.txt Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-07-08 16:12:03 -07:00
bus-virt-phys-mapping.txt
Changes
CodingStyle
conf.py doc-rst: Programmatically render MAINTAINERS into ReST 2019-10-02 10:03:17 -06:00
COPYING-logo docs: logo.txt: rename it to COPYING-logo 2019-07-15 09:20:27 -03:00
crc32.txt
debugging-modules.txt
debugging-via-ohci1394.txt
digsig.txt
DMA-API-HOWTO.txt docs: DMA-API-HOWTO.txt: fix an unmarked code block 2019-07-15 09:20:24 -03:00
DMA-API.txt dma-mapping: remove dma_release_declared_memory 2019-09-04 11:13:19 +02:00
DMA-attributes.txt dma-mapping: remove the DMA_ATTR_WRITE_BARRIER flag 2019-11-14 12:01:54 -04:00
DMA-ISA-LPC.txt
docutils.conf doc-rst: Add missing newline at end of file 2019-06-20 14:16:56 -06:00
dontdiff modpost: dump missing namespaces into a single modules.nsdeps file 2019-11-11 20:10:01 +09:00
futex-requeue-pi.txt
hwspinlock.txt hwspinlock: add the 'in_atomic' API 2019-06-29 21:08:14 -07:00
index.rst Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
io_ordering.txt
io-mapping.txt
IPMI.txt
IRQ-affinity.txt
IRQ-domain.txt
IRQ.txt
irqflags-tracing.txt
Kconfig docs: Kbuild/Makefile: allow check for missing docs at build time 2019-06-07 11:33:16 -06:00
kobject.txt
kprobes.txt
kref.txt
logo.gif
lzo.txt
mailbox.txt
Makefile Kbuild updates for v5.5 2019-12-02 17:35:04 -08:00
memory-barriers.txt docs/memory-barriers.txt: Remove remaining references to mmiowb() 2019-11-22 10:00:46 -07:00
nommu-mmap.txt
padata.txt padata: allocate workqueue internally 2019-09-13 21:15:39 +10:00
percpu-rw-semaphore.txt
pi-futex.txt docs: locking: convert docs to ReST and rename to *.rst 2019-07-15 08:53:27 -03:00
preempt-locking.txt
rbtree.txt docs: rbtree.txt: fix Sphinx build warnings 2019-07-15 09:20:24 -03:00
remoteproc.txt remoteproc: add vendor resources handling 2019-06-29 12:02:17 -07:00
robust-futex-ABI.txt
robust-futexes.txt
rpmsg.txt
speculation.txt
static-keys.txt
SubmittingPatches
tee.txt Documentation: tee: Grammar s/the its/its/ 2019-06-07 11:23:38 -06:00
this_cpu_ops.txt
unaligned-memory-access.txt
xz.txt