linux/kernel/bpf
David Vernet 156ed20d22 bpf: Don't use rcu_users to refcount in task kfuncs
A series of prior patches added some kfuncs that allow struct
task_struct * objects to be used as kptrs. These kfuncs leveraged the
'refcount_t rcu_users' field of the task for performing refcounting.
This field was used instead of 'refcount_t usage', as we wanted to
leverage the safety provided by RCU for ensuring a task's lifetime.

A struct task_struct is refcounted by two different refcount_t fields:

1. p->usage:     The "true" refcount field which task lifetime. The
		 task is freed as soon as this refcount drops to 0.

2. p->rcu_users: An "RCU users" refcount field which is statically
		 initialized to 2, and is co-located in a union with
		 a struct rcu_head field (p->rcu). p->rcu_users
		 essentially encapsulates a single p->usage
		 refcount, and when p->rcu_users goes to 0, an RCU
		 callback is scheduled on the struct rcu_head which
		 decrements the p->usage refcount.

Our logic was that by using p->rcu_users, we would be able to use RCU to
safely issue refcount_inc_not_zero() a task's rcu_users field to
determine if a task could still be acquired, or was exiting.
Unfortunately, this does not work due to p->rcu_users and p->rcu sharing
a union. When p->rcu_users goes to 0, an RCU callback is scheduled to
drop a single p->usage refcount, and because the fields share a union,
the refcount immediately becomes nonzero again after the callback is
scheduled.

If we were to split the fields out of the union, this wouldn't be a
problem. Doing so should also be rather non-controversial, as there are
a number of places in struct task_struct that have padding which we
could use to avoid growing the structure by splitting up the fields.

For now, so as to fix the kfuncs to be correct, this patch instead
updates bpf_task_acquire() and bpf_task_release() to use the p->usage
field for refcounting via the get_task_struct() and put_task_struct()
functions. Because we can no longer rely on RCU, the change also guts
the bpf_task_acquire_not_zero() and bpf_task_kptr_get() functions
pending a resolution on the above problem.

In addition, the task fixes the kfunc and rcu_read_lock selftests to
expect this new behavior.

Fixes: 90660309b0 ("bpf: Add kfuncs for storing struct task_struct * as a kptr")
Fixes: fca1aa7551 ("bpf: Handle MEM_RCU type properly")
Reported-by: Matus Jokay <matus.jokay@stuba.sk>
Signed-off-by: David Vernet <void@manifault.com>
Link: https://lore.kernel.org/r/20221206210538.597606-1-void@manifault.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2022-12-06 16:40:16 -08:00
..
preload bpf: iterators: Build and use lightweight bootstrap version of bpftool 2022-07-15 12:01:30 -07:00
arraymap.c bpf: Do btf_record_free outside map_free callback 2022-11-17 19:11:31 -08:00
bloom_filter.c treewide: use get_random_u32() when possible 2022-10-11 17:42:58 -06:00
bpf_cgrp_storage.c bpf: Fix a compilation failure with clang lto build 2022-11-30 17:13:25 -08:00
bpf_inode_storage.c bpf: Fix a compilation failure with clang lto build 2022-11-30 17:13:25 -08:00
bpf_iter.c bpf: Initialize the bpf_run_ctx in bpf_iter_run_prog() 2022-08-18 17:06:13 -07:00
bpf_local_storage.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2022-11-29 13:04:52 -08:00
bpf_lru_list.c bpf_lru_list: Read double-checked variable once without lock 2021-02-10 15:54:26 -08:00
bpf_lru_list.h printk: stop including cache.h from printk.h 2022-05-13 07:20:07 -07:00
bpf_lsm.c bpf: Do not mark certain LSM hook arguments as trusted 2022-12-04 12:59:58 -08:00
bpf_struct_ops_types.h bpf: Add dummy BPF STRUCT_OPS for test purpose 2021-11-01 14:10:00 -07:00
bpf_struct_ops.c bpf: Remove is_valid_bpf_tramp_flags() 2022-07-11 21:04:58 +02:00
bpf_task_storage.c bpf: Fix a compilation failure with clang lto build 2022-11-30 17:13:25 -08:00
btf.c bpf: Do not mark certain LSM hook arguments as trusted 2022-12-04 12:59:58 -08:00
cgroup_iter.c bpf: Pin the start cgroup in cgroup_iter_seq_init() 2022-11-21 17:40:42 +01:00
cgroup.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2022-10-03 17:44:18 -07:00
core.c bpf: Introduce bpf_obj_new 2022-11-17 19:22:14 -08:00
cpumap.c bpf: Expand map key argument of bpf_redirect_map to u64 2022-11-15 09:00:27 -08:00
devmap.c bpf: Expand map key argument of bpf_redirect_map to u64 2022-11-15 09:00:27 -08:00
disasm.c bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause 2021-09-02 14:49:23 +02:00
disasm.h bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause 2021-09-02 14:49:23 +02:00
dispatcher.c bpf: Add explicit cast to 'void *' for __BPF_DISPATCHER_UPDATE() 2022-11-07 16:07:35 -08:00
hashtab.c bpf: Do btf_record_free outside map_free callback 2022-11-17 19:11:31 -08:00
helpers.c bpf: Don't use rcu_users to refcount in task kfuncs 2022-12-06 16:40:16 -08:00
inode.c bpf: Convert bpf_preload.ko to use light skeleton. 2022-02-10 23:31:51 +01:00
Kconfig rcu: Make the TASKS_RCU Kconfig option be selected 2022-04-20 16:52:58 -07:00
link_iter.c bpf: Add bpf_link iterator 2022-05-10 11:20:45 -07:00
local_storage.c bpf: Consolidate spin_lock, timer management into btf_record 2022-11-03 22:19:40 -07:00
lpm_trie.c bpf: Use bpf_map_area_alloc consistently on bpf map creation 2022-08-10 11:50:43 -07:00
Makefile bpf: Implement cgroup storage available to non-cgroup-attached bpf progs 2022-10-25 23:19:19 -07:00
map_in_map.c bpf: Add comments for map BTF matching requirement for bpf_list_head 2022-11-17 19:22:14 -08:00
map_in_map.h
map_iter.c bpf: Introduce MEM_RDONLY flag 2021-12-18 13:27:41 -08:00
memalloc.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2022-10-24 13:44:11 -07:00
mmap_unlock_work.h bpf: Introduce helper bpf_find_vma 2021-11-07 11:54:51 -08:00
net_namespace.c net: Add includes masked by netdevice.h including uapi/bpf.h 2021-12-29 20:03:05 -08:00
offload.c bpf: Use bpf_map_area_alloc consistently on bpf map creation 2022-08-10 11:50:43 -07:00
percpu_freelist.c bpf: Initialize same number of free nodes for each pcpu_freelist 2022-11-11 12:05:14 -08:00
percpu_freelist.h
prog_iter.c
queue_stack_maps.c bpf: Remove unneeded memset in queue_stack_map creation 2022-08-10 11:48:22 -07:00
reuseport_array.c net: Fix suspicious RCU usage in bpf_sk_reuseport_detach() 2022-08-17 16:42:59 -07:00
ringbuf.c bpf: Rename MEM_ALLOC to MEM_RINGBUF 2022-11-14 21:52:45 -08:00
stackmap.c perf/bpf: Always use perf callchains if exist 2022-09-13 15:03:22 +02:00
syscall.c bpf: Add comments for map BTF matching requirement for bpf_list_head 2022-11-17 19:22:14 -08:00
sysfs_btf.c
task_iter.c - Yu Zhao's Multi-Gen LRU patches are here. They've been under test in 2022-10-10 17:53:04 -07:00
tnum.c bpf, tnums: Provably sound, faster, and more precise algorithm for tnum_mul 2021-06-01 13:34:15 +02:00
trampoline.c bpf: Remove prog->active check for bpf_lsm and bpf_iter 2022-10-25 23:11:46 -07:00
verifier.c bpf: Enable sleeptable support for cgrp local storage 2022-12-04 16:51:18 -08:00