iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9cacf81f81
linux/kernel/bpf
History
Stanislav Fomichev 9cacf81f81 bpf: Remove extra lock_sock for TCP_ZEROCOPY_RECEIVE 
Add custom implementation of getsockopt hook for TCP_ZEROCOPY_RECEIVE.
We skip generic hooks for TCP_ZEROCOPY_RECEIVE and have a custom
call in do_tcp_getsockopt using the on-stack data. This removes
3% overhead for locking/unlocking the socket.

Without this patch:
     3.38%     0.07%  tcp_mmap  [kernel.kallsyms]  [k] __cgroup_bpf_run_filter_getsockopt
            |
             --3.30%--__cgroup_bpf_run_filter_getsockopt
                       |
                        --0.81%--__kmalloc

With the patch applied:
     0.52%     0.12%  tcp_mmap  [kernel.kallsyms]  [k] __cgroup_bpf_run_filter_getsockopt_kern

Note, exporting uapi/tcp.h requires removing netinet/tcp.h
from test_progs.h because those headers have confliciting
definitions.

Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Link: https://lore.kernel.org/bpf/20210115163501.805133-2-sdf@google.com
2021-01-20 14:23:00 -08:00
..
preload bpf: Remove unnecessary <argp.h> include from preload/iterators 2021-01-08 13:39:24 -08:00
arraymap.c bpf: Eliminate rlimit-based memory accounting for arraymap maps 2020-12-02 18:32:46 -08:00
bpf_inode_storage.c bpf: Fix typo in bpf_inode_storage.c 2021-01-12 16:07:57 +01:00
bpf_iter.c bpf: Permit cond_resched for some iterators 2020-10-28 14:54:31 -07:00
bpf_local_storage.c bpf: Eliminate rlimit-based memory accounting for bpf local storage maps 2020-12-02 18:32:47 -08:00
bpf_lru_list.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
bpf_lru_list.h bpf: Fix a typo "inacitve" -> "inactive" 2020-04-06 21:54:10 +02:00
bpf_lsm.c bpf: Add a BPF helper for getting the IMA hash of an inode 2020-11-26 00:04:04 +01:00
bpf_struct_ops_types.h bpf: tcp: Support tcp_congestion_ops in bpf 2020-01-09 08:46:18 -08:00
bpf_struct_ops.c bpf: Eliminate rlimit-based memory accounting for bpf_struct_ops maps 2020-12-02 18:32:46 -08:00
bpf_task_storage.c bpf: Local storage helpers should check nullness of owner ptr passed 2021-01-12 16:07:56 +01:00
btf.c bpf: Permit size-0 datasec 2021-01-20 14:14:09 -08:00
cgroup.c bpf: Remove extra lock_sock for TCP_ZEROCOPY_RECEIVE 2021-01-20 14:23:00 -08:00
core.c bpf: Add bitwise atomic instructions 2021-01-14 18:34:29 -08:00
cpumap.c net, xdp: Introduce __xdp_build_skb_from_frame utility routine 2021-01-20 14:10:35 -08:00
devmap.c bpf: Eliminate rlimit-based memory accounting for devmap maps 2020-12-02 18:32:46 -08:00
disasm.c bpf: Add bitwise atomic instructions 2021-01-14 18:34:29 -08:00
disasm.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 295 2019-06-05 17:36:38 +02:00
dispatcher.c bpf: Remove bpf_image tree 2020-03-13 12:49:52 -07:00
hashtab.c bpf: Add schedule point in htab_init_buckets() 2020-12-22 00:14:31 +01:00
helpers.c bpf: Fix helper bpf_map_peek_elem_proto pointing to wrong callback 2021-01-19 22:04:08 +01:00
inode.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2020-09-22 16:45:34 -07:00
local_storage.c bpf: Eliminate rlimit-based memory accounting for cgroup storage maps 2020-12-02 18:32:46 -08:00
lpm_trie.c bpf: Eliminate rlimit-based memory accounting for lpm_trie maps 2020-12-02 18:32:46 -08:00
Makefile Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2020-11-14 09:13:41 -08:00
map_in_map.c bpf: Relax max_entries check for most of the inner map types 2020-08-28 15:41:30 +02:00
map_in_map.h bpf: Add map_meta_equal map ops 2020-08-28 15:41:30 +02:00
map_iter.c bpf: Implement link_query callbacks in map element iterators 2020-08-21 14:01:39 -07:00
net_namespace.c bpf: Add support for forced LINK_DETACH command 2020-08-01 20:38:28 -07:00
offload.c bpf, offload: Replace bitwise AND by logical AND in bpf_prog_offload_info_fill 2020-02-17 16:53:49 +01:00
percpu_freelist.c bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI 2020-10-06 00:04:11 +02:00
percpu_freelist.h bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI 2020-10-06 00:04:11 +02:00
prog_iter.c bpf: Refactor bpf_iter_reg to have separate seq_info member 2020-07-25 20:16:32 -07:00
queue_stack_maps.c bpf: Eliminate rlimit-based memory accounting for queue_stack_maps maps 2020-12-02 18:32:46 -08:00
reuseport_array.c bpf: Eliminate rlimit-based memory accounting for reuseport_array maps 2020-12-02 18:32:47 -08:00
ringbuf.c bpf: Eliminate rlimit-based memory accounting for bpf ringbuffer 2020-12-02 18:32:47 -08:00
stackmap.c bpf: Add size arg to build_id_parse function 2021-01-14 19:29:58 -08:00
syscall.c bpf: Prevent double bpf_prog_put call from bpf_tracing_prog_attach 2021-01-12 00:17:34 +01:00
sysfs_btf.c bpf: Load and verify kernel module BTFs 2020-11-10 15:25:53 -08:00
task_iter.c bpf: Fix a task_iter bug caused by a merge conflict resolution 2021-01-03 01:41:32 +01:00
tnum.c bpf: Verifier, do explicit ALU32 bounds tracking 2020-03-30 14:59:53 -07:00
trampoline.c bpf: Move prog->aux->linked_prog and trampoline into bpf_link on attach 2020-09-29 13:09:23 -07:00
verifier.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2021-01-20 12:16:11 -08:00