linux/kernel/bpf
Alexei Starovoitov 1c2a088a66 bpf: x64: add JIT support for multi-function programs
Typical JIT does several passes over bpf instructions to
compute total size and relative offsets of jumps and calls.
With multitple bpf functions calling each other all relative calls
will have invalid offsets intially therefore we need to additional
last pass over the program to emit calls with correct offsets.
For example in case of three bpf functions:
main:
  call foo
  call bpf_map_lookup
  exit
foo:
  call bar
  exit
bar:
  exit

We will call bpf_int_jit_compile() indepedently for main(), foo() and bar()
x64 JIT typically does 4-5 passes to converge.
After these initial passes the image for these 3 functions
will be good except call targets, since start addresses of
foo() and bar() are unknown when we were JITing main()
(note that call bpf_map_lookup will be resolved properly
during initial passes).
Once start addresses of 3 functions are known we patch
call_insn->imm to point to right functions and call
bpf_int_jit_compile() again which needs only one pass.
Additional safety checks are done to make sure this
last pass doesn't produce image that is larger or smaller
than previous pass.

When constant blinding is on it's applied to all functions
at the first pass, since doing it once again at the last
pass can change size of the JITed code.

Tested on x64 and arm64 hw with JIT on/off, blinding on/off.
x64 jits bpf-to-bpf calls correctly while arm64 falls back to interpreter.
All other JITs that support normal BPF_CALL will behave the same way
since bpf-to-bpf call is equivalent to bpf-to-kernel call from
JITs point of view.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2017-12-17 20:34:36 +01:00
..
arraymap.c perf/bpf: Extend the perf_event_read_local() interface, a.k.a. "bpf: perf event change needed for subsequent bpf helpers" 2017-10-27 10:31:56 +02:00
bpf_lru_list.c bpf: lru: Lower the PERCPU_NR_SCANS from 16 to 4 2017-04-17 13:55:52 -04:00
bpf_lru_list.h bpf: Only set node->ref = 1 if it has not been set 2017-09-01 09:57:39 -07:00
cgroup.c bpf, cgroup: implement eBPF-based device controller for cgroup v2 2017-11-05 23:26:51 +09:00
core.c bpf: x64: add JIT support for multi-function programs 2017-12-17 20:34:36 +01:00
cpumap.c bpf: cpumap micro-optimization in cpu_map_enqueue 2017-11-02 16:13:14 +09:00
devmap.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2017-10-22 13:39:14 +01:00
disasm.c bpf: introduce function calls (function boundaries) 2017-12-17 20:34:35 +01:00
disasm.h bpf: move instruction printing into a separate file 2017-10-10 12:30:16 -07:00
hashtab.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2017-10-22 13:39:14 +01:00
helpers.c bpf: rename ARG_PTR_TO_STACK 2017-01-09 16:56:27 -05:00
inode.c bpf: Add file mode configuration into bpf maps 2017-10-20 13:32:59 +01:00
lpm_trie.c bpf: Add file mode configuration into bpf maps 2017-10-20 13:32:59 +01:00
Makefile bpf: offload: add infrastructure for loading programs for a specific netdev 2017-11-05 22:26:18 +09:00
map_in_map.c bpf: Add syscall lookup support for fd array and htab 2017-06-29 13:13:25 -04:00
map_in_map.h bpf: Add syscall lookup support for fd array and htab 2017-06-29 13:13:25 -04:00
offload.c bpf: offload: add a license header 2017-11-27 22:24:51 +01:00
percpu_freelist.c bpf: fix lockdep splat 2017-11-15 19:46:32 +09:00
percpu_freelist.h bpf: introduce percpu_freelist 2016-03-08 15:28:31 -05:00
sockmap.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2017-11-02 15:23:39 +09:00
stackmap.c bpf: Add file mode configuration into bpf maps 2017-10-20 13:32:59 +01:00
syscall.c bpf: x64: add JIT support for multi-function programs 2017-12-17 20:34:36 +01:00
tnum.c bpf/verifier: track signed and unsigned min/max values 2017-08-08 17:51:34 -07:00
verifier.c bpf: x64: add JIT support for multi-function programs 2017-12-17 20:34:36 +01:00