linux/arch/arm64/kernel
Mark Rutland ca708599ca arm64: use XPACLRI to strip PAC
Currently we strip the PAC from pointers using C code, which requires
generating bitmasks, and conditionally clearing/setting bits depending
on bit 55. We can do better by using XPACLRI directly.

When the logic was originally written to strip PACs from user pointers,
contemporary toolchains used for the kernel had assemblers which were
unaware of the PAC instructions. As stripping the PAC from userspace
pointers required unconditional clearing of a fixed set of bits (which
could be performed with a single instruction), it was simpler to
implement the masking in C than it was to make use of XPACI or XPACLRI.

When support for in-kernel pointer authentication was added, the
stripping logic was extended to cover TTBR1 pointers, requiring several
instructions to handle whether to clear/set bits dependent on bit 55 of
the pointer.

This patch simplifies the stripping of PACs by using XPACLRI directly,
as contemporary toolchains do within __builtin_return_address(). This
saves a number of instructions, especially where
__builtin_return_address() does not implicitly strip the PAC but is
heavily used (e.g. with tracepoints). As the kernel might be compiled
with an assembler without knowledge of XPACLRI, it is assembled using
the 'HINT #7' alias, which results in an identical opcode.

At the same time, I've split ptrauth_strip_insn_pac() into
ptrauth_strip_user_insn_pac() and ptrauth_strip_kernel_insn_pac()
helpers so that we can avoid unnecessary PAC stripping when pointer
authentication is not in use in userspace or kernel respectively.

The underlying xpaclri() macro uses inline assembly which clobbers x30.
The clobber causes the compiler to save/restore the original x30 value
in a frame record (protected with PACIASP and AUTIASP when in-kernel
authentication is enabled), so this does not provide a gadget to alter
the return address. Similarly this does not adversely affect unwinding
due to the presence of the frame record.

The ptrauth_user_pac_mask() and ptrauth_kernel_pac_mask() are exported
from the kernel in ptrace and core dumps, so these are retained. A
subsequent patch will move them out of <asm/compiler.h>.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Amit Daniel Kachhap <amit.kachhap@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Kristina Martsenko <kristina.martsenko@arm.com>
Cc: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20230412160134.306148-3-mark.rutland@arm.com
Signed-off-by: Will Deacon <will@kernel.org>
2023-04-13 12:27:11 +01:00
..
pi arm64: unwind: add asynchronous unwind tables to kernel and modules 2022-11-09 18:06:35 +00:00
probes arm64: kprobes: Drop ID map text from kprobes blacklist 2023-02-07 18:25:10 +00:00
vdso kbuild: upgrade the orphan section warning to an error if CONFIG_WERROR is set 2022-11-01 10:04:52 -07:00
vdso32 kbuild: upgrade the orphan section warning to an error if CONFIG_WERROR is set 2022-11-01 10:04:52 -07:00
.gitignore
acpi_numa.c arm64: numa: Don't check node against MAX_NUMNODES 2022-07-19 19:10:28 +01:00
acpi_parking_protocol.c treewide: Drop function_nocfi 2022-09-26 10:13:14 -07:00
acpi.c arm64: acpi: Fix possible memory leak of ffh_ctxt 2023-02-24 14:21:49 +00:00
alternative.c arm64: alternatives: add __init/__initconst to some functions/variables 2022-12-05 13:47:06 +00:00
armv8_deprecated.c arm64: armv8_deprecated: fix unused-function error 2022-11-25 12:16:22 +00:00
asm-offsets.c arm64: Implement HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS 2023-01-24 11:49:43 +00:00
cacheinfo.c ARM: 2023-02-25 11:30:21 -08:00
compat_alignment.c arm64: compat: Implement misalignment fixups for multiword loads 2022-09-06 09:34:53 +01:00
cpu_errata.c arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption 2023-01-06 17:14:55 +00:00
cpu_ops.c
cpu-reset.S arm64: Add types to indirect called assembly functions 2022-09-26 10:13:13 -07:00
cpufeature.c arm64 fixes: 2023-03-02 14:57:53 -08:00
cpuidle.c cpuidle, arm64: Fix the ARM64 cpuidle logic 2023-01-18 12:27:17 +01:00
cpuinfo.c Merge branches 'for-next/sysreg', 'for-next/sme', 'for-next/kselftest', 'for-next/misc', 'for-next/sme2', 'for-next/tpidr2', 'for-next/scs', 'for-next/compat-hwcap', 'for-next/ftrace', 'for-next/efi-boot-mmu-on', 'for-next/ptrauth' and 'for-next/pseudo-nmi', remote-tracking branch 'arm64/for-next/perf' into for-next/core 2023-02-10 18:51:49 +00:00
crash_core.c arm64: crash_core: Export MODULES, VMALLOC, and VMEMMAP ranges 2022-03-07 21:25:47 +00:00
crash_dump.c vmcore: convert copy_oldmem_page() to take an iov_iter 2022-04-29 14:37:59 -07:00
debug-monitors.c arm64/sysreg: Add _EL1 into ID_AA64DFR0_EL1 definition names 2022-09-16 12:38:57 +01:00
efi-header.S
efi-rt-wrapper.S arm64: efi: Avoid workqueue to check whether EFI runtime is live 2023-01-16 15:27:31 +01:00
efi.c A healthy mix of EFI contributions this time: 2023-02-23 14:41:48 -08:00
elfcore.c arm64: mte: Avoid the racy walk of the vma list during core dump 2023-01-05 15:12:12 +00:00
entry-common.c Merge branch 'for-next/undef-traps' into for-next/core 2022-12-06 11:34:25 +00:00
entry-fpsimd.S arm64/sme: Implement context switching for ZT0 2023-01-20 12:23:06 +00:00
entry-ftrace.S arm64: Implement HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS 2023-01-24 11:49:43 +00:00
entry.S Merge branches 'for-next/sysreg', 'for-next/sme', 'for-next/kselftest', 'for-next/misc', 'for-next/sme2', 'for-next/tpidr2', 'for-next/scs', 'for-next/compat-hwcap', 'for-next/ftrace', 'for-next/efi-boot-mmu-on', 'for-next/ptrauth' and 'for-next/pseudo-nmi', remote-tracking branch 'arm64/for-next/perf' into for-next/core 2023-02-10 18:51:49 +00:00
fpsimd.c arm64 fixes: 2023-03-02 14:57:53 -08:00
ftrace.c arm64: Implement HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS 2023-01-24 11:49:43 +00:00
head.S arm64: fix .idmap.text assertion for large kernels 2023-02-20 18:23:35 +00:00
hibernate-asm.S arm64: kexec: install a copy of the linear-map 2021-10-01 13:31:00 +01:00
hibernate.c arm64: mte: Fix/clarify the PG_mte_tagged semantics 2022-11-29 09:26:07 +00:00
hw_breakpoint.c arm64: Treat ESR_ELx as a 64-bit register 2022-04-29 19:26:27 +01:00
hyp-stub.S arm64/sme: Fix __finalise_el2 SMEver check 2023-02-06 16:34:29 +00:00
idle.c arch/idle: Change arch_cpu_idle() behavior: always exit with IRQs disabled 2023-01-13 11:48:15 +01:00
idreg-override.c arm64/sme: Enable host kernel to access ZT0 2023-01-20 12:23:06 +00:00
image-vars.h Merge branches 'for-next/sysreg', 'for-next/sme', 'for-next/kselftest', 'for-next/misc', 'for-next/sme2', 'for-next/tpidr2', 'for-next/scs', 'for-next/compat-hwcap', 'for-next/ftrace', 'for-next/efi-boot-mmu-on', 'for-next/ptrauth' and 'for-next/pseudo-nmi', remote-tracking branch 'arm64/for-next/perf' into for-next/core 2023-02-10 18:51:49 +00:00
image.h
io.c
irq.c Merge branch 'for-next/stacks' into for-next/core 2022-12-06 11:26:40 +00:00
jump_label.c jump_label: make initial NOP patching the special case 2022-06-24 09:48:55 +02:00
kaslr.c arm64: kaslr: don't pretend KASLR is enabled if offset < MIN_KIMG_ALIGN 2023-02-28 11:21:04 +00:00
kexec_image.c arm64: kexec_file: use more system keyrings to verify kernel image signature 2022-07-15 12:21:16 -04:00
kgdb.c arm64: Treat ESR_ELx as a 64-bit register 2022-04-29 19:26:27 +01:00
kuser32.S arm64: compat: Move kuser32.S to .rodata section 2022-06-23 16:01:42 +01:00
machine_kexec_file.c arm64/kexec: Fix missing extra range for crashkres_low. 2022-09-01 11:50:00 +01:00
machine_kexec.c treewide: Drop function_nocfi 2022-09-26 10:13:14 -07:00
Makefile EFI updates for v6.2: 2022-12-13 14:31:47 -08:00
module-plts.c arm64: module: Make plt_equals_entry() static 2022-09-29 17:47:18 +01:00
module.c Merge branch 'for-next/ftrace' into for-next/core 2022-12-06 11:07:39 +00:00
mte.c arm64: mte: Lock a page for MTE tag initialisation 2022-11-29 09:26:07 +00:00
paravirt.c arm64: paravirt: remove conduit check in has_pv_steal_clock 2022-11-09 18:11:56 +00:00
patch-scs.c arm64: Apply dynamic shadow call stack patching in two passes 2023-01-26 17:54:15 +00:00
patching.c arm64: patching: Add aarch64_insn_write_literal_u64() 2023-01-24 11:49:43 +00:00
pci.c arm64: PCI: Support root bridge preparation for Hyper-V 2021-08-23 10:59:26 +01:00
perf_callchain.c arm64: use XPACLRI to strip PAC 2023-04-13 12:27:11 +01:00
perf_event.c arm64: perf: reject CHAIN events at creation time 2023-02-16 21:23:52 +00:00
perf_regs.c perf: arm64: Add SVE vector granule register to user regs 2022-09-22 15:06:02 +01:00
pointer_auth.c arm64: move preemption disablement to prctl handlers 2021-07-28 18:33:49 +01:00
process.c arm64: use XPACLRI to strip PAC 2023-04-13 12:27:11 +01:00
proton-pack.c arm64: factor out EL1 SSBS emulation hook 2022-11-15 13:46:18 +00:00
psci.c treewide: Drop function_nocfi 2022-09-26 10:13:14 -07:00
ptrace.c arm64 updates for 6.3: 2023-02-21 15:27:48 -08:00
reloc_test_core.c ARM64: reloc_test: add __init/__exit annotations to module init/exit funcs 2022-09-29 18:04:25 +01:00
reloc_test_syms.S
relocate_kernel.S arm64: kexec: load from kimage prior to clobbering 2022-05-17 14:25:35 +01:00
return_address.c arm64: Make return_address() use arch_stack_walk() 2021-12-10 14:06:04 +00:00
sdei.c arm64: implement dynamic shadow call stack for Clang 2022-11-09 18:06:35 +00:00
setup.c arm64: head: record the MMU state at primary entry 2023-01-24 11:51:07 +00:00
signal32.c signal: Deliver SIGTRAP on perf event asynchronously if blocked 2022-04-22 12:14:05 +02:00
signal.c arm64 updates for 6.3: 2023-02-21 15:27:48 -08:00
sigreturn32.S arm64: compat: Move sigreturn32.S to .rodata section 2022-07-01 12:05:45 +01:00
sleep.S arm64: head: Clean the ID map and the HYP text to the PoC if needed 2023-01-24 11:51:08 +00:00
smccc-call.S arm64: smccc: Save lr before calling __arm_smccc_sve_check() 2021-07-21 11:23:25 +01:00
smp_spin_table.c treewide: Drop function_nocfi 2022-09-26 10:13:14 -07:00
smp.c arm64, smp: Remove trace_.*_rcuidle() usage 2023-01-13 11:48:15 +01:00
stacktrace.c arm64: use XPACLRI to strip PAC 2023-04-13 12:27:11 +01:00
suspend.c cpuidle, arm64: Fix the ARM64 cpuidle logic 2023-01-18 12:27:17 +01:00
sys32.c
sys_compat.c arm64: compat: Do not treat syscall number as ESR_ELx for a bad syscall 2022-04-29 19:26:27 +01:00
sys.c
syscall.c arm64/sme: Optimise SME exit on syscall entry 2023-01-12 17:09:21 +00:00
time.c arm64: Make profile_pc() use arch_stack_walk() 2021-12-10 14:06:04 +00:00
topology.c RISC-V Patches for the 6.1 Merge Window, Part 1 2022-10-09 13:24:01 -07:00
trace-events-emulation.h
traps.c arm64 fixes: 2023-03-02 14:57:53 -08:00
vdso32-wrap.S arm64: do not descend to vdso directories twice 2021-01-20 12:18:46 +00:00
vdso-wrap.S arm64: do not descend to vdso directories twice 2021-01-20 12:18:46 +00:00
vdso.c mm: remove zap_page_range and create zap_vma_pages 2023-01-18 17:12:55 -08:00
vmlinux.lds.S arm64 updates for 6.3: 2023-02-21 15:27:48 -08:00