linux/arch/s390/kernel
Heiko Carstens 82d3edb50a s390/cpum_sf: add READ_ONCE() semantics to compare and swap loops
The current cmpxchg_double() loops within the perf hw sampling code do not
have READ_ONCE() semantics to read the old value from memory. This allows
the compiler to generate code which reads the "old" value several times
from memory, which again allows for inconsistencies.

For example:

        /* Reset trailer (using compare-double-and-swap) */
        do {
                te_flags = te->flags & ~SDB_TE_BUFFER_FULL_MASK;
                te_flags |= SDB_TE_ALERT_REQ_MASK;
        } while (!cmpxchg_double(&te->flags, &te->overflow,
                 te->flags, te->overflow,
                 te_flags, 0ULL));

The compiler could generate code where te->flags used within the
cmpxchg_double() call may be refetched from memory and which is not
necessarily identical to the previous read version which was used to
generate te_flags. Which in turn means that an incorrect update could
happen.

Fix this by adding READ_ONCE() semantics to all cmpxchg_double()
loops. Given that READ_ONCE() cannot generate code on s390 which atomically
reads 16 bytes, use a private compare-and-swap-double implementation to
achieve that.

Also replace cmpxchg_double() with the private implementation to be able to
re-use the old value within the loops.

As a side effect this converts the whole code to only use bit fields
to read and modify bits within the hws trailer header.

Reported-by: Alexander Gordeev <agordeev@linux.ibm.com>
Acked-by: Alexander Gordeev <agordeev@linux.ibm.com>
Acked-by: Hendrik Brueckner <brueckner@linux.ibm.com>
Reviewed-by: Thomas Richter <tmricht@linux.ibm.com>
Cc: <stable@vger.kernel.org>
Link: https://lore.kernel.org/linux-s390/Y71QJBhNTIatvxUT@osiris/T/#ma14e2a5f7aa8ed4b94b6f9576799b3ad9c60f333
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2023-01-11 15:28:34 +01:00
..
syscalls arch: syscalls: simplify uapi/kapi directory creation 2022-03-31 12:03:46 +09:00
vdso32
vdso64
.gitignore
abs_lowcore.c s390/smp: rework absolute lowcore access 2022-09-14 16:46:00 +02:00
alternative.c s390/alternatives: remove padding generation code 2022-05-17 15:16:28 +02:00
asm-offsets.c s390/entry: sort out physical vs virtual pointers usage in sie64a 2022-10-26 14:27:41 +02:00
audit.c
audit.h
cache.c
compat_audit.c
compat_linux.c
compat_linux.h s390/compat: cleanup compat_linux.h header file 2022-05-06 20:45:16 +02:00
compat_ptrace.h
compat_signal.c
cpcmd.c
cpufeature.c s390/uvdevice: autoload module based on CPU facility 2022-07-19 16:18:49 +02:00
crash_dump.c iov_iter work; most of that is about getting rid of 2022-12-12 18:29:54 -08:00
debug.c s390/debug: remove function type cast 2022-11-23 16:24:07 +01:00
diag.c
dis.c
dumpstack.c
early_printk.c
early.c s390: move from strlcpy with unused retval to strscpy 2022-08-30 22:00:33 +02:00
earlypgm.S s390: generate register offsets into pt_regs automatically 2022-05-25 11:46:02 +02:00
ebcdic.c
entry.h s390: simplify early program check handler 2022-05-25 11:46:02 +02:00
entry.S ARM64: 2022-12-15 11:12:21 -08:00
fpu.c s390/vx: add vx-insn.h wrapper include file 2022-12-06 16:18:23 +01:00
ftrace.c ftrace: Remove return value of ftrace_arch_modify_*() 2022-05-26 21:13:00 -04:00
ftrace.h
guarded_storage.c
head64.S
idle.c
ima_arch.c
ipl_vmparm.c
ipl.c s390/ipl: use octal values instead of S_* macros 2022-12-06 16:18:22 +01:00
irq.c s390/irq: utilize RCU instead of irq_lock_sparse() in show_msi_interrupt() 2022-04-27 12:53:34 +02:00
jump_label.c jump_label: make initial NOP patching the special case 2022-06-24 09:48:55 +02:00
kdebugfs.c
kexec_elf.c
kexec_image.c
kprobes_insn_page.S
kprobes.c s390/kprobes: define insn cache ops within private header file 2022-11-23 16:24:07 +01:00
kprobes.h s390/kprobes: define insn cache ops within private header file 2022-11-23 16:24:07 +01:00
lgr.c
machine_kexec_file.c s390/kexec: fix ipl report address for kdump 2023-01-10 14:37:02 +01:00
machine_kexec_reloc.c
machine_kexec.c s390/smp: rework absolute lowcore access 2022-09-14 16:46:00 +02:00
Makefile Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
mcount.S
module.c jump_label: mips: move module NOP patching into arch code 2022-06-24 09:48:55 +02:00
nmi.c s390/nmi: get rid of private slab cache 2022-12-06 16:18:26 +01:00
nospec-branch.c
nospec-sysfs.c
numa.c
os_info.c s390/mm: uninline copy_oldmem_kernel() function 2022-09-14 16:46:01 +02:00
perf_cpum_cf_common.c
perf_cpum_cf_events.c s390/cpumf: add new extended counter set for IBM z16 2022-05-16 10:58:33 +02:00
perf_cpum_cf.c perf: Use sample_flags for raw_data 2022-09-27 22:50:24 +02:00
perf_cpum_sf.c s390/cpum_sf: add READ_ONCE() semantics to compare and swap loops 2023-01-11 15:28:34 +01:00
perf_event.c s390/stack: add union to reflect kvm stack slot usages 2022-06-01 12:03:17 +02:00
perf_pai_crypto.c Perf events updates for v6.2: 2022-12-12 15:19:38 -08:00
perf_pai_ext.c Perf events updates for v6.2: 2022-12-12 15:19:38 -08:00
perf_regs.c
process.c treewide: use get_random_u32_below() instead of deprecated function 2022-11-18 02:15:15 +01:00
processor.c s390/cpufeature: rework to allow more than only hwcap bits 2022-07-19 16:18:49 +02:00
ptrace.c ptrace: Cleanups for v5.18 2022-03-28 17:29:53 -07:00
reipl.S
relocate_kernel.S s390/kexec: set end-of-ipl flag in last diag308 call 2022-04-25 13:54:12 +02:00
runtime_instr.c
setup.c random: do not include <asm/archrandom.h> from random.h 2022-12-20 03:13:45 +01:00
signal.c ptrace: Cleanups for v5.18 2022-03-28 17:29:53 -07:00
smp.c s390/mm: uninline copy_oldmem_kernel() function 2022-09-14 16:46:01 +02:00
stacktrace.c
sthyi.c
syscall.c
sysinfo.c
text_amode31.S
time.c s390/stp: clock_delta should be signed 2022-05-11 14:40:57 +02:00
topology.c
trace.c
traps.c s390/traps: improve panic message for translation-specification exception 2022-03-27 22:18:38 +02:00
unwind_bc.c s390/unwind: avoid duplicated unwinding entries for kretprobes 2022-03-27 22:18:39 +02:00
uprobes.c
uv.c KVM: s390: pv: avoid export before import if possible 2022-11-23 09:06:50 +00:00
vdso.c Random number generator updates for Linux 6.2-rc1. 2022-12-12 16:22:22 -08:00
vmlinux.lds.S s390: define RUNTIME_DISCARD_EXIT to fix link error with GNU ld < 2.36 2023-01-06 16:38:21 +01:00
vtime.c