iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
01eb01877f
linux/arch/powerpc/kernel
History
Nicholas Piggin 01eb01877f powerpc/64s: Fix restore_math unnecessarily changing MSR 
Before returning to user, if there are missing FP/VEC/VSX bits from the
user MSR then those registers had been saved and must be restored again
before use. restore_math will decide whether to restore immediately, or
skip the restore and let fp/vec/vsx unavailable faults demand load the
registers.

Each time restore_math restores one of the FP/VSX or VEC register sets
is loaded, an 8-bit counter is incremented (load_fp and load_vec). When
these wrap to zero, restore_math no longer restores that register set
until after they are next demand faulted.

It's quite usual for those counters to have different values, so if one
wraps to zero and restore_math no longer restores its registers or user
MSR bit but the other is not zero yet does not need to be restored
(because the kernel is not frequently using the FPU), then restore_math
will be called and it will also not return in the early exit check.
This causes msr_check_and_set to test and set the MSR at every kernel
exit despite having no work to do.

This can cause workloads (e.g., a NULL syscall microbenchmark) to run
fast for a time while both counters are non-zero, then slow down when
one of the counters reaches zero, then speed up again after the second
counter reaches zero. The cost is significant, about 10% slowdown on a
NULL syscall benchmark, and the jittery behaviour is very undesirable.

Fix this by having restore_math test all conditions first, and only
update MSR if we will be loading registers.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Link: https://lore.kernel.org/r/20200623234139.2262227-2-npiggin@gmail.com
2020-07-16 13:00:24 +10:00
..
ptrace powerpc/watchpoint: Use builtin ALIGN*() macros 2020-05-19 00:11:05 +10:00
syscalls vfs: add faccessat2 syscall 2020-05-14 16:44:25 +02:00
trace maccess: rename probe_kernel_{read,write} to copy_{from,to}_kernel_nofault 2020-06-17 10:57:41 -07:00
vdso32 powerpc/vdso32: Fallback on getres syscall when clock is unknown 2020-05-11 19:24:29 +10:00
vdso64 powerpc/vdso64: Switch from __get_datapage() to get_datapage inline macro 2020-07-15 12:04:40 +10:00
.gitignore .gitignore: add SPDX License Identifier 2020-03-25 11:50:48 +01:00
align.c powerpc: Define and use get_user_instr() et. al. 2020-05-19 00:10:37 +10:00
asm-offsets.c mm: don't include asm/pgtable.h if linux/mm.h is already included 2020-06-09 09:39:13 -07:00
audit.c
btext.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
cacheinfo.c powerpc/cacheinfo: Add per cpu per index shared_cpu_list 2020-07-15 11:07:20 +10:00
cacheinfo.h powerpc/cacheinfo: add cacheinfo_teardown, cacheinfo_rebuild 2019-06-15 16:52:06 +10:00
compat_audit.c
cpu_setup_6xx.S powerpc/32: Blacklist functions running with MMU disabled for kprobe 2020-06-02 20:59:11 +10:00
cpu_setup_44x.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
cpu_setup_fsl_booke.S powerpc/booke: Spelling s/date/data/ 2019-11-17 01:56:31 -06:00
cpu_setup_pa6t.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 333 2019-06-05 17:37:06 +02:00
cpu_setup_power.S powerpc: Add POWER10 architected mode 2020-06-02 20:59:20 +10:00
cpu_setup_ppc970.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
cputable.c powerpc: Add POWER10 architected mode 2020-06-02 20:59:20 +10:00
crash_dump.c powerpc: Add prefixed instructions to instruction data type 2020-05-19 00:10:39 +10:00
dawr.c powerpc/watchpoint: Disable all available watchpoints when !dawr_force_enable 2020-05-19 00:11:05 +10:00
dbell.c KVM: PPC: Book3S HV: use smp_mb() when setting/clearing host_ipi flag 2019-09-24 12:46:26 +10:00
dma-iommu.c powerpc updates for 5.4 2019-09-20 11:48:06 -07:00
dma-mask.c dma-mapping, powerpc: simplify the arch dma_set_mask override 2019-02-18 22:41:03 +11:00
dma-swiotlb.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
dt_cpu_ftrs.c powerpc/mm: Enable radix GTSE only if supported. 2020-07-16 13:00:21 +10:00
early_32.c powerpc/fsl_booke/32: implement KASLR infrastructure 2019-11-13 19:27:40 +11:00
eeh_cache.c powerpc/eeh_cache: Don't use pci_dn when inserting new ranges 2020-01-23 21:31:18 +11:00
eeh_dev.c powerpc/eeh: Add bdfn field to eeh_dev 2019-08-22 23:12:46 +10:00
eeh_driver.c powerpc/eeh: Fix deadlock handling dead PHB 2020-02-17 12:47:05 +11:00
eeh_event.c powerpc/eeh: Fix build with STACKTRACE=n 2019-09-14 00:01:14 +10:00
eeh_pe.c powerpc/eeh: Clean up EEH PEs after recovery finishes 2019-09-05 14:22:37 +10:00
eeh_sysfs.c powerpc/eeh_sysfs: Make clearing EEH_DEV_SYSFS saner 2020-01-23 21:31:19 +11:00
eeh.c powerpc/eeh: Release EEH device state synchronously 2020-05-18 21:58:44 +10:00
entry_32.S powerpc/entry32: Blacklist exception exit points for kprobe. 2020-06-02 20:59:13 +10:00
entry_64.S powerpc/64: indirect function call use bctrl rather than blrl in ret_from_kernel_thread 2020-06-22 10:37:55 +10:00
epapr_hcalls.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
epapr_paravirt.c powerpc: Use a datatype for instructions 2020-05-19 00:10:37 +10:00
exceptions-64e.S powerpc/64s: Implement interrupt exit logic in C 2020-04-01 13:42:14 +11:00
exceptions-64s.S powerpc/64s: Fix KVM interrupt using wrong save area 2020-06-16 12:52:43 +10:00
fadump.c powerpc/fadump: Account for memory_limit while reserving memory 2020-06-02 20:59:05 +10:00
firmware.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
fpu.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
fsl_booke_entry_mapping.S powerpc/fsl_booke/32: implement KASLR infrastructure 2019-11-13 19:27:40 +11:00
head_8xx.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
head_32.h Merge branch 'fixes' into next 2020-03-10 15:16:42 +11:00
head_32.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
head_40x.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
head_44x.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
head_64.S powerpc/64: Don't initialise init_task->thread.regs 2020-05-15 11:58:54 +10:00
head_booke.h powerpc: Replace _ALIGN_UP() by ALIGN() 2020-05-11 23:15:15 +10:00
head_fsl_booke.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
hw_breakpoint.c powerpc/watchpoint: Don't allow concurrent perf and ptrace events 2020-05-19 00:14:45 +10:00
idle_6xx.S powerpc/32: Blacklist functions running with MMU disabled for kprobe 2020-06-02 20:59:11 +10:00
idle_book3e.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
idle_book3s.S powerpc/64s: Reimplement power4_idle code in C 2020-01-16 14:59:37 +10:00
idle_e500.S powerpc/32: Blacklist functions running with MMU disabled for kprobe 2020-06-02 20:59:11 +10:00
idle.c powerpc/64s: Reimplement power4_idle code in C 2020-01-16 14:59:37 +10:00
ima_arch.c powerpc/ima: Fix secure boot rules in ima arch policy 2020-05-07 17:25:54 +10:00
io-workarounds.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
io.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
iomap.c
iommu.c Merge branch 'topic/ppc-kvm' into next 2019-08-30 09:52:57 +10:00
irq.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
isa-bridge.c powerpc: remove __ioremap_at and __iounmap_at 2020-06-02 10:59:10 -07:00
jump_label.c powerpc: Use a datatype for instructions 2020-05-19 00:10:37 +10:00
kgdb.c maccess: make get_kernel_nofault() check for minimal type compatibility 2020-06-18 12:10:37 -07:00
kprobes-ftrace.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156 2019-05-30 11:26:35 -07:00
kprobes.c maccess: rename probe_kernel_address to get_kernel_nofault 2020-06-18 11:14:40 -07:00
kvm_emul.S powerpc/kvm: Add ifdefs around template code 2019-09-14 00:04:40 +10:00
kvm.c powerpc/kvm: Explicitly mark kvm guest code as __init 2019-09-14 00:04:40 +10:00
l2cr_6xx.S powerpc/32: Blacklist functions running with MMU disabled for kprobe 2020-06-02 20:59:11 +10:00
legacy_serial.c tty/serial: Migrate 8250_fsl to use has_sysrq 2019-12-18 15:04:42 +01:00
Makefile powerpc/64s: Disable sanitisers for C syscall/interrupt entry/exit code 2020-05-29 21:12:09 +10:00
mce_power.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
mce.c powerpc/64s: machine check do not trace real-mode handler 2020-05-19 00:10:34 +10:00
misc_32.S powerpc/32: Blacklist functions running with MMU disabled for kprobe 2020-06-02 20:59:11 +10:00
misc_64.S Merge branch 'topic/kaslr-book3e32' into next 2019-11-14 19:23:33 +11:00
misc.S powerpc/32: Blacklist functions running with MMU disabled for kprobe 2020-06-02 20:59:11 +10:00
module_32.c powerpc: module_[32|64].c: replace swap function with built-in one 2020-05-11 23:15:14 +10:00
module_64.c maccess: rename probe_kernel_{read,write} to copy_{from,to}_kernel_nofault 2020-06-17 10:57:41 -07:00
module.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156 2019-05-30 11:26:35 -07:00
module.lds
msi.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
note.S powerpc: Add PowerPC Capabilities ELF note 2019-08-30 09:40:15 +10:00
nvram_64.c powerpc updates for 5.8 2020-06-05 12:39:30 -07:00
of_platform.c powerpc/eeh: Make early EEH init pseries specific 2020-03-25 12:09:39 +11:00
optprobes_head.S powerpc: Add prefixed instructions to instruction data type 2020-05-19 00:10:39 +10:00
optprobes.c powerpc: Add ppc_inst_as_u64() 2020-05-26 23:36:57 +10:00
paca.c powerpc/pseries/svm: Drop unused align argument in alloc_shared_lppaca() function 2020-06-22 10:37:59 +10:00
pci_32.c powerpc/64: Adjust order in pcibios_init() 2019-08-22 23:11:48 +10:00
pci_64.c powerpc updates for 5.8 2020-06-05 12:39:30 -07:00
pci_dn.c powerpc/pcidn: Warn when sriov pci_dn management is used incorrectly 2020-01-23 21:31:19 +11:00
pci_of_scan.c powerpc/pci: Remove pcibios_setup_bus_devices() 2020-01-06 16:25:29 +11:00
pci-common.c powerpc updates for 5.7 2020-04-05 11:12:59 -07:00
pci-hotplug.c powerpc/eeh: Release EEH device state synchronously 2020-05-18 21:58:44 +10:00
pmc.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
ppc32.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
ppc_save_regs.S powerpc: Improve ppc_save_regs() 2020-04-04 21:40:57 +11:00
proc_powerpc.c proc: convert everything to "struct proc_ops" 2020-02-04 03:05:26 +00:00
process.c powerpc/64s: Fix restore_math unnecessarily changing MSR 2020-07-16 13:00:24 +10:00
prom_init_check.sh powerpc/prom_init: Undo relocation before entering secure mode 2019-10-29 15:12:17 +11:00
prom_init.c powerpc/mm: Enable radix GTSE only if supported. 2020-07-16 13:00:21 +10:00
prom_parse.c
prom.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
reloc_32.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
reloc_64.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
rtas_flash.c proc: convert everything to "struct proc_ops" 2020-02-04 03:05:26 +00:00
rtas_pci.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
rtas-proc.c proc: convert everything to "struct proc_ops" 2020-02-04 03:05:26 +00:00
rtas-rtc.c
rtas.c powerpc/rtas: Implement reentrant rtas call 2020-06-02 20:59:08 +10:00
rtasd.c proc: convert everything to "struct proc_ops" 2020-02-04 03:05:26 +00:00
secure_boot.c powerpc: Detect the trusted boot state of the system 2019-11-12 12:25:49 +11:00
security.c powerpc: Use a macro for creating instructions from u32s 2020-05-19 00:10:36 +10:00
secvar-ops.c powerpc/powernv: Add OPAL API interface to access secure variable 2019-11-13 00:33:22 +11:00
secvar-sysfs.c powerpc: expose secure variables to userspace via sysfs 2019-11-13 00:33:22 +11:00
setup_32.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
setup_64.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
setup-common.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
setup.h powerpc/64: Prevent stack protection in early boot 2020-03-25 12:09:38 +11:00
signal_32.c powerpc/signal_32: Simplify loop in PPC64 save_general_regs() 2020-07-15 12:04:40 +10:00
signal_64.c powerpc/signal64: Don't opencode page prefaulting 2020-07-15 12:04:40 +10:00
signal.c powerpc/watchpoint: Convert thread_struct->hw_brk to an array 2020-05-19 00:11:05 +10:00
signal.h powerpc/64/sycall: Implement syscall entry/exit logic in C 2020-04-01 13:42:13 +11:00
smp-tbsync.c
smp.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
stacktrace.c kernel: rename show_stack_loglvl() => show_stack() 2020-06-09 09:39:13 -07:00
suspend.c PM: hibernate: powerpc: Expose pfn_is_nosave() prototype 2019-06-14 10:48:56 +02:00
swsusp_32.S powerpc/32: Blacklist functions running with MMU disabled for kprobe 2020-06-02 20:59:11 +10:00
swsusp_64.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 505 2019-06-19 17:11:22 +02:00
swsusp_asm64.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 505 2019-06-19 17:11:22 +02:00
swsusp_booke.S
swsusp.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
sys_ppc32.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
syscall_64.c powerpc/64s: Fix restore_math unnecessarily changing MSR 2020-07-16 13:00:24 +10:00
syscalls.c y2038: syscalls: change remaining timeval to __kernel_old_timeval 2019-11-15 14:38:29 +01:00
sysfs.c powerpc/sysfs: Show idle_purr and idle_spurr for every CPU 2020-04-30 12:35:26 +10:00
systbl_chk.sh treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
systbl.S powerpc/64/sycall: Implement syscall entry/exit logic in C 2020-04-01 13:42:13 +11:00
tau_6xx.c
time.c Revert "powerpc/64: irq_work avoid interrupt when called with hardware irqs enabled" 2020-04-03 16:55:34 +11:00
tm.S powerpc/tm: update comment about interrupt re-entrancy 2019-07-02 21:39:49 +10:00
traps.c mm: don't include asm/pgtable.h if linux/mm.h is already included 2020-06-09 09:39:13 -07:00
ucall.S powerpc/kernel: Add ucall_norets() ultravisor call handler 2019-08-30 09:40:15 +10:00
udbg_16550.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
udbg.c powerpc/udbg: Make it safe to call udbg_printf() always 2019-10-11 19:33:25 +11:00
uprobes.c powerpc: Add ppc_inst_next() 2020-05-26 23:36:51 +10:00
vdso.c mmap locking API: use coccinelle to convert mmap_sem rwsem call sites 2020-06-09 09:39:14 -07:00
vecemu.c powerpc: Define and use get_user_instr() et. al. 2020-05-19 00:10:37 +10:00
vector.S powerpc/32: Blacklist functions running with MMU disabled for kprobe 2020-06-02 20:59:11 +10:00
vmlinux.lds.S powerpc updates for 5.8 2020-06-05 12:39:30 -07:00
watchdog.c powerpc/watchdog: Use hrtimers for per-CPU heartbeat 2019-04-30 11:31:02 +10:00