linux/arch/powerpc/lib
Michael Ellerman 22db71bcba Merge branch 'topic/qspinlock' into next
Merge Nick's powerpc qspinlock implementation. From his cover letter:

This replaces the generic queued spinlock code (like s390 does) with our
own implementation.

Generic PV qspinlock code is causing latency / starvation regressions on
large systems that are resulting in hard lockups reported (mostly in
pathoogical cases). The generic qspinlock code has a number of issues
important for powerpc hardware and hypervisors that aren't easily solved
without changing code that would impact other architectures. Follow
s390's lead and implement our own for now.

Issues for powerpc using generic qspinlocks:
  - The previous lock value should not be loaded with simple loads, and
    need not be passed around from previous loads or cmpxchg results,
    because powerpc uses ll/sc-style atomics which can perform more
    complex operations that do not require this. powerpc implementations
    tend to prefer loads use larx for improved coherency performance.
  - The queueing process should absolutely minimise the number of stores
    to the lock word to reduce exclusive coherency probes, important for
    large system scalability. The pending logic is counter productive
    here.
  - Non-atomic unlock for paravirt locks is important (atomic
    instructions tend to still be more expensive than x86 CPUs).
  - Yielding to the lock owner is important in the oversubscribed
    paravirt case, which requires storing the owner CPU in the lock
    word.
  - More control of lock stealing for the paravirt case is important to
    keep latency down on large systems.
  - The lock acquisition operation should always be made with a special
    variant of atomic instructions with the lock hint bit set,
    including (especially) in the queueing paths. This is more a matter
    of adding more arch lock helpers so not an insurmountable problem
    for generic code.
2022-12-02 18:04:56 +11:00
..
alloc.c treewide: add checks for the return value of memblock_alloc*() 2019-03-12 10:04:02 -07:00
checksum_32.S powerpc/32: Remove remaining .stabs annotations 2022-02-07 21:03:10 +11:00
checksum_64.S ppc: propagate the calling conventions change down to csum_partial_copy_generic() 2020-08-20 15:45:22 -04:00
checksum_wrappers.c net: unexport csum_and_copy_{from,to}_user 2022-04-29 14:37:59 -07:00
code-patching.c powerpc/code-patching: Consolidate and cache per-cpu patching context 2022-12-02 17:54:06 +11:00
copy_32.S powerpc/32: Remove remaining .stabs annotations 2022-02-07 21:03:10 +11:00
copy_mc_64.S x86, powerpc: Rename memcpy_mcsafe() to copy_mc_to_{user, kernel}() 2020-10-06 11:18:04 +02:00
copypage_64.S powerpc/64: asm use consistent global variable declaration and access 2022-09-28 19:22:12 +10:00
copypage_power7.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156 2019-05-30 11:26:35 -07:00
copyuser_64.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
copyuser_power7.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156 2019-05-30 11:26:35 -07:00
crtsavres.S
div64.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
error-inject.c powerpc/64s: avoid reloading (H)SRR registers if they are still valid 2021-06-25 00:06:55 +10:00
feature-fixups-test.S powerpc: Test prefixed instructions in feature fixups 2020-05-19 00:11:02 +10:00
feature-fixups.c powerpc: Remove CONFIG_PPC_FSL_BOOK3E 2022-09-26 23:00:13 +10:00
hweight_64.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156 2019-05-30 11:26:35 -07:00
ldstfp.S powerpc updates for 5.3 2019-07-13 16:08:36 -07:00
locks.c powerpc/pseries: Move some PAPR paravirt functions to their own file 2020-07-26 23:34:26 +10:00
Makefile powerpc/qspinlock: powerpc qspinlock implementation 2022-12-02 17:48:02 +11:00
mem_64.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
memcmp_32.S powerpc/lib: optimise PPC32 memcmp 2018-06-04 00:39:21 +10:00
memcmp_64.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
memcpy_64.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
memcpy_power7.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156 2019-05-30 11:26:35 -07:00
pmem.c powerpc/pmem: Include pmem prototypes 2021-01-30 11:39:28 +11:00
qspinlock.c powerpc/qspinlock: add compile-time tuning adjustments 2022-12-02 17:48:50 +11:00
quad.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
restart_table.c powerpc/64s: add a table of implicit soft-masked addresses 2021-06-30 22:21:20 +10:00
rheap.c treewide: kmalloc() -> kmalloc_array() 2018-06-12 16:19:22 -07:00
sstep.c powerpc: add compile-time support for lbarx, lharx 2022-11-24 23:31:47 +11:00
string_32.S powerpc: Fix __clear_user() with KUAP enabled 2019-12-16 23:19:44 +11:00
string_64.S powerpc/64: asm use consistent global variable declaration and access 2022-09-28 19:22:12 +10:00
string.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
strlen_32.S powerpc/lib: Implement strlen() in assembly for PPC32 2018-08-07 21:49:30 +10:00
test_emulate_step_exec_instr.S powerpc: add definition for pt_regs offset within an interrupt frame 2022-12-02 17:54:08 +11:00
test_emulate_step.c powerpc/ppc-opcode: Define and use PPC_RAW_SETB() 2022-07-27 21:36:05 +10:00
test-code-patching.c powerpc/code-patching: Replace patch_instruction() by ppc_inst_write() in selftests 2021-12-23 22:36:58 +11:00
vmx-helper.c powerpc: Fix reschedule bug in KUAP-unlocked user copy 2022-10-18 22:46:19 +11:00
xor_vmx_glue.c lib/xor: make xor prototypes more friendly to compiler vectorization 2022-02-11 20:39:39 +11:00
xor_vmx.c lib/xor: make xor prototypes more friendly to compiler vectorization 2022-02-11 20:39:39 +11:00
xor_vmx.h lib/xor: make xor prototypes more friendly to compiler vectorization 2022-02-11 20:39:39 +11:00