linux/arch/powerpc/mm
Nicholas Piggin e5f7cb58c2 powerpc/64s/radix: do not flush TLB when relaxing access
Radix flushes the TLB when updating ptes to increase permissiveness
of protection (increase access authority). Book3S does not require
TLB flushing in this case, and it is not done on hash. This patch
avoids the flush for radix.

>From Power ISA v3.0B, p.1090:

    Setting a Reference or Change Bit or Upgrading Access Authority
    (PTE Subject to Atomic Hardware Updates)

    If the only change being made to a valid PTE that is subject to
    atomic hardware updates is to set the Reference or Change bit to 1
    or to add access authorities, a simpler sequence suffices because
    the translation hardware will refetch the PTE if an access is
    attempted for which the only problems were reference and/or change
    bits needing to be set or insufficient access authority.

The nest MMU on POWER9 does not re-fetch the PTE after such an access
attempt before faulting, so address spaces with a coprocessor
attached will continue to flush in these cases.

This reduces tlbies for a kernel compile workload from 1.28M to 0.95M,
tlbiels from 20.17M 19.68M.

fork --fork --exec benchmark improved 2.77% (12000->12300).

Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-06-03 20:40:34 +10:00
..
8xx_mmu.c powerpc/mm/slice: Fix hugepage allocation at hint address on 8xx 2018-03-06 09:21:23 +11:00
40x_mmu.c
44x_mmu.c
copro_fault.c powerpc/mm: Add support for handling > 512TB address in SLB miss 2018-03-31 00:10:38 +11:00
dma-noncoherent.c powerpc/mm: Rename map_page() to map_kernel_page() on 32-bit 2017-06-05 19:59:03 +10:00
drmem.c powerpc/mm/drmem: Fix unexpected flag value in ibm,dynamic-memory-v2 2018-02-23 16:45:51 +11:00
dump_hashpagetable.c powerpc/64s: Replace CONFIG_PPC_STD_MMU_64 with CONFIG_PPC_BOOK3S_64 2017-11-06 16:48:14 +11:00
dump_linuxpagetables.c powerpc/mm: Introduce _PAGE_NA 2018-01-16 23:47:14 +11:00
fault.c powerpc/mm: Only read faulting instruction when necessary in do_page_fault() 2018-05-25 00:08:25 +10:00
fsl_booke_mmu.c
hash64_4k.c powerpc/mm/hash64: Store the slot information at the right offset for hugetlb 2018-02-13 22:37:48 +11:00
hash64_64k.c powerpc/mm/hash64: Store the slot information at the right offset for hugetlb 2018-02-13 22:37:48 +11:00
hash_low_32.S powerpc: fix location of two EXPORT_SYMBOL 2017-09-01 16:42:45 +10:00
hash_native_64.c Merge branch 'fixes' into next 2018-03-28 22:59:50 +11:00
hash_utils_64.c Merge branch 'topic/ppc-kvm' into next 2018-06-03 20:23:54 +10:00
highmem.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hugepage-hash64.c
hugetlbpage-book3e.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hugetlbpage-hash64.c powerpc/mm/hash64: Store the slot information at the right offset for hugetlb 2018-02-13 22:37:48 +11:00
hugetlbpage-radix.c powerpc updates for 4.15 2017-11-16 12:47:46 -08:00
hugetlbpage.c powerpc/fadump: Do not use hugepages when fadump is active 2018-05-03 23:09:25 +10:00
init_32.c powerpc/mm/32: Remove the reserved memory hack 2018-04-01 00:47:44 +11:00
init_64.c powerpc/mm/radix: Parse disable_radix commandline correctly. 2018-04-04 16:59:36 +10:00
init-common.c powerpc/mm: Fix crashes with 16G huge pages 2018-02-13 22:37:47 +11:00
Makefile powerpc: initial pkey plumbing 2018-01-20 21:45:03 +11:00
mem.c powerpc/mm: Flush cache on memory hot(un)plug 2018-04-24 09:46:56 +10:00
mmap.c exec: pass stack rlimit into mm layout functions 2018-04-11 10:28:37 -07:00
mmu_context_book3s64.c powerpc/mm: Implement helpers for pagetable fragment support at PMD level 2018-05-15 22:29:12 +10:00
mmu_context_hash32.c
mmu_context_iommu.c mm, migrate: remove reason argument from new_page_t 2018-04-11 10:28:32 -07:00
mmu_context_nohash.c powerpc/mm/slice: Simplify and optimise slice context initialisation 2018-03-13 23:43:05 +11:00
mmu_context.c powerpc, membarrier: Skip memory barrier in switch_mm() 2018-02-05 21:34:02 +01:00
mmu_decl.h powerpc/mm/32: Remove the reserved memory hack 2018-04-01 00:47:44 +11:00
numa.c Merge branch 'topic/paca' into next 2018-03-31 09:09:36 +11:00
pgtable_32.c powerpc/mm/32: Remove the reserved memory hack 2018-04-01 00:47:44 +11:00
pgtable_64.c powerpc/mm: Use page fragments for allocation page table at PMD level 2018-05-15 22:29:12 +10:00
pgtable-book3e.c
pgtable-book3s64.c powerpc/mm/radix: Change pte relax sequence to handle nest MMU hang 2018-06-03 20:40:34 +10:00
pgtable-hash64.c powerpc/mm: Use pmd_lockptr instead of opencoding it 2018-05-15 22:29:09 +10:00
pgtable-radix.c powerpc/64s/radix: do not flush TLB when relaxing access 2018-06-03 20:40:34 +10:00
pgtable.c powerpc/mm/radix: Change pte relax sequence to handle nest MMU hang 2018-06-03 20:40:34 +10:00
pkeys.c powerpc/mm/keys: Update documentation and remove unnecessary check 2018-04-04 15:23:09 +10:00
ppc_mmu_32.c powerpc/sparse: Fix plain integer as NULL pointer warning 2018-05-25 12:04:38 +10:00
slb_low.S powerpc/mm: Add support for handling > 512TB address in SLB miss 2018-03-31 00:10:38 +11:00
slb.c powerpc/mm: Add support for handling > 512TB address in SLB miss 2018-03-31 00:10:38 +11:00
slice.c powerpc/8xx: Fix build with hugetlbfs enabled 2018-04-11 12:00:23 +10:00
subpage-prot.c powerpc: Fix build by disabling attribute-alias warning for SYSCALL_DEFINEx 2018-06-03 20:40:24 +10:00
tlb_hash32.c powerpc/sparse: Fix plain integer as NULL pointer warning 2018-05-25 12:04:38 +10:00
tlb_hash64.c powerpc/mm: Add support for handling > 512TB address in SLB miss 2018-03-31 00:10:38 +11:00
tlb_low_64e.S
tlb_nohash_low.S powerpc/8xx: Getting rid of remaining use of CONFIG_8xx 2017-08-10 23:32:12 +10:00
tlb_nohash.c powerpc/mm/nohash: do not flush the entire mm when range is a single page 2018-01-27 20:24:44 +11:00
tlb-radix.c powerpc/mm/radix: implement LPID based TLB flushes to be used by KVM 2018-05-17 11:13:29 +10:00
vphn.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
vphn.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00