iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
f901b4bfd03ed234f72f28e27d965569531dc505
linux/include/asm-generic
History
Jann Horn 4a1cdb49d0 mm/khugepaged: fix GUP-fast interaction by sending IPI 
commit 2ba99c5e08 upstream.

Since commit 70cbc3cc78 ("mm: gup: fix the fast GUP race against THP
collapse"), the lockless_pages_from_mm() fastpath rechecks the pmd_t to
ensure that the page table was not removed by khugepaged in between.

However, lockless_pages_from_mm() still requires that the page table is
not concurrently freed.  Fix it by sending IPIs (if the architecture uses
semi-RCU-style page table freeing) before freeing/reusing page tables.

Link: https://lkml.kernel.org/r/20221129154730.2274278-2-jannh@google.com
Link: https://lkml.kernel.org/r/20221128180252.1684965-2-jannh@google.com
Link: https://lkml.kernel.org/r/20221125213714.4115729-2-jannh@google.com
Fixes: ba76149f47 ("thp: khugepaged")
Signed-off-by: Jann Horn <jannh@google.com>
Reviewed-by: Yang Shi <shy828301@gmail.com>
Acked-by: David Hildenbrand <david@redhat.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
[manual backport: two of the three places in khugepaged that can free
ptes were refactored into a common helper between 5.15 and 6.0]
Signed-off-by: Jann Horn <jannh@google.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2022-12-14 11:31:55 +01:00
..
bitops
locking/atomic: Make test_and_*_bit() ordered on failure
2022-08-25 11:38:10 +02:00
vdso
lib/vdso: Avoid highres update if clocksource is not VDSO capable
2020-02-17 20:12:17 +01:00
asm-offsets.h
…
asm-prototypes.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
atomic64.h
locking/atomic: Use s64 for atomic64
2019-06-03 12:32:56 +02:00
atomic-instrumented.h
locking/atomics: Use read-write instrumentation for atomic RMWs
2020-08-24 15:09:59 -07:00
atomic-long.h
asm-generic/atomic: Use __always_inline for pure wrappers
2020-01-07 07:47:23 -08:00
atomic.h
locking/atomic: Move ATOMIC_INIT into linux/types.h
2020-07-29 16:14:18 +02:00
audit_change_attr.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
audit_dir_write.h
audit: Avoid build failures on systems without renameat
2018-01-30 19:07:54 -08:00
audit_read.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
audit_signal.h
…
audit_write.h
Merge tag 'audit-pr-20171113' of git://git.kernel.org/pub/scm/linux/kernel/git/pcmoore/audit
2017-11-15 13:28:48 -08:00
barrier.h
compiler.h: fix barrier_data() on clang
2020-11-14 11:26:03 -08:00
bitops.h
asm-generic/bitops: Update stale comment
2020-03-06 11:06:19 +01:00
bitsperlong.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
bug.h
treewide: Convert macro and uses of __section(foo) to __section("foo")
2020-10-25 14:51:49 -07:00
bugs.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
cache.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
cacheflush.h
make asm-generic/cacheflush.h more standalone
2020-06-26 00:27:37 -07:00
checksum.h
unify generic instances of csum_partial_copy_nocheck()
2020-08-20 15:45:14 -04:00
cmpxchg-local.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
cmpxchg.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
compat.h
compat: lift compat_s64 and compat_u64 to <asm-generic/compat.h>
2020-09-17 13:00:46 -04:00
current.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
delay.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
device.h
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 428
2019-06-05 17:37:16 +02:00
div64.h
Merge tag 'asm-generic-5.4' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/asm-generic
2019-09-17 14:30:30 -07:00
dma-mapping.h
dma-mapping: bypass indirect calls for dma-direct
2018-12-13 21:06:18 +01:00
dma.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
early_ioremap.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
emergency-restart.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
error-injection.h
treewide: Convert macro and uses of __section(foo) to __section("foo")
2020-10-25 14:51:49 -07:00
exec.h
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36
2019-05-24 17:27:11 +02:00
export.h
export.h: reduce __ksymtab_strings string duplication by using "MS" section flags
2019-12-16 10:35:33 +01:00
extable.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
fb.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
fixmap.h
mm: introduce common STRUCT_PAGE_MAX_SHIFT define
2018-12-14 15:05:45 -08:00
flat.h
binfmt_flat: remove the persistent argument from flat_get_addr_from_rp
2019-06-24 09:16:47 +10:00
ftrace.h
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500
2019-06-19 17:09:55 +02:00
futex.h
generic arch_futex_atomic_op_inuser() doesn't need access_ok()
2020-03-27 23:58:55 -04:00
getorder.h
asm-generic: fix -Wtype-limits compiler warnings
2019-08-03 07:02:01 -07:00
gpio.h
gpio: Avoid kernel.h inclusion where it's possible
2020-02-10 12:58:36 +01:00
hardirq.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
hugetlb.h
mm: Allow arches to provide ptep_get()
2020-06-20 22:14:53 +10:00
hw_irq.h
…
hyperv-tlfs.h
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
2020-06-03 15:13:47 -07:00
ide_iops.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
int-ll64.h
int-ll64.h: define u{8,16,32,64} and s{8,16,32,64} based on uapi header
2018-06-07 17:34:38 -07:00
io.h
asm-generic/io.h: Fix !CONFIG_GENERIC_IOMAP pci_iounmap() implementation
2020-10-05 09:44:16 +01:00
ioctl.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
iomap.h
iomap: constify ioreadX() iomem argument (as in generic implementation)
2020-08-14 19:56:57 -07:00
irq_regs.h
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152
2019-05-30 11:26:32 -07:00
irq_work.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
irq.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
irqflags.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
Kbuild
local64.h: make <asm/local64.h> mandatory
2021-01-12 20:18:16 +01:00
kdebug.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
kmap_types.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
kprobes.h
treewide: Convert macro and uses of __section(foo) to __section("foo")
2020-10-25 14:51:49 -07:00
kvm_para.h
KVM: Introduce paravirtualization hints and KVM_HINTS_DEDICATED
2018-03-06 18:40:44 +01:00
kvm_types.h
KVM: Move x86's version of struct kvm_mmu_memory_cache to common code
2020-07-09 13:29:42 -04:00
linkage.h
…
local64.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
local.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
mcs_spinlock.h
…
memory_model.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
mm_hooks.h
mm: remove arch_bprm_mm_init() hook
2020-01-23 10:41:16 -08:00
mm-arch-hooks.h
…
mmiowb_types.h
asm-generic/mmiowb: Add generic implementation of mmiowb() tracking
2019-04-08 11:59:39 +01:00
mmiowb.h
asm-generic/mmiowb: Allow mmiowb_set_pending() when preemptible()
2020-07-17 10:02:03 +01:00
mmu_context.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
mmu.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
module.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
module.lds.h
kbuild: preprocess module linker script
2020-09-25 00:36:41 +09:00
mshyperv.h
Drivers: hv: vmbus: Add parsing of VMbus interrupt in ACPI DSDT
2020-10-14 19:14:51 +00:00
msi.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
page.h
asm-generic/page.h: fix typo in #error text requiring a real asm/page.h
2019-02-22 21:53:25 +01:00
param.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
parport.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
pci_iomap.h
mn10300: Remove the architecture
2018-03-09 23:19:56 +01:00
pci.h
PCI: remove PCI_DMA_BUS_IS_PHYS
2018-05-07 07:15:41 +02:00
percpu.h
asm-generic: percpu: avoid Wshadow warning
2020-10-26 23:54:48 +00:00
pgalloc.h
asm-generic: pgalloc.h: use correct #ifdef to enable pud_alloc_one()
2020-08-14 19:56:55 -07:00
pgtable_uffd.h
userfaultfd: wp: add pmd_swp_*uffd_wp() helpers
2020-04-07 10:43:39 -07:00
pgtable-nop4d.h
asm-generic/tlb: stub out p4d_free_tlb() if nop4d ...
2019-12-01 06:29:19 -08:00
pgtable-nopmd.h
mm: consolidate pte_index() and pte_offset_*() definitions
2020-06-09 09:39:14 -07:00
pgtable-nopud.h
mm: consolidate pte_index() and pte_offset_*() definitions
2020-06-09 09:39:14 -07:00
preempt.h
sched/core: Initialize the idle task with preemption disabled
2021-07-14 16:55:50 +02:00
qrwlock_types.h
locking/qrwlock: include asm/byteorder.h as needed
2018-02-06 10:28:58 +01:00
qrwlock.h
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157
2019-05-30 11:26:37 -07:00
qspinlock_types.h
locking/qspinlock: Do not include atomic.h from qspinlock_types.h
2020-07-29 16:14:19 +02:00
qspinlock.h
Merge tag 'powerpc-5.9-1' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux
2020-08-07 10:33:50 -07:00
resource.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
rwonce.h
asm/rwonce: Don't pull <asm/barrier.h> into 'asm-generic/rwonce.h'
2020-07-21 10:50:36 +01:00
seccomp.h
seccomp: Use -1 marker for end of mode 1 syscall list
2020-07-10 16:01:52 -07:00
sections.h
asm-generic: sections: refactor memory_intersects
2022-08-31 17:15:22 +02:00
serial.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
set_memory.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
shmparam.h
treewide: remove SPDX "WITH Linux-syscall-note" from kernel-space headers
2019-05-14 19:52:48 -07:00
signal.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
simd.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
spinlock.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
statfs.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
string.h
…
switch_to.h
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36
2019-05-24 17:27:11 +02:00
syscall.h
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 193
2019-05-30 11:29:21 -07:00
syscalls.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
termios-base.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
termios.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
timex.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
tlb.h
mm/khugepaged: fix GUP-fast interaction by sending IPI
2022-12-14 11:31:55 +01:00
tlbflush.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
topology.h
include/asm-generic/topology.h: guard cpumask_of_node() macro argument
2020-05-28 11:35:41 -07:00
trace_clock.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
uaccess.h
asm-generic: mark __{get,put}_user_fn as __always_inline
2020-10-27 16:13:09 +01:00
unaligned.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
user.h
…
vermagic.h
arch: split MODULE_ARCH_VERMAGIC definitions out to <asm/vermagic.h>
2020-04-23 10:50:26 +09:00
vga.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
vmlinux.lds.h
vmlinux.lds.h: Fix placement of '.data..decrypted' section
2022-11-16 09:57:16 +01:00
vtime.h
…
word-at-a-time.h
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
xor.h
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 47
2019-05-24 17:27:13 +02:00