linux/include/asm-generic
Rik van Riel 2084140594 mm: fix TLB flush race between migration, and change_protection_range
There are a few subtle races, between change_protection_range (used by
mprotect and change_prot_numa) on one side, and NUMA page migration and
compaction on the other side.

The basic race is that there is a time window between when the PTE gets
made non-present (PROT_NONE or NUMA), and the TLB is flushed.

During that time, a CPU may continue writing to the page.

This is fine most of the time, however compaction or the NUMA migration
code may come in, and migrate the page away.

When that happens, the CPU may continue writing, through the cached
translation, to what is no longer the current memory location of the
process.

This only affects x86, which has a somewhat optimistic pte_accessible.
All other architectures appear to be safe, and will either always flush,
or flush whenever there is a valid mapping, even with no permissions
(SPARC).

The basic race looks like this:

CPU A			CPU B			CPU C

						load TLB entry
make entry PTE/PMD_NUMA
			fault on entry
						read/write old page
			start migrating page
			change PTE/PMD to new page
						read/write old page [*]
flush TLB
						reload TLB from new entry
						read/write new page
						lose data

[*] the old page may belong to a new user at this point!

The obvious fix is to flush remote TLB entries, by making sure that
pte_accessible aware of the fact that PROT_NONE and PROT_NUMA memory may
still be accessible if there is a TLB flush pending for the mm.

This should fix both NUMA migration and compaction.

[mgorman@suse.de: fix build]
Signed-off-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Cc: Alex Thorlton <athorlton@sgi.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-12-18 19:04:51 -08:00
..
bitops bitops/find: clarify and extend documentation 2013-11-13 12:09:22 +09:00
4level-fixup.h
atomic64.h
atomic-long.h
atomic.h asm-generic: move cmpxchg*_local defs to cmpxchg.h 2013-03-13 06:11:05 +01:00
audit_change_attr.h
audit_dir_write.h
audit_read.h
audit_signal.h
audit_write.h
barrier.h Create asm-generic/barrier.h 2012-03-28 18:30:03 +01:00
bitops.h
bitsperlong.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
bug.h bug.h: Fix up CONFIG_BUG=n implicit function declarations. 2012-06-25 10:32:49 -07:00
bugs.h
cache.h
cacheflush.h
checksum.h asm-generic headers: Allow yet more arch overrides in checksum.h 2013-02-11 20:00:33 +05:30
clkdev.h asm-generic: Add default clkdev.h 2012-10-03 21:33:53 +02:00
cmpxchg-local.h generic: Use raw local irq variant for generic cmpxchg 2013-02-19 08:43:37 +01:00
cmpxchg.h asm-generic: move cmpxchg*_local defs to cmpxchg.h 2013-03-13 06:11:05 +01:00
cputime_jiffies.h cputime: Move default nsecs_to_cputime() to jiffies based cputime file 2013-01-27 19:23:25 +01:00
cputime_nsecs.h cputime_nsecs: use math64.h for nsec resolution conversion helpers 2013-04-26 18:58:12 +02:00
cputime.h cputime: Generic on-demand virtual cputime accounting 2013-01-27 19:23:27 +01:00
current.h
delay.h asm-generic: delay.h fix udelay and ndelay for 8 bit args 2011-07-22 18:45:33 +02:00
device.h
div64.h
dma-coherent.h common: dma-mapping: add support for generic dma_mmap_* calls 2012-07-30 12:25:46 +02:00
dma-mapping-broken.h asm-generic/dma-mapping-broken.h: Provide dma_alloc_attrs()/dma_free_attrs() 2012-12-25 20:14:54 +01:00
dma-mapping-common.h common: dma-mapping: introduce dma_get_sgtable() function 2012-07-30 12:25:46 +02:00
dma.h
emergency-restart.h
exec.h Split arch_align_stack() out from asm-generic/system.h 2012-03-28 18:30:03 +01:00
fb.h
ftrace.h
futex.h
getorder.h bitops: Add missing parentheses to new get_order macro 2012-02-24 10:39:27 -08:00
gpio.h GPIO bulk changes for the v3.13 development cycle 2013-11-12 15:50:46 +09:00
hardirq.h
hugetlb.h mm: Fix generic hugetlb pte check return type. 2013-10-02 20:02:35 -04:00
hw_irq.h
ide_iops.h
int-l64.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
int-ll64.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
io-64-nonatomic-hi-lo.h asm-generic: architecture independent readq/writeq for 32bit environment 2012-02-21 16:47:28 -08:00
io-64-nonatomic-lo-hi.h asm-generic: architecture independent readq/writeq for 32bit environment 2012-02-21 16:47:28 -08:00
io.h kernel: Fix s390 absolute memory access for /dev/mem 2013-05-22 09:45:57 +02:00
ioctl.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
iomap.h [PARISC] fix compile break caused by iomap: make IOPORT/PCI mapping functions conditional 2012-02-27 09:43:30 -06:00
irq_regs.h
irq.h
irqflags.h
Kbuild.asm UAPI: Set up uapi/asm/Kbuild.asm 2012-10-02 18:01:56 +01:00
kdebug.h
kmap_types.h asm-generic: remove km_type definitions 2012-07-24 15:27:30 +08:00
kvm_para.h KVM: add kvm_para_available to asm-generic/kvm_para.h 2013-06-05 13:21:29 +03:00
libata-portmap.h
linkage.h
local64.h atomic: use <linux/atomic.h> 2011-07-26 16:49:47 -07:00
local.h atomic: use <linux/atomic.h> 2011-07-26 16:49:47 -07:00
memory_model.h __page_to_pfn: Fix typo in comment 2013-10-14 15:28:29 +02:00
mm_hooks.h
mmu_context.h
mmu.h asm-generic/mmu.h: Add support for FDPIC 2012-12-09 23:14:14 +01:00
module.h Make most arch asm/module.h files use asm-generic/module.h 2012-09-28 14:31:03 +09:30
mutex-dec.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
mutex-null.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
mutex-xchg.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
mutex.h
page.h The following changes since commit 3ee72ca992 2012-01-10 17:39:40 -08:00
param.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
parport.h include: remove __dev* attributes. 2013-01-03 15:57:16 -08:00
pci_iomap.h [PARISC] fix compile break caused by iomap: make IOPORT/PCI mapping functions conditional 2012-02-27 09:43:30 -06:00
pci-bridge.h PCI: work around Stratus ftServer broken PCIe hierarchy 2012-04-30 15:21:02 -06:00
pci-dma-compat.h
pci.h PCI: collapse pcibios_resource_to_bus 2012-02-23 20:19:04 -07:00
percpu.h
pgalloc.h
pgtable-nopmd.h
pgtable-nopud.h
pgtable.h mm: fix TLB flush race between migration, and change_protection_range 2013-12-18 19:04:51 -08:00
preempt.h sched: Remove PREEMPT_NEED_RESCHED from generic code 2013-12-11 15:52:32 +01:00
ptrace.h
resource.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
rtc.h
rwsem.h Hexagon: Add locking types and functions 2011-11-01 07:34:20 -07:00
scatterlist.h
sections.h vmlinux.lds: add comments for global variables and clean up useless declarations 2013-07-03 16:07:34 -07:00
segment.h
serial.h
siginfo.h constify copy_siginfo_to_user{,32}() 2013-11-09 00:16:29 -05:00
signal.h unify default ptrace_signal_deliver 2012-11-29 00:01:23 -05:00
simd.h crypto: create generic version of ablk_helper 2013-09-24 06:02:24 +10:00
sizes.h ARM: 7430/1: sizes.h: move from asm-generic to <linux/sizes.h> 2012-06-28 17:14:34 +01:00
spinlock.h
statfs.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
string.h
switch_to.h Split the switch_to() wrapper out of asm-generic/system.h 2012-03-28 18:30:03 +01:00
syscall.h asm/syscall.h: add syscall_get_arch 2012-04-14 11:13:19 +10:00
syscalls.h burying unused conditionals 2013-02-14 09:21:15 -05:00
termios-base.h
termios.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
timex.h
tlb.h Fix TLB gather virtual address range invalidation corner cases 2013-08-16 08:52:46 -07:00
tlbflush.h BUG: headers with BUG/BUG_ON etc. need linux/bug.h 2012-03-04 17:54:34 -05:00
topology.h
trace_clock.h tracing,x86: Add a TSC trace_clock 2012-11-13 15:48:27 -05:00
uaccess-unaligned.h
uaccess.h asm-generic: uaccess s/might_sleep/might_fault/ 2013-05-28 09:41:05 +02:00
unaligned.h
unistd.h We get rid of the general module prefix confusion with a binary config option, 2013-05-05 10:58:06 -07:00
user.h
vga.h
vmlinux.lds.h kernel: add support for init_array constructors 2013-10-17 15:05:17 +10:30
vtime.h include/asm-generic/vtime.h: avoid zero-length file 2013-09-30 14:31:02 -07:00
word-at-a-time.h word-at-a-time: provide generic big-endian zero_bytemask implementation 2013-12-12 10:39:01 -08:00
xor.h asm-generic: xor: mark static functions as __maybe_unused 2012-10-03 21:21:06 +02:00