linux/fs
Mike Kravetz 40549ba8f8 hugetlb: use new vma_lock for pmd sharing synchronization
The new hugetlb vma lock is used to address this race:

Faulting thread                                 Unsharing thread
...                                                  ...
ptep = huge_pte_offset()
      or
ptep = huge_pte_alloc()
...
                                                i_mmap_lock_write
                                                lock page table
ptep invalid   <------------------------        huge_pmd_unshare()
Could be in a previously                        unlock_page_table
sharing process or worse                        i_mmap_unlock_write
...

The vma_lock is used as follows:
- During fault processing. The lock is acquired in read mode before
  doing a page table lock and allocation (huge_pte_alloc).  The lock is
  held until code is finished with the page table entry (ptep).
- The lock must be held in write mode whenever huge_pmd_unshare is
  called.

Lock ordering issues come into play when unmapping a page from all
vmas mapping the page.  The i_mmap_rwsem must be held to search for the
vmas, and the vma lock must be held before calling unmap which will
call huge_pmd_unshare.  This is done today in:
- try_to_migrate_one and try_to_unmap_ for page migration and memory
  error handling.  In these routines we 'try' to obtain the vma lock and
  fail to unmap if unsuccessful.  Calling routines already deal with the
  failure of unmapping.
- hugetlb_vmdelete_list for truncation and hole punch.  This routine
  also tries to acquire the vma lock.  If it fails, it skips the
  unmapping.  However, we can not have file truncation or hole punch
  fail because of contention.  After hugetlb_vmdelete_list, truncation
  and hole punch call remove_inode_hugepages.  remove_inode_hugepages
  checks for mapped pages and call hugetlb_unmap_file_page to unmap them.
  hugetlb_unmap_file_page is designed to drop locks and reacquire in the
  correct order to guarantee unmap success.

Link: https://lkml.kernel.org/r/20220914221810.95771-9-mike.kravetz@oracle.com
Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: James Houghton <jthoughton@google.com>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Miaohe Lin <linmiaohe@huawei.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mina Almasry <almasrymina@google.com>
Cc: Muchun Song <songmuchun@bytedance.com>
Cc: Naoya Horiguchi <naoya.horiguchi@linux.dev>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Prakash Sangappa <prakash.sangappa@oracle.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2022-10-03 14:03:17 -07:00
..
9p 9p: Fix some kernel-doc comments 2022-07-02 18:52:21 +09:00
adfs fs: Convert block_read_full_page() to block_read_full_folio() 2022-05-09 16:21:44 -04:00
affs affs: use memcpy_to_page and remove replace kmap_atomic() 2022-08-01 19:53:31 +02:00
afs afs: Enable multipage folio support 2022-08-13 17:20:51 -07:00
autofs autofs: remove unused ino field inode 2022-07-17 17:31:42 -07:00
befs befs: Convert befs_symlink_read_folio() to use a folio 2022-08-02 12:34:03 -04:00
bfs fs: Convert block_read_full_page() to block_read_full_folio() 2022-05-09 16:21:44 -04:00
btrfs btrfs: convert process_page_range() to use filemap_get_folios_contig() 2022-09-11 20:26:03 -07:00
cachefiles cachefiles: narrow the scope of flushed requests when releasing fd 2022-07-05 16:12:21 +01:00
ceph We have a good pile of various fixes and cleanups from Xiubo, Jeff, 2022-08-11 12:41:07 -07:00
cifs cifs: Add helper function to check smb1+ server 2022-08-24 22:30:09 -05:00
coda coda: Convert coda_symlink_filler() to use a folio 2022-08-02 12:34:03 -04:00
configfs configfs: fix a race in configfs_{,un}register_subsystem() 2022-02-22 18:30:28 +01:00
cramfs cramfs: read_mapping_page() is synchronous 2022-08-02 12:34:02 -04:00
crypto We have a good pile of various fixes and cleanups from Xiubo, Jeff, 2022-08-11 12:41:07 -07:00
debugfs debugfs: Document that debugfs_create functions need not be error checked 2022-02-25 11:56:13 +01:00
devpts fsnotify: fix fsnotify hooks in pseudo filesystems 2022-01-24 14:17:02 +01:00
dlm fs: dlm: move kref_put assert for lkb structs 2022-08-01 09:31:46 -05:00
ecryptfs ecryptfs: Convert ecryptfs to read_folio 2022-05-09 16:21:45 -04:00
efivarfs efi: vars: Move efivar caching layer into efivarfs 2022-06-24 20:40:19 +02:00
efs efs: Convert efs symlinks to read_folio 2022-05-09 16:21:45 -04:00
erofs - The usual batches of cleanups from Baoquan He, Muchun Song, Miaohe 2022-08-05 16:32:45 -07:00
exfat exfat: Drop superfluous new line for error messages 2022-08-01 10:14:07 +09:00
exportfs exportfs: support idmapped mounts 2022-04-28 16:31:10 +02:00
ext2 ext2: replace bh_submit_read() helper with bh_read() 2022-09-11 20:26:08 -07:00
ext4 - The usual batches of cleanups from Baoquan He, Muchun Song, Miaohe 2022-08-05 16:32:45 -07:00
f2fs f2fs-for-6.0 2022-08-08 11:18:31 -07:00
fat Updates to various subsystems which I help look after. lib, ocfs2, 2022-08-07 10:03:24 -07:00
freevxfs freevxfs: Convert vxfs_immed_read_folio() to use a folio 2022-08-02 12:34:03 -04:00
fscache fscache: add tracepoint when failing cookie 2022-08-09 14:13:59 +01:00
fuse mm: multi-gen LRU: groundwork 2022-09-26 19:46:09 -07:00
gfs2 gfs2: replace ll_rw_block() 2022-09-11 20:26:06 -07:00
hfs hfs: Remove check for PageError 2022-06-29 08:51:06 -04:00
hfsplus Folio changes for 6.0 2022-08-03 10:35:43 -07:00
hostfs hostfs: Handle page write errors correctly 2022-08-02 12:34:02 -04:00
hpfs hpfs: Convert symlinks to read_folio 2022-05-09 16:21:45 -04:00
hugetlbfs hugetlb: use new vma_lock for pmd sharing synchronization 2022-10-03 14:03:17 -07:00
iomap New code for 6.0: 2022-08-11 13:11:49 -07:00
isofs isofs: replace ll_rw_block() 2022-09-11 20:26:06 -07:00
jbd2 jbd2: replace ll_rw_block() 2022-09-11 20:26:06 -07:00
jffs2 This pull request contains fixes for JFFS2, UBI and UBIFS 2022-06-03 14:42:24 -07:00
jfs Folio changes for 6.0 2022-08-03 10:35:43 -07:00
kernfs kernfs: Fix typo 'the the' in comment 2022-07-28 10:57:25 +02:00
ksmbd ksmbd: don't remove dos attribute xattr on O_TRUNC open 2022-08-15 21:07:01 -05:00
lockd lockd: detect and reject lock arguments that overflow 2022-08-04 10:28:48 -04:00
minix fs: Convert block_read_full_page() to block_read_full_folio() 2022-05-09 16:21:44 -04:00
netfs netfs: do not unlock and put the folio twice 2022-07-14 10:10:12 +02:00
nfs NFS client bugfixes for Linux 6.0 2022-08-22 11:40:01 -07:00
nfs_common
nfsd NFSD 6.0 Release Notes 2022-08-09 14:56:49 -07:00
nilfs2 nilfs2: convert nilfs_find_uncommited_extent() to use filemap_get_folios_contig() 2022-09-11 20:26:03 -07:00
nls
notify fsnotify: Fix comment typo 2022-07-26 13:38:47 +02:00
ntfs ntfs: fix BUG_ON in ntfs_lookup_inode_by_name() 2022-09-11 16:22:29 -07:00
ntfs3 ntfs3: replace ll_rw_block() 2022-09-11 20:26:07 -07:00
ocfs2 ocfs2: replace ll_rw_block() 2022-09-11 20:26:07 -07:00
omfs fs: Convert block_read_full_page() to block_read_full_folio() 2022-05-09 16:21:44 -04:00
openpromfs fs: allocate inode by using alloc_inode_sb() 2022-03-22 15:57:03 -07:00
orangefs orangefs: Remove test for folio error 2022-06-29 08:51:07 -04:00
overlayfs acl: handle idmapped mounts for idmapped filesystems 2022-08-17 11:23:31 +02:00
proc ksm: count allocated ksm rmap_items for each process 2022-09-26 19:46:29 -07:00
pstore EFI updates for v5.20 2022-08-03 14:38:02 -07:00
qnx4 fs: Convert block_read_full_page() to block_read_full_folio() 2022-05-09 16:21:44 -04:00
qnx6 fs: Convert mpage_readpage to mpage_read_folio 2022-05-09 16:21:44 -04:00
quota - The usual batches of cleanups from Baoquan He, Muchun Song, Miaohe 2022-08-05 16:32:45 -07:00
ramfs ramfs: convert ramfs_nommu_get_unmapped_area() to use filemap_get_folios_contig() 2022-09-11 20:26:03 -07:00
reiserfs reiserfs: replace ll_rw_block() 2022-09-11 20:26:07 -07:00
romfs romfs: Convert romfs to read_folio 2022-05-09 16:21:46 -04:00
smbfs_common Add various fsctl structs 2022-05-23 20:24:12 -05:00
squashfs squashfs: don't call kmalloc in decompressors 2022-08-28 14:02:45 -07:00
sysfs kobject: kobj_type: remove default_attrs 2022-04-05 15:39:19 +02:00
sysv Not a lot of material this cycle. Many singleton patches against various 2022-05-27 11:22:03 -07:00
tracefs tracefs: Fix syntax errors in comments 2022-06-17 19:01:28 -04:00
ubifs - The usual batches of cleanups from Baoquan He, Muchun Song, Miaohe 2022-08-05 16:32:45 -07:00
udf udf: replace ll_rw_block() 2022-09-11 20:26:07 -07:00
ufs ufs: replace ll_rw_block() 2022-09-11 20:26:07 -07:00
unicode kbuild: unify cmd_copy and cmd_shipped 2022-02-14 10:37:32 +09:00
vboxsf vboxsf: Convert vboxsf to read_folio 2022-05-09 16:21:46 -04:00
verity fs-verity: mention btrfs support 2022-07-15 23:42:30 -07:00
xfs xfs: fix SB_BORN check in xfs_dax_notify_failure() 2022-09-11 16:22:29 -07:00
zonefs New code for 6.0: 2022-08-11 13:11:49 -07:00
aio.c iov_iter work, part 1 - isolated cleanups and optimizations. 2022-08-03 13:50:22 -07:00
anon_inodes.c
attr.c vfs: Check the truncate maximum size in inode_newsize_ok() 2022-08-08 10:39:29 -07:00
bad_inode.c
binfmt_aout.c
binfmt_elf_fdpic.c coredump: Snapshot the vmas in do_coredump 2022-03-08 12:55:29 -06:00
binfmt_elf_test.c binfmt_elf: Introduce KUnit test 2022-03-03 20:38:56 -08:00
binfmt_elf.c revert "fs/binfmt_elf: use PT_LOAD p_align values for static PIE" 2022-04-15 14:49:56 -07:00
binfmt_flat.c binfmt_flat: Remove shared library support 2022-04-22 10:57:18 -07:00
binfmt_misc.c Fix regression due to "fs: move binfmt_misc sysctl to its own file" 2022-02-09 09:50:02 -08:00
binfmt_script.c
buffer.c fs/buffer: remove bh_submit_read() helper 2022-09-11 20:26:08 -07:00
char_dev.c
compat_binfmt_elf.c binfmt_elf: Introduce KUnit test 2022-03-03 20:38:56 -08:00
coredump.c coredump: remove vma linked list walk 2022-09-26 19:46:20 -07:00
d_path.c
dax.c - The usual batches of cleanups from Baoquan He, Muchun Song, Miaohe 2022-08-05 16:32:45 -07:00
dcache.c dcache: move the DCACHE_OP_COMPARE case out of the __d_lookup_rcu loop 2022-08-17 14:33:03 -07:00
direct-io.c iov_iter: advancing variants of iov_iter_get_pages{,_alloc}() 2022-08-08 22:37:22 -04:00
drop_caches.c
eventfd.c
eventpoll.c epoll: autoremove wakers even more aggressively 2022-07-17 17:31:40 -07:00
exec.c exec: use VMA iterator instead of linked list 2022-09-26 19:46:21 -07:00
fcntl.c keep iocb_flags() result cached in struct file 2022-06-10 16:10:23 -04:00
fhandle.c
file_table.c iov_iter work, part 1 - isolated cleanups and optimizations. 2022-08-03 13:50:22 -07:00
file.c fix the breakage in close_fd_get_file() calling conventions change 2022-06-05 15:03:03 -04:00
filesystems.c
fs_context.c vfs: fs_context: fix up param length parsing in legacy_parse_param 2022-01-18 09:23:19 +02:00
fs_parser.c
fs_pin.c
fs_struct.c
fs_types.c
fs-writeback.c writeback: avoid use-after-free after removing device 2022-08-28 14:02:43 -07:00
fsopen.c uninline may_mount() and don't opencode it in fspick(2)/fsopen(2) 2022-05-19 23:25:10 -04:00
init.c
inode.c fs: __file_remove_privs(): restore call to inode_has_no_xattr() 2022-08-18 09:39:33 +02:00
internal.h Cleanups (and one fix) around struct mount handling. 2022-06-04 19:00:05 -07:00
ioctl.c Fixes for 5.18-rc1: 2022-04-01 19:35:56 -07:00
Kconfig hugetlb: make hugetlb depends on SYSFS or SYSCTL 2022-09-11 20:26:10 -07:00
Kconfig.binfmt m68knommu: changes for linux 5.19 2022-05-30 10:56:18 -07:00
kernel_read_file.c fs/kernel_read_file: allow to read files up-to ssize_t 2022-06-16 19:58:21 -07:00
libfs.c fs: Convert simple_readpage to simple_read_folio 2022-05-09 16:21:44 -04:00
locks.c locks: Fix dropped call to ->fl_release_private() 2022-08-17 15:08:58 -04:00
Makefile io_uring: move to separate directory 2022-07-24 18:39:10 -06:00
mbcache.c - The usual batches of cleanups from Baoquan He, Muchun Song, Miaohe 2022-08-05 16:32:45 -07:00
mount.h switch try_to_unlazy_next() to __legitimize_mnt() 2022-07-05 16:18:21 -04:00
mpage.c Folio changes for 6.0 2022-08-03 10:35:43 -07:00
namei.c fs.setgid.v6.0 2022-08-09 09:52:28 -07:00
namespace.c fs: require CAP_SYS_ADMIN in target namespace for idmapped mounts 2022-08-17 11:27:11 +02:00
no-block.c
nsfs.c
open.c iov_iter work, part 1 - isolated cleanups and optimizations. 2022-08-03 13:50:22 -07:00
pipe.c Not a lot of material this cycle. Many singleton patches against various 2022-05-27 11:22:03 -07:00
pnode.c
pnode.h
posix_acl.c tmpfs: add support for an i_version counter 2022-10-03 14:03:06 -07:00
proc_namespace.c vfs: escape hash as well 2022-06-28 13:58:05 -04:00
read_write.c switch new_sync_{read,write}() to ITER_UBUF 2022-08-08 22:37:15 -04:00
readdir.c
remap_range.c - The usual batches of cleanups from Baoquan He, Muchun Song, Miaohe 2022-08-05 16:32:45 -07:00
select.c select: Fix indefinitely sleeping task in poll_schedule_timeout() 2022-01-11 09:03:05 -08:00
seq_file.c rxrpc: Fix locking issue 2022-05-22 21:03:01 +01:00
signalfd.c Merge branch 'signal-for-v5.17' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2022-01-17 05:49:30 +02:00
splice.c iter_to_pipe(): switch to advancing variant of iov_iter_get_pages() 2022-08-08 22:37:23 -04:00
stack.c
stat.c RISC-V Patches for the 5.19 Merge Window, Part 1 2022-05-31 14:10:54 -07:00
statfs.c
super.c fuse update for 6.0 2022-08-08 11:10:02 -07:00
sync.c riscv: compat: syscall: Add compat_sys_call_table implementation 2022-04-26 13:36:25 -07:00
sysctls.c fs: move namespace sysctls and declare fs base directory 2022-01-22 08:33:36 +02:00
timerfd.c
userfaultfd.c userfaultfd: use maple tree iterator to iterate VMAs 2022-09-26 19:46:21 -07:00
utimes.c
xattr.c acl: move idmapped mount fixup into vfs_{g,s}etxattr() 2022-07-15 22:08:59 +02:00