linux/fs
David Howells 22650f1481 afs: Fix speculative status fetches
The generic/464 xfstest causes kAFS to emit occasional warnings of the
form:

        kAFS: vnode modified {100055:8a} 30->31 YFS.StoreData64 (c=6015)

This indicates that the data version received back from the server did not
match the expected value (the DV should be incremented monotonically for
each individual modification op committed to a vnode).

What is happening is that a lookup call is doing a bulk status fetch
speculatively on a bunch of vnodes in a directory besides getting the
status of the vnode it's actually interested in.  This is racing with a
StoreData operation (though it could also occur with, say, a MakeDir op).

On the client, a modification operation locks the vnode, but the bulk
status fetch only locks the parent directory, so no ordering is imposed
there (thereby avoiding an avenue to deadlock).

On the server, the StoreData op handler doesn't lock the vnode until it's
received all the request data, and downgrades the lock after committing the
data until it has finished sending change notifications to other clients -
which allows the status fetch to occur before it has finished.

This means that:

 - a status fetch can access the target vnode either side of the exclusive
   section of the modification

 - the status fetch could start before the modification, yet finish after,
   and vice-versa.

 - the status fetch and the modification RPCs can complete in either order.

 - the status fetch can return either the before or the after DV from the
   modification.

 - the status fetch might regress the locally cached DV.

Some of these are handled by the previous fix[1], but that's not sufficient
because it checks the DV it received against the DV it cached at the start
of the op, but the DV might've been updated in the meantime by a locally
generated modification op.

Fix this by the following means:

 (1) Keep track of when we're performing a modification operation on a
     vnode.  This is done by marking vnode parameters with a 'modification'
     note that causes the AFS_VNODE_MODIFYING flag to be set on the vnode
     for the duration.

 (2) Alter the speculation race detection to ignore speculative status
     fetches if either the vnode is marked as being modified or the data
     version number is not what we expected.

Note that whilst the "vnode modified" warning does get recovered from as it
causes the client to refetch the status at the next opportunity, it will
also invalidate the pagecache, so changes might get lost.

Fixes: a9e5c87ca7 ("afs: Fix speculative status fetch going out of order wrt to modifications")
Reported-by: Marc Dionne <marc.dionne@auristor.com>
Signed-off-by: David Howells <dhowells@redhat.com>
Tested-and-reviewed-by: Marc Dionne <marc.dionne@auristor.com>
cc: linux-afs@lists.infradead.org
Link: https://lore.kernel.org/r/160605082531.252452.14708077925602709042.stgit@warthog.procyon.org.uk/ [1]
Link: https://lore.kernel.org/linux-fsdevel/161961335926.39335.2552653972195467566.stgit@warthog.procyon.org.uk/ # v1
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2021-05-01 11:55:36 -07:00
..
9p 9p: missing chunk of "fs/9p: Don't update file type when updating file attributes" 2021-03-12 22:15:22 -05:00
adfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
affs idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
afs afs: Fix speculative status fetches 2021-05-01 11:55:36 -07:00
autofs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
befs fs/befs: Delete obsolete TODO file 2021-03-30 16:54:49 -07:00
bfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
btrfs Merge branch 'miklos.fileattr' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-04-27 11:18:24 -07:00
cachefiles fscache, cachefiles: Add alternate API to use kiocb for read/write to cache 2021-04-23 10:14:32 +01:00
ceph ceph: don't allow type or device number to change on non-I_NEW inodes 2021-03-08 10:19:37 -05:00
cifs Merge branch 'work.inode-type-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-04-27 10:57:42 -07:00
coda coda: fix reference counting in coda_file_mmap error path 2021-04-23 14:42:39 -07:00
configfs configfs: fix a use-after-free in __configfs_open_file 2021-03-11 12:13:48 +01:00
cramfs cramfs: use %pD instead of messing with file_dentry()->d_name 2021-01-05 23:02:47 -05:00
crypto Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2021-04-26 08:51:23 -07:00
debugfs Scheduler updates for this cycle are: 2021-04-28 13:33:57 -07:00
devpts
dlm fs: dlm: fix missing unlock on error in accept_from_sock() 2021-03-29 13:28:18 -05:00
ecryptfs ecryptfs: stack fileattr ops 2021-04-12 15:04:29 +02:00
efivarfs efivars: convert to fileattr 2021-04-12 15:04:29 +02:00
efs [PATCH] reduce boilerplate in fsid handling 2020-09-18 16:45:50 -04:00
erofs erofs: enable big pcluster feature 2021-04-10 03:20:19 +08:00
exfat exfat: speed up iterate/lookup by fixing start point of traversing cluster chain 2021-04-27 20:45:07 +09:00
exportfs exportfs: Add a function to return the raw output from fh_to_dentry() 2020-12-09 09:39:38 -05:00
ext2 \n 2021-04-29 11:06:13 -07:00
ext4 New features for ext4 this cycle include support for encrypted 2021-04-30 15:35:30 -07:00
f2fs Network filesystem helper library 2021-04-27 13:08:12 -07:00
fat idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
freevxfs
fscache fscache, cachefiles: Add alternate API to use kiocb for read/write to cache 2021-04-23 10:14:32 +01:00
fuse fuse update for 5.13 2021-04-30 15:23:16 -07:00
gfs2 Changes in gfs2: 2021-04-29 10:33:35 -07:00
hfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
hfsplus hfsplus: convert to fileattr 2021-04-12 15:04:29 +02:00
hostfs Merge branch 'work.inode-type-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-04-27 10:57:42 -07:00
hpfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
hugetlbfs hugetlbfs: remove unneeded return value of hugetlb_vmtruncate() 2021-02-24 13:38:35 -08:00
iomap iomap: use filemap_range_needs_writeback() for O_DIRECT reads 2021-04-30 11:20:36 -07:00
isofs isofs: handle large user and group ID 2021-02-03 19:05:52 +01:00
jbd2 ext4: fix debug format string warning 2021-04-09 23:32:16 -04:00
jffs2 fs/jffs2: Delete obsolete TODO file 2021-03-30 16:54:49 -07:00
jfs jfs: convert to fileattr 2021-04-12 15:04:29 +02:00
kernfs idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
lockd SUNRPC: Make trace_svc_process() display the RPC procedure symbolically 2021-01-25 09:36:23 -05:00
minix fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
netfs netfs: Miscellaneous fixes 2021-04-26 23:23:41 +01:00
nfs selinux/stable-5.13 PR 20210426 2021-04-27 13:42:11 -07:00
nfs_common NFSD: Add an xdr_stream-based encoder for NFSv2/3 ACLs 2021-03-22 10:19:00 -04:00
nfsd Merge branch 'work.inode-type-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-04-27 10:57:42 -07:00
nilfs2 nilfs2: convert to fileattr 2021-04-12 15:04:30 +02:00
nls
notify fanotify_user: use upper_32_bits() to verify mask 2021-03-25 15:33:45 +01:00
ntfs ntfs: check for valid standard information attribute 2021-02-24 13:38:26 -08:00
ocfs2 ocfs2/dlm: remove unused function 2021-04-30 11:20:35 -07:00
omfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
openpromfs openpromfs: don't do unlock_new_inode() until the new inode is set up 2021-03-12 22:15:22 -05:00
orangefs Merge branch 'miklos.fileattr' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-04-27 11:18:24 -07:00
overlayfs overlayfs update for 5.13 2021-04-30 15:17:08 -07:00
proc Kbuild updates for v5.13 2021-04-29 14:24:39 -07:00
pstore printk changes for 5.13 2021-04-27 18:09:44 -07:00
qnx4 [PATCH] reduce boilerplate in fsid handling 2020-09-18 16:45:50 -04:00
qnx6 [PATCH] reduce boilerplate in fsid handling 2020-09-18 16:45:50 -04:00
quota quota: report warning limits for realtime space quotas 2021-03-18 17:03:54 +01:00
ramfs ramfs: support O_TMPFILE 2021-02-24 13:38:26 -08:00
reiserfs \n 2021-04-29 10:51:29 -07:00
romfs Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2020-10-24 12:26:05 -07:00
squashfs squashfs: fix xattr id and id lookup sanity checks 2021-03-25 09:22:55 -07:00
sysfs sysfs: Support zapping of binary attr mmaps 2021-01-12 14:26:31 +01:00
sysv fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
tracefs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
ubifs Merge branch 'miklos.fileattr' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-04-27 11:18:24 -07:00
udf idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
ufs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
unicode unicode: Add utf8_casefold_hash 2020-09-10 14:03:31 -07:00
vboxsf vboxsf: don't allow to change the inode type 2021-03-12 22:15:00 -05:00
verity fsverity: relax build time dependency on CRYPTO_SHA256 2021-04-22 17:31:32 +10:00
xfs New code for 5.13: 2021-04-29 10:43:51 -07:00
zonefs \n 2021-04-29 11:06:13 -07:00
aio.c Revert "mremap: don't allow MREMAP_DONTUNMAP on special_mappings and aio" 2021-04-30 11:20:39 -07:00
anon_inodes.c fs: anon_inodes: rephrase to appropriate kernel-doc 2021-01-15 12:17:25 -05:00
attr.c ima: handle idmapped mounts 2021-01-24 14:27:20 +01:00
bad_inode.c fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
binfmt_aout.c
binfmt_elf_fdpic.c coredump: don't bother with do_truncate() 2021-03-08 10:21:11 -05:00
binfmt_elf.c coredump: don't bother with do_truncate() 2021-03-08 10:21:11 -05:00
binfmt_em86.c
binfmt_flat.c binfmt_flat: revert "binfmt_flat: don't offset the data start" 2020-08-24 08:49:13 +10:00
binfmt_misc.c binfmt_misc: fix possible deadlock in bm_register_write 2021-03-13 11:27:30 -08:00
binfmt_script.c
block_dev.c for-5.13/drivers-2021-04-27 2021-04-28 14:39:37 -07:00
buffer.c fs: buffer: use raw page_memcg() on locked page 2021-02-24 13:38:30 -08:00
char_dev.c
compat_binfmt_elf.c get rid of COMPAT_ELF_EXEC_PAGESIZE 2021-01-06 08:42:51 -05:00
coredump.c coredump: don't bother with do_truncate() 2021-03-08 10:21:11 -05:00
d_path.c fs: fix NULL dereference due to data race in prepend_path() 2020-10-14 14:54:45 -07:00
dax.c mm: provide a saner PTE walking API for modules 2021-02-09 07:05:44 -05:00
dcache.c fs: delete repeated words in comments 2021-02-24 13:38:26 -08:00
direct-io.c fs: direct-io: fix missing sdio->boundary 2021-04-09 14:54:23 -07:00
drop_caches.c
eventfd.c eventfd: Export eventfd_ctx_do_read() 2020-11-15 09:49:10 -05:00
eventpoll.c fs: eventpoll: fix comments & kernel-doc notation 2021-03-06 17:36:53 -07:00
exec.c fs: delete repeated words in comments 2021-02-24 13:38:26 -08:00
fcntl.c idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
fhandle.c fs: delete repeated words in comments 2021-02-24 13:38:26 -08:00
file_table.c epoll: take epitem list out of struct file 2020-10-25 20:02:08 -04:00
file.c file: fix close_range() for unshare+cloexec 2021-04-02 14:11:10 +02:00
filesystems.c
fs_context.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
fs_parser.c vfs: fs_parser: clean up kernel-doc warnings 2021-04-30 11:20:35 -07:00
fs_pin.c
fs_struct.c
fs_types.c
fs-writeback.c fs: improve comments for writeback_single_inode() 2021-01-13 17:26:50 +01:00
fsopen.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
init.c init: handle idmapped mounts 2021-01-24 14:27:19 +01:00
inode.c fs.idmapped.helpers.v5.13 2021-04-27 12:49:42 -07:00
internal.h idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
io_uring.c for-5.13/io_uring-2021-04-27 2021-04-28 14:56:09 -07:00
io-wq.c io-wq: remove unused io_wqe_need_worker() function 2021-04-20 11:24:22 -06:00
io-wq.h io-wq: eliminate the need for a manager thread 2021-04-11 17:42:00 -06:00
ioctl.c vfs: add fileattr ops 2021-04-12 15:04:23 +02:00
Kconfig netfs: Provide readahead and readpage netfs helpers 2021-04-23 10:14:32 +01:00
Kconfig.binfmt Merge branch 'work.elf-compat' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-02-21 09:29:23 -08:00
kernel_read_file.c fs/kernel_file_read: Add "offset" arg for partial reads 2020-10-05 13:37:04 +02:00
libfs.c libfs: fix kernel-doc for mnt_userns 2021-03-23 11:20:25 +01:00
locks.c File locking fixes for v5.13 2021-04-26 13:24:39 -07:00
Makefile netfs: Provide readahead and readpage netfs helpers 2021-04-23 10:14:32 +01:00
mbcache.c
mount.h mount: make {lock,unlock}_mount_hash() static 2021-01-24 14:29:34 +01:00
mpage.c block: rename BIO_MAX_PAGES to BIO_MAX_VECS 2021-03-11 07:47:48 -07:00
namei.c fs.idmapped.helpers.v5.13 2021-04-27 12:49:42 -07:00
namespace.c fs/namespace: correct/improve kernel-doc notation 2021-03-31 14:22:55 -06:00
no-block.c
nsfs.c
open.c idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
pipe.c fs: delete repeated words in comments 2021-02-24 13:38:26 -08:00
pnode.c
pnode.h mount: fix mounting of detached mounts onto targets that reside on shared mounts 2021-03-08 15:18:43 +01:00
posix_acl.c fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
proc_namespace.c fs: introduce MOUNT_ATTR_IDMAP 2021-01-24 14:43:45 +01:00
read_write.c teach sendfile(2) to handle send-to-pipe directly 2021-01-25 23:29:36 -05:00
readdir.c readdir: make sure to verify directory entry for legacy interfaces too 2021-04-17 11:39:49 -07:00
remap_range.c ioctl: handle idmapped mounts 2021-01-24 14:27:19 +01:00
select.c kernel, fs: Introduce and use set_restart_fn() and arch_set_restart_data() 2021-03-16 22:13:10 +01:00
seq_file.c seq_file: Add a seq_bprintf function 2021-04-27 15:50:15 -07:00
signalfd.c signal: Introduce TRAP_PERF si_code and si_perf to siginfo 2021-04-16 16:32:41 +02:00
splice.c for-5.12/block-2021-02-17 2021-02-21 11:02:48 -08:00
stack.c
stat.c fs: fix reporting supported extra file attributes for statx() 2021-04-17 23:03:50 -04:00
statfs.c s390,alpha: switch to 64-bit ino_t 2021-02-13 17:17:53 +01:00
super.c It has been a relatively quiet cycle in docsland. 2021-02-22 10:57:46 -08:00
sync.c
timerfd.c
userfaultfd.c userfaultfd: use secure anon inodes for userfaultfd 2021-01-14 17:40:57 -05:00
utimes.c utimes: handle idmapped mounts 2021-01-24 14:27:18 +01:00
xattr.c xattr: fix kernel-doc for mnt_userns and vfs xattr helpers 2021-03-23 11:20:26 +01:00