iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9c9883744d
linux/fs
History
Jens Axboe aac8d41cd4 writeback: only allow one inflight and pending full flush 
When someone calls wakeup_flusher_threads() or
wakeup_flusher_threads_bdi(), they schedule writeback of all dirty
pages in the system (or on that bdi). If we are tight on memory, we
can get tons of these queued from kswapd/vmscan. This causes (at
least) two problems:

1) We consume a ton of memory just allocating writeback work items.
   We've seen as much as 600 million of these writeback work items
   pending. That's a lot of memory to pointlessly hold hostage,
   while the box is under memory pressure.

2) We spend so much time processing these work items, that we
   introduce a softlockup in writeback processing. This is because
   each of the writeback work items don't end up doing any work (it's
   hard when you have millions of identical ones coming in to the
   flush machinery), so we just sit in a tight loop pulling work
   items and deleting/freeing them.

Fix this by adding a 'start_all' bit to the writeback structure, and
set that when someone attempts to flush all dirty pages. The bit is
cleared when we start writeback on that work item. If the bit is
already set when we attempt to queue !nr_pages writeback, then we
simply ignore it.

This provides us one full flush in flight, with one pending as well,
and makes for more efficient handling of this type of writeback.

Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Tested-by: Chris Mason <clm@fb.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2017-10-03 08:38:17 -06:00
..
9p Merge branch 'akpm' (patches from Andrew) 2017-09-06 20:49:49 -07:00
adfs
affs Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
afs Merge branch 'akpm' (patches from Andrew) 2017-09-06 20:49:49 -07:00
autofs4 Merge branch 'work.set_fs' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:13:32 -07:00
befs VFS: Convert sb->s_flags & MS_RDONLY to sb_rdonly(sb) 2017-07-17 08:45:34 +01:00
bfs bfs: fix sanity checks for empty files 2017-07-12 16:26:00 -07:00
btrfs Merge branch 'work.read_write' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 19:29:55 -07:00
cachefiles VFS: Convert sb->s_flags & MS_RDONLY to sb_rdonly(sb) 2017-07-17 08:45:34 +01:00
ceph ceph: avoid panic in create_session_open_msg() if utsname() returns NULL 2017-09-19 21:04:52 +02:00
cifs SMB3: Don't ignore O_SYNC/O_DSYNC and O_DIRECT flags 2017-09-22 19:47:17 -05:00
coda fs: fix kernel_read prototype 2017-09-04 19:05:15 -04:00
configfs
cramfs
crypto block: replace bi_bdev with a gendisk pointer and partitions index 2017-08-23 12:49:55 -06:00
debugfs Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-07-15 12:00:42 -07:00
devpts pty: Repair TIOCGPTPEER 2017-08-24 13:23:03 -07:00
dlm File locking related changes for v4.14 2017-09-06 13:43:26 -07:00
ecryptfs Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
efivarfs VFS: Kill off s_options and helpers 2017-07-11 06:09:21 -04:00
efs VFS: Convert sb->s_flags & MS_RDONLY to sb_rdonly(sb) 2017-07-17 08:45:34 +01:00
exofs Merge branch 'for-4.14/block' of git://git.kernel.dk/linux-block 2017-09-07 11:59:42 -07:00
exportfs
ext2 Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
ext4 Merge branch 'work.read_write' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 19:29:55 -07:00
f2fs for-f2fs-4.14 2017-09-12 20:05:58 -07:00
fat Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
freevxfs
fscache fscache: fix fscache_objlist_show format processing 2017-09-13 18:53:15 -07:00
fuse Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse 2017-09-13 10:10:19 -07:00
gfs2 GFS2: Fix an old regression in GFS2's debugfs interface 2017-09-25 15:41:56 -07:00
hfs Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
hfsplus Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
hostfs fs: convert a pile of fsync routines to errseq_t based reporting 2017-08-01 08:39:29 -04:00
hpfs Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
hugetlbfs lib/interval_tree: fast overlap detection 2017-09-08 18:26:49 -07:00
isofs Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
jbd2 Writeback error handling fixes (pile #2) 2017-07-07 19:38:17 -07:00
jffs2 Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
jfs Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
kernfs Merge branch 'for-4.14/block' of git://git.kernel.dk/linux-block 2017-09-07 11:59:42 -07:00
lockd NFS client updates for Linux 4.14 2017-09-11 22:01:44 -07:00
minix VFS: Convert sb->s_flags & MS_RDONLY to sb_rdonly(sb) 2017-07-17 08:45:34 +01:00
ncpfs fs: convert a pile of fsync routines to errseq_t based reporting 2017-08-01 08:39:29 -04:00
nfs NFS client bugfixes for Linux 4.14 2017-09-14 20:04:32 -07:00
nfs_common
nfsd More RDMA work and some op-structure constification from Chuck Lever, 2017-09-09 13:31:49 -07:00
nilfs2 Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
nls
notify fsnotify: make dnotify_fsnotify_ops const 2017-08-30 16:02:48 +02:00
ntfs buffer: have alloc_page_buffers() use __GFP_NOFAIL 2017-10-03 08:38:17 -06:00
ocfs2 Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
omfs omfs: Implement show_options 2017-07-06 03:31:46 -04:00
openpromfs
orangefs orangefs: Adjust three checks for null pointers 2017-09-14 14:58:31 -04:00
overlayfs Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
proc fs/proc: Report eip/esp in /prod/PID/stat for coredumping 2017-09-15 23:31:16 +02:00
pstore Revert "pstore: Honor dmesg_restrict sysctl on dmesg dumps" 2017-08-17 16:29:19 -07:00
qnx4
qnx6
quota Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
ramfs mm: make pagevec_lookup() update index 2017-09-06 17:27:26 -07:00
reiserfs Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
romfs
squashfs squashfs: Add zstd support 2017-09-08 19:33:25 -07:00
sysfs
sysv VFS: Convert sb->s_flags & MS_RDONLY to sb_rdonly(sb) 2017-07-17 08:45:34 +01:00
tracefs VFS: Don't use save/replace_mount_options if not using generic_show_options 2017-07-06 03:31:46 -04:00
ubifs Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
udf Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
ufs VFS: Convert sb->s_flags & MS_RDONLY to sb_rdonly(sb) 2017-07-17 08:45:34 +01:00
xfs Merge branch 'work.read_write' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 19:29:55 -07:00
aio.c Merge branch 'work.read_write' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 19:29:55 -07:00
anon_inodes.c
attr.c
bad_inode.c
binfmt_aout.c fs: fix kernel_read prototype 2017-09-04 19:05:15 -04:00
binfmt_elf_fdpic.c Merge branch 'work.set_fs' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:13:32 -07:00
binfmt_elf.c Merge branch 'work.set_fs' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:13:32 -07:00
binfmt_em86.c
binfmt_flat.c Merge branch 'work.set_fs' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:13:32 -07:00
binfmt_misc.c fs: fix kernel_read prototype 2017-09-04 19:05:15 -04:00
binfmt_script.c
block_dev.c Merge branch 'work.read_write' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 19:29:55 -07:00
buffer.c buffer: eliminate the need to call free_more_memory() in __getblk_slow() 2017-10-03 08:38:17 -06:00
char_dev.c char_dev: order /proc/devices by major number 2017-07-17 15:28:50 +02:00
compat_binfmt_elf.c
compat_ioctl.c media: get rid of removed DMX_GET_CAPS and DMX_SET_SOURCE leftovers 2017-09-05 08:25:07 -04:00
compat.c
coredump.c mm: treewide: remove GFP_TEMPORARY allocation flag 2017-09-13 18:53:16 -07:00
dax.c - Some request-based DM core and DM multipath fixes and cleanups 2017-09-14 13:43:16 -07:00
dcache.c Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-07-15 12:00:42 -07:00
dcookies.c
direct-io.c fs: Fix page cache inconsistency when mixing buffered and AIO DIO 2017-09-25 08:56:05 -06:00
drop_caches.c
eventfd.c There has been a fair amount of activity in the docs tree this time 2017-07-03 21:13:25 -07:00
eventpoll.c fs/epoll: use faster rb_first_cached() 2017-09-08 18:26:49 -07:00
exec.c vfs: constify path argument to kernel_read_file_from_path 2017-09-14 20:18:45 -07:00
fcntl.c fcntl: Don't set si_code to SI_SIGIO when sig == SIGPOLL 2017-09-18 22:51:14 -05:00
fhandle.c
file_table.c fput: Don't reinvent the wheel but use existing llist API 2017-08-28 00:50:23 -04:00
file.c fs/file.c: replace alloc_fdmem() with kvmalloc() alternative 2017-07-06 16:24:30 -07:00
filesystems.c Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-07-15 12:00:42 -07:00
fs_pin.c
fs_struct.c
fs-writeback.c writeback: only allow one inflight and pending full flush 2017-10-03 08:38:17 -06:00
inode.c Merge branch 'overlayfs-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs 2017-09-13 09:11:44 -07:00
internal.h Merge branch 'overlayfs-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs 2017-09-13 09:11:44 -07:00
ioctl.c
iomap.c fs: Fix page cache inconsistency when mixing buffered and AIO DIO 2017-09-25 08:56:05 -06:00
Kconfig fs/Kconfig: kill CONFIG_PERCPU_RWSEM some more 2017-07-12 16:26:00 -07:00
Kconfig.binfmt
libfs.c fs: convert __generic_file_fsync to use errseq_t based reporting 2017-07-06 07:02:29 -04:00
locks.c locks: restore a warn for leaked locks on close 2017-07-21 13:57:31 -04:00
Makefile
mbcache.c
mount.h Now that IPC and other changes have landed, enable manual markings for 2017-07-19 08:55:18 -07:00
mpage.c block: replace bi_bdev with a gendisk pointer and partitions index 2017-08-23 12:49:55 -06:00
namei.c Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
namespace.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 20:01:41 -07:00
no-block.c
nsfs.c VFS: Provide empty name qstr 2017-07-06 03:27:09 -04:00
open.c ovl: don't allow writing ioctl on lower layer 2017-09-05 12:53:12 +02:00
pipe.c VFS: Provide empty name qstr 2017-07-06 03:27:09 -04:00
pnode.c
pnode.h
posix_acl.c
proc_namespace.c VFS: Convert sb->s_flags & MS_RDONLY to sb_rdonly(sb) 2017-07-17 08:45:34 +01:00
read_write.c Merge branch 'work.set_fs' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:13:32 -07:00
readdir.c
select.c fs/select: Fix memory corruption in compat_get_fd_set() 2017-08-28 16:09:19 -07:00
seq_file.c
signalfd.c signal: Remove kernel interal si_code magic 2017-07-24 14:30:28 -05:00
splice.c fs: move kernel_write to fs/read_write.c 2017-09-04 19:05:15 -04:00
stack.c
stat.c fs: Provide __inode_get_bytes() 2017-08-17 22:06:03 +02:00
statfs.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-07-08 10:50:54 -07:00
super.c Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:54:01 -07:00
sync.c fs: kill 'nr_pages' argument from wakeup_flusher_threads() 2017-10-03 08:38:17 -06:00
timerfd.c timerfd: Use get_itimerspec64() and put_itimerspec64() 2017-06-30 04:14:38 -04:00
userfaultfd.c userfaultfd: non-cooperative: closing the uffd without triggering SIGBUS 2017-09-08 18:26:47 -07:00
utimes.c utimes: Make utimes y2038 safe 2017-09-03 20:24:30 -04:00
xattr.c Merge branch 'overlayfs-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs 2017-09-13 09:11:44 -07:00