iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
linux/fs/bcachefs
History
Kent Overstreet 916abefd43 bcachefs: better journal pipelining 
Recently a severe performance regression was discovered, which bisected
to

  a6548c8b5eb5 bcachefs: Avoid flushing the journal in the discard path

It turns out the old behaviour, which issued excessive journal flushes,
worked around a performance issue where queueing delays would cause the
journal to not be able to write quickly enough and stall.

The journal flushes masked the issue because they periodically flushed
the device write cache, reducing write latency for non flushes.

This patch reworks the journalling code to allow more than one
(non-flush) write to be in flight at a time. With this patch, doing 4k
random writes and an iodepth of 128, we are now able to hit 560k iops to
a Samsung 970 EVO Plus - previously, we were stuck in the ~200k range.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2024-03-10 15:34:08 -04:00
..
acl.c
bcachefs: make RO snapshots actually RO
2024-01-01 11:47:07 -05:00
acl.h
…
alloc_background_format.h
bcachefs: alloc_background_format.h
2024-01-21 13:27:10 -05:00
alloc_background.c
bcachefs: discard path uses unlock_long()
2024-01-24 17:27:46 -05:00
alloc_background.h
bcachefs: Combine .trans_trigger, .atomic_trigger
2024-01-05 23:24:20 -05:00
alloc_foreground.c
bcachefs: helpers for printing data types
2024-01-21 06:01:45 -05:00
alloc_foreground.h
…
alloc_types.h
…
backpointers.c
bcachefs: fix backpointer_to_text() when dev does not exist
2024-02-24 20:41:37 -05:00
backpointers.h
bcachefs: Prep work for variable size btree node buffers
2024-01-21 13:27:10 -05:00
bbpos_types.h
…
bbpos.h
…
bcachefs_format.h
bcachefs: jset_entry_datetime
2024-03-10 15:34:08 -04:00
bcachefs_ioctl.h
bcachefs: Replace zero-length array with flex-array member and use __counted_by
2024-01-01 11:47:41 -05:00
bcachefs.h
bcachefs: Clamp replicas_required to replicas
2024-02-13 20:33:38 -05:00
bkey_buf.h
…
bkey_cmp.h
…
bkey_methods.c
bcachefs: Add .val_to_text() for KEY_TYPE_cookie
2024-01-21 13:27:09 -05:00
bkey_methods.h
bcachefs: BTREE_TRIGGER_ATOMIC
2024-01-21 06:01:45 -05:00
bkey_sort.c
bcachefs: bkey_copy() is no longer a macro
2023-11-05 13:12:18 -05:00
bkey_sort.h
…
bkey.c
bcachefs: Improve move_extent tracepoint
2024-01-21 13:27:09 -05:00
bkey.h
bcachefs: Correctly validate k->u64s in btree node read path
2024-03-10 15:21:04 -04:00
bset.c
bcachefs: Prep work for variable size btree node buffers
2024-01-21 13:27:10 -05:00
bset.h
bcachefs: Prep work for variable size btree node buffers
2024-01-21 13:27:10 -05:00
btree_cache.c
bcachefs: Prep work for variable size btree node buffers
2024-01-21 13:27:10 -05:00
btree_cache.h
bcachefs: Prep work for variable size btree node buffers
2024-01-21 13:27:10 -05:00
btree_gc.c
bcachefs: Fix degraded mode fsck
2024-03-10 15:18:45 -04:00
btree_gc.h
…
btree_io.c
bcachefs: Correctly validate k->u64s in btree node read path
2024-03-10 15:21:04 -04:00
btree_io.h
bcachefs: Include btree_trans in more tracepoints
2024-01-01 11:47:40 -05:00
btree_iter.c
bcachefs: Set path->uptodate when no node at level
2024-03-10 15:30:56 -04:00
btree_iter.h
bcachefs: Fix excess transaction restarts in __bchfs_fallocate()
2024-01-21 13:27:10 -05:00
btree_journal_iter.c
bcachefs: __journal_keys_sort() refactoring
2024-01-05 23:24:19 -05:00
btree_journal_iter.h
bcachefs: Proper refcounting for journal_keys
2023-11-24 02:43:12 -05:00
btree_key_cache_types.h
bcachefs: Run btree key cache shrinker less aggressively
2023-11-13 21:45:01 -05:00
btree_key_cache.c
bcachefs: trans_for_each_path() no longer uses path->idx
2024-01-01 11:47:43 -05:00
btree_key_cache.h
bcachefs; kill bch2_btree_key_cache_flush()
2024-01-01 11:47:41 -05:00
btree_locking.c
bcachefs: Add gfp flags param to bch2_prt_task_backtrace()
2024-01-22 12:37:51 -05:00
btree_locking.h
bcachefs: Improve trace_trans_restart_relock
2024-01-21 13:27:10 -05:00
btree_trans_commit.c
bcachefs: add missing __GFP_NOWARN
2024-01-21 13:27:10 -05:00
btree_types.h
bcachefs: Improve trace_trans_restart_relock
2024-01-21 13:27:10 -05:00
btree_update_interior.c
bcachefs: Fix journal replay with unreadable btree roots
2024-03-10 15:18:13 -04:00
btree_update_interior.h
bcachefs: Fix journal replay with unreadable btree roots
2024-03-10 15:18:13 -04:00
btree_update.c
bcachefs: btree_trans always has stats
2024-01-05 23:24:19 -05:00
btree_update.h
bcachefs: Clean up btree_trans
2024-01-01 11:47:44 -05:00
btree_write_buffer_types.h
bcachefs: Inline btree write buffer sort
2024-01-01 11:47:41 -05:00
btree_write_buffer.c
bcachefs: Prep work for variable size btree node buffers
2024-01-21 13:27:10 -05:00
btree_write_buffer.h
bcachefs: __bch2_journal_key_to_wb -> bch2_journal_key_to_wb_slowpath
2024-01-05 23:24:19 -05:00
buckets_types.h
bcachefs: bch_fs_usage_base
2024-01-21 06:01:45 -05:00
buckets_waiting_for_journal_types.h
…
buckets_waiting_for_journal.c
…
buckets_waiting_for_journal.h
…
buckets.c
bcachefs: bch2_trans_account_disk_usage_change()
2024-01-21 06:01:45 -05:00
buckets.h
bcachefs: bch2_trans_account_disk_usage_change()
2024-01-21 06:01:45 -05:00
chardev.c
bcachefs: Online fsck can now fix errors
2024-01-05 23:24:20 -05:00
chardev.h
…
checksum.c
bcachefs: fixup for building in userspace
2024-03-10 15:34:08 -04:00
checksum.h
bcachefs: improve checksum error messages
2024-01-05 23:24:21 -05:00
clock_types.h
…
clock.c
bcachefs: bch2_kthread_io_clock_wait() no longer sleeps until full amount
2024-01-21 13:27:09 -05:00
clock.h
…
compress.c
bcachefs: Remove obsolete comment about zstd
2024-01-01 11:47:40 -05:00
compress.h
bcachefs: bch2_prt_compression_type()
2024-01-21 06:01:45 -05:00
darray.c
bcachefs: DARRAY_PREALLOCATED()
2024-01-01 11:46:52 -05:00
darray.h
bcachefs: Convert split_devs() to darray
2024-01-01 11:47:43 -05:00
data_update.c
bcachefs: opts->compression can now also be applied in the background
2024-01-21 13:27:10 -05:00
data_update.h
bcachefs: Data update path won't accidentaly grow replicas
2023-11-25 21:48:42 -05:00
debug.c
bcachefs: Add gfp flags param to bch2_prt_task_backtrace()
2024-01-22 12:37:51 -05:00
debug.h
…
dirent_format.h
bcachefs: dirent_format.h
2024-01-21 13:27:10 -05:00
dirent.c
bcachefs: Cleanup bch2_dirent_lookup_trans()
2024-03-10 15:34:08 -04:00
dirent.h
bcachefs: Cleanup bch2_dirent_lookup_trans()
2024-03-10 15:34:08 -04:00
disk_groups_types.h
…
disk_groups.c
bcachefs: %pg is banished
2024-01-05 23:24:21 -05:00
disk_groups.h
…
ec_format.h
bcachefs: ec_format.h
2024-01-21 13:27:11 -05:00
ec_types.h
bcachefs: Rename bch_replicas_entry -> bch_replicas_entry_v1
2024-01-01 11:47:38 -05:00
ec.c
bcachefs: helpers for printing data types
2024-01-21 06:01:45 -05:00
ec.h
bcachefs: Combine .trans_trigger, .atomic_trigger
2024-01-05 23:24:20 -05:00
errcode.c
…
errcode.h
bcachefs: Split brain detection
2024-01-05 23:24:19 -05:00
error.c
bcachefs: Don't autofix errors we can't fix
2024-01-05 23:24:20 -05:00
error.h
bcachefs: fix BCH_FSCK_ERR enum
2023-12-19 19:01:52 -05:00
extent_update.c
bcachefs: growable btree_paths
2024-01-01 11:47:44 -05:00
extent_update.h
…
extents_format.h
bcachefs: reflink_format.h
2024-01-21 13:27:11 -05:00
extents_types.h
…
extents.c
bcachefs: opts->compression can now also be applied in the background
2024-01-21 13:27:10 -05:00
extents.h
bcachefs: extent_entry_next_safe()
2024-03-10 15:12:13 -04:00
eytzinger.h
bcachefs: eytzinger_for_each() declares loop iter
2024-01-21 06:01:45 -05:00
fifo.h
…
fs-common.c
bcachefs: Cleanup bch2_dirent_lookup_trans()
2024-03-10 15:34:08 -04:00
fs-common.h
…
fs-io-buffered.c
bcachefs: Kill __GFP_NOFAIL in buffered read path
2024-02-24 20:41:42 -05:00
fs-io-buffered.h
…
fs-io-direct.c
bcachefs: fix iov_iter count underflow on sub-block dio read
2024-02-24 20:45:24 -05:00
fs-io-direct.h
…
fs-io-pagecache.c
bcachefs: Fix excess transaction restarts in __bchfs_fallocate()
2024-01-21 13:27:10 -05:00
fs-io-pagecache.h
bcachefs: Fix excess transaction restarts in __bchfs_fallocate()
2024-01-21 13:27:10 -05:00
fs-io.c
bcachefs: fix incorrect usage of REQ_OP_FLUSH
2024-01-22 12:37:51 -05:00
fs-io.h
…
fs-ioctl.c
bcachefs: unlock parent dir if entry is not found in subvolume deletion
2024-01-28 21:41:09 -05:00
fs-ioctl.h
bcachefs: x-macro-ify inode flags enum
2023-11-05 13:12:18 -05:00
fs.c
bcachefs: Fix missing bch2_err_class() calls
2024-02-10 17:37:32 -05:00
fs.h
bcachefs: kill INODE_LOCK, use lock_two_nondirectories()
2024-01-01 11:47:36 -05:00
fsck.c
bcachefs: bch2_hash_set_snapshot() -> bch2_hash_set_in_snapshot()
2024-03-10 15:34:08 -04:00
fsck.h
…
inode_format.h
bcachefs: inode_format.h
2024-01-21 13:27:10 -05:00
inode.c
bcachefs: skip invisible entries in empty subvolume checking
2024-03-10 15:34:07 -04:00
inode.h
bcachefs: Combine .trans_trigger, .atomic_trigger
2024-01-05 23:24:20 -05:00
io_misc.c
bcachefs: opts->compression can now also be applied in the background
2024-01-21 13:27:10 -05:00
io_misc.h
…
io_read.c
bcachefs: improve checksum error messages
2024-01-05 23:24:21 -05:00
io_read.h
…
io_write_types.h
…
io_write.c
bcachefs: Clamp replicas_required to replicas
2024-02-13 20:33:38 -05:00
io_write.h
closures: CLOSURE_CALLBACK() to fix type punning
2023-11-24 00:29:58 -05:00
journal_io.c
bcachefs: better journal pipelining
2024-03-10 15:34:08 -04:00
journal_io.h
bcachefs: jset_entry_datetime
2024-03-10 15:34:08 -04:00
journal_reclaim.c
bcachefs: Kill unnecessary wakeups in journal reclaim
2024-03-10 15:34:08 -04:00
journal_reclaim.h
bcachefs: btree write buffer now slurps keys from journal
2024-01-01 11:47:41 -05:00
journal_sb.c
…
journal_sb.h
…
journal_seq_blacklist.c
bcachefs: journal_seq_blacklist_add() now handles entries being added out of order
2024-03-10 15:09:59 -04:00
journal_seq_blacklist.h
…
journal_types.h
bcachefs: better journal pipelining
2024-03-10 15:34:08 -04:00
journal.c
bcachefs: better journal pipelining
2024-03-10 15:34:08 -04:00
journal.h
bcachefs: better journal pipelining
2024-03-10 15:34:08 -04:00
Kconfig
bcachefs: optimize __bch2_trans_get(), kill DEBUG_TRANSACTIONS
2024-01-01 11:47:44 -05:00
keylist_types.h
…
keylist.c
bcachefs: for_each_keylist_key() declares loop iter
2024-01-01 11:47:43 -05:00
keylist.h
bcachefs: for_each_keylist_key() declares loop iter
2024-01-01 11:47:43 -05:00
logged_ops_format.h
bcachefs: logged_ops_format.h
2024-01-21 13:27:11 -05:00
logged_ops.c
bcachefs: for_each_btree_key() now declares loop iter
2024-01-01 11:47:42 -05:00
logged_ops.h
…
lru.c
bcachefs: for_each_btree_key() now declares loop iter
2024-01-01 11:47:42 -05:00
lru.h
…
Makefile
bcachefs: counters.c -> sb-counters.c
2024-01-21 13:27:10 -05:00
mean_and_variance_test.c
…
mean_and_variance.c
bcachefs: mean and variance: fix kernel-doc for function params
2024-01-01 11:47:42 -05:00
mean_and_variance.h
bcachefs: Fix build on parisc by avoiding __multi3()
2024-01-28 21:29:23 -05:00
migrate.c
bcachefs: for_each_btree_key() now declares loop iter
2024-01-01 11:47:42 -05:00
migrate.h
…
move_types.h
…
move.c
bcachefs: opts->compression can now also be applied in the background
2024-01-21 13:27:10 -05:00
move.h
bcachefs: rebalance should wakeup on shutdown if disabled
2024-01-01 11:47:39 -05:00
movinggc.c
bcachefs: for_each_member_device() now declares loop iter
2024-01-01 11:47:42 -05:00
movinggc.h
…
nocow_locking_types.h
…
nocow_locking.c
…
nocow_locking.h
…
opts.c
bcachefs: bch2_prt_compression_type()
2024-01-21 06:01:45 -05:00
opts.h
bcachefs: no_splitbrain_check option
2024-03-10 15:12:54 -04:00
printbuf.c
bcachefs: Fix missing va_end()
2024-02-13 21:59:27 -05:00
printbuf.h
bcachefs: prt_bitflags_vector()
2024-01-01 11:47:07 -05:00
quota_format.h
bcachefs; quota_format.h
2024-01-21 13:27:10 -05:00
quota_types.h
…
quota.c
bcachefs: for_each_btree_key() now declares loop iter
2024-01-01 11:47:42 -05:00
quota.h
…
rebalance_types.h
…
rebalance.c
bcachefs: opts->compression can now also be applied in the background
2024-01-21 13:27:10 -05:00
rebalance.h
…
recovery_types.h
bcachefs: check_directory_structure() can now be run online
2024-01-01 11:47:44 -05:00
recovery.c
bcachefs: Fix journal replay with unreadable btree roots
2024-03-10 15:18:13 -04:00
recovery.h
bcachefs: bch2_run_online_recovery_passes()
2024-01-01 11:47:40 -05:00
reflink_format.h
bcachefs: reflink_format.h
2024-01-21 13:27:11 -05:00
reflink.c
bcachefs: opts->compression can now also be applied in the background
2024-01-21 13:27:10 -05:00
reflink.h
bcachefs: BTREE_TRIGGER_ATOMIC
2024-01-21 06:01:45 -05:00
replicas_types.h
bcachefs: Replace zero-length arrays with flexible-array members
2024-01-01 11:47:39 -05:00
replicas.c
bcachefs: Don't pass memcmp() as a pointer
2024-01-21 13:27:04 -05:00
replicas.h
bcachefs: Rename bch_replicas_entry -> bch_replicas_entry_v1
2024-01-01 11:47:38 -05:00
sb-clean.c
bcachefs: jset_entry_datetime
2024-03-10 15:34:08 -04:00
sb-clean.h
…
sb-counters_format.h
bcachefs: sb-counters_format.h
2024-01-21 13:27:10 -05:00
sb-counters.c
bcachefs: counters.c -> sb-counters.c
2024-01-21 13:27:10 -05:00
sb-counters.h
bcachefs: counters.c -> sb-counters.c
2024-01-21 13:27:10 -05:00
sb-downgrade.c
bcachefs: fsck_err()s don't need to manually check c->sb.version anymore
2024-01-05 23:24:21 -05:00
sb-downgrade.h
bcachefs: Upgrades now specify errors to fix, like downgrades
2024-01-05 23:24:20 -05:00
sb-errors_types.h
bcachefs: Correctly validate k->u64s in btree node read path
2024-03-10 15:21:04 -04:00
sb-errors.c
bcachefs: bch_sb_field_downgrade
2024-01-01 11:47:07 -05:00
sb-errors.h
bcachefs: bch_sb.recovery_passes_required
2024-01-01 11:47:07 -05:00
sb-members.c
bcachefs: fix missing endiannes conversion in sb_members
2024-02-10 17:37:34 -05:00
sb-members.h
bcachefs: for_each_member_device_rcu() now declares loop iter
2024-01-01 11:47:42 -05:00
seqmutex.h
…
siphash.c
…
siphash.h
…
six.c
bcachefs: six locks: Simplify optimistic spinning
2024-01-01 11:47:38 -05:00
six.h
bcachefs: six lock: fix typos
2024-01-01 11:47:40 -05:00
snapshot_format.h
bcachefs: snapshot_format.h
2024-01-21 13:27:10 -05:00
snapshot.c
bcachefs: Fix check_snapshot() memcpy
2024-02-24 20:47:47 -05:00
snapshot.h
bcachefs: Combine .trans_trigger, .atomic_trigger
2024-01-05 23:24:20 -05:00
str_hash.h
bcachefs: bch2_hash_set_snapshot() -> bch2_hash_set_in_snapshot()
2024-03-10 15:34:08 -04:00
subvolume_format.h
bcachefs: subvolume_format.h
2024-01-21 13:27:11 -05:00
subvolume_types.h
bcachefs: Fixes for rust bindgen
2024-01-01 11:47:42 -05:00
subvolume.c
bcachefs: for_each_btree_key() now declares loop iter
2024-01-01 11:47:42 -05:00
subvolume.h
bcachefs: make RO snapshots actually RO
2024-01-01 11:47:07 -05:00
super_types.h
bcachefs updates for 6.8:
2024-01-10 16:34:17 -08:00
super-io.c
bcachefs: fix kmemleak in __bch2_read_super error handling path
2024-02-10 17:37:34 -05:00
super-io.h
bcachefs: __bch2_sb_field_to_text()
2024-01-05 23:24:21 -05:00
super.c
bcachefs: Workqueues should be WQ_HIGHPRI
2024-03-10 15:34:08 -04:00
super.h
bcachefs: convert bch_fs_flags to x-macro
2024-01-01 11:47:38 -05:00
sysfs.c
bcachefs: opts->compression can now also be applied in the background
2024-01-21 13:27:10 -05:00
sysfs.h
…
tests.c
bcachefs: for_each_btree_key() now declares loop iter
2024-01-01 11:47:42 -05:00
tests.h
…
thread_with_file_types.h
bcachefs: factor out thread_with_file, thread_with_stdio
2024-01-05 23:24:19 -05:00
thread_with_file.c
bcachefs: install fd later to avoid race with close
2024-02-05 01:16:15 -05:00
thread_with_file.h
bcachefs: factor out thread_with_file, thread_with_stdio
2024-01-05 23:24:19 -05:00
trace.c
…
trace.h
bcachefs: Prep work for variable size btree node buffers
2024-01-21 13:27:10 -05:00
two_state_shared_lock.c
…
two_state_shared_lock.h
…
util.c
bcachefs: fix bch2_save_backtrace()
2024-02-25 15:45:36 -05:00
util.h
bcachefs: Add gfp flags param to bch2_prt_task_backtrace()
2024-01-22 12:37:51 -05:00
varint.c
…
varint.h
…
vstructs.h
bcachefs: vstruct_for_each() now declares loop iter
2024-01-01 11:47:42 -05:00
xattr_format.h
bcachefs: xattr_format.h
2024-01-21 13:27:10 -05:00
xattr.c
bcachefs: opts->compression can now also be applied in the background
2024-01-21 13:27:10 -05:00
xattr.h
…