linux/fs/xfs
Darrick J. Wong 18a1e644b0 xfs: define an in-memory btree for storing refcount bag info during repairs
Create a new in-memory btree type so that we can store refcount bag info
in a much more memory-efficient and performant format.  Recall that the
refcount recordset regenerator computes the new recordset from browsing
the rmap records.  Let's say that the rmap records are:

{agbno: 10, length: 40, ...}
{agbno: 11, length: 3, ...}
{agbno: 12, length: 20, ...}
{agbno: 15, length: 1, ...}

It is convenient to have a data structure that could quickly tell us the
refcount for an arbitrary agbno without wasting memory.  An array or a
list could do that pretty easily.  List suck because of the pointer
overhead.  xfarrays are a lot more compact, but we want to minimize
sparse holes in the xfarray to constrain memory usage.  Maintaining any
kind of record order isn't needed for correctness, so I created the
"rcbag", which is shorthand for an unordered list of (excerpted) reverse
mappings.

So we add the first rmap to the rcbag, and it looks like:

0: {agbno: 10, length: 40}

The refcount for agbno 10 is 1.  Then we move on to block 11, so we add
the second rmap:

0: {agbno: 10, length: 40}
1: {agbno: 11, length: 3}

The refcount for agbno 11 is 2.  We move on to block 12, so we add the
third:

0: {agbno: 10, length: 40}
1: {agbno: 11, length: 3}
2: {agbno: 12, length: 20}

The refcount for agbno 12 and 13 is 3.  We move on to block 14, and
remove the second rmap:

0: {agbno: 10, length: 40}
1: NULL
2: {agbno: 12, length: 20}

The refcount for agbno 14 is 2.  We move on to block 15, and add the
last rmap.  But we don't care where it is and we don't want to expand
the array so we put it in slot 1:

0: {agbno: 10, length: 40}
1: {agbno: 15, length: 1}
2: {agbno: 12, length: 20}

The refcount for block 15 is 3.  Notice how order doesn't matter in this
list?  That's why repair uses an unordered list, or "bag".  The data
structure is not a set because it does not guarantee uniqueness.

That said, adding and removing specific items is now an O(n) operation
because we have no idea where that item might be in the list.  Overall,
the runtime is O(n^2) which is bad.

I realized that I could easily refactor the btree code and reimplement
the refcount bag with an xfbtree.  Adding and removing is now O(log2 n),
so the runtime is at least O(n log2 n), which is much faster.  In the
end, the rcbag becomes a sorted list, but that's merely a detail of the
implementation.  The repair code doesn't care.

(Note: That horrible xfs_db bmap_inflate command can be used to exercise
this sort of rcbag insanity by cranking up refcounts quickly.)

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-02-22 12:43:40 -08:00
..
libxfs xfs: hook live rmap operations during a repair operation 2024-02-22 12:43:40 -08:00
scrub xfs: define an in-memory btree for storing refcount bag info during repairs 2024-02-22 12:43:40 -08:00
Kconfig xfs: support in-memory btrees 2024-02-22 12:43:35 -08:00
Makefile xfs: define an in-memory btree for storing refcount bag info during repairs 2024-02-22 12:43:40 -08:00
xfs_acl.c xfs: convert kmem_free() for kvmalloc users to kvfree() 2024-02-13 18:07:34 +05:30
xfs_acl.h
xfs_aops.c
xfs_aops.h
xfs_attr_inactive.c xfs: report dir/attr block corruption errors to the health system 2024-02-22 12:32:18 -08:00
xfs_attr_item.c xfs: place intent recovery under NOFS allocation context 2024-02-13 18:07:35 +05:30
xfs_attr_item.h
xfs_attr_list.c xfs: report XFS_IS_CORRUPT errors to the health system 2024-02-22 12:32:55 -08:00
xfs_bio_io.c
xfs_bmap_item.c xfs: place intent recovery under NOFS allocation context 2024-02-13 18:07:35 +05:30
xfs_bmap_item.h
xfs_bmap_util.c xfs: Replace xfs_isilocked with xfs_assert_ilocked 2024-02-19 21:19:33 +05:30
xfs_bmap_util.h xfs: indicate if xfs_bmap_adjacent changed ap->blkno 2023-12-22 11:18:11 +05:30
xfs_buf_item_recover.c xfs: convert remaining kmem_free() to kfree() 2024-02-13 18:07:34 +05:30
xfs_buf_item.c xfs: convert remaining kmem_free() to kfree() 2024-02-13 18:07:34 +05:30
xfs_buf_item.h
xfs_buf_mem.c xfs: launder in-memory btree buffers before transaction commit 2024-02-22 12:43:36 -08:00
xfs_buf_mem.h xfs: launder in-memory btree buffers before transaction commit 2024-02-22 12:43:36 -08:00
xfs_buf.c xfs: support in-memory buffer cache targets 2024-02-22 12:43:21 -08:00
xfs_buf.h xfs: support in-memory buffer cache targets 2024-02-22 12:43:21 -08:00
xfs_dahash_test.c
xfs_dahash_test.h
xfs_dir2_readdir.c xfs: report XFS_IS_CORRUPT errors to the health system 2024-02-22 12:32:55 -08:00
xfs_discard.c xfs: split xfs_allocbt_init_cursor 2024-02-22 12:40:12 -08:00
xfs_discard.h
xfs_dquot_item_recover.c
xfs_dquot_item.c
xfs_dquot_item.h
xfs_dquot.c xfs: report quota block corruption errors to the health system 2024-02-22 12:32:44 -08:00
xfs_dquot.h xfs: repair quotas 2023-12-15 10:03:45 -08:00
xfs_drain.c
xfs_drain.h
xfs_error.c xfs: convert remaining kmem_free() to kfree() 2024-02-13 18:07:34 +05:30
xfs_error.h
xfs_export.c
xfs_export.h
xfs_extent_busy.c xfs: convert remaining kmem_free() to kfree() 2024-02-13 18:07:34 +05:30
xfs_extent_busy.h
xfs_extfree_item.c xfs: convert remaining kmem_free() to kfree() 2024-02-13 18:07:34 +05:30
xfs_extfree_item.h
xfs_file.c xfs: Replace xfs_isilocked with xfs_assert_ilocked 2024-02-19 21:19:33 +05:30
xfs_filestream.c xfs: convert remaining kmem_free() to kfree() 2024-02-13 18:07:34 +05:30
xfs_filestream.h
xfs_fsmap.c xfs: split xfs_allocbt_init_cursor 2024-02-22 12:40:12 -08:00
xfs_fsmap.h
xfs_fsops.c New code for 6.8: 2024-01-10 08:45:22 -08:00
xfs_fsops.h
xfs_globals.c
xfs_health.c xfs: support in-memory btrees 2024-02-22 12:43:35 -08:00
xfs_hooks.c xfs: allow scrub to hook metadata updates in other writers 2024-02-22 12:30:45 -08:00
xfs_hooks.h xfs: allow scrub to hook metadata updates in other writers 2024-02-22 12:30:45 -08:00
xfs_icache.c xfs: report inode corruption errors to the health system 2024-02-22 12:32:43 -08:00
xfs_icache.h
xfs_icreate_item.c xfs: convert kmem_free() for kvmalloc users to kvfree() 2024-02-13 18:07:34 +05:30
xfs_icreate_item.h
xfs_inode_item_recover.c xfs: convert remaining kmem_free() to kfree() 2024-02-13 18:07:34 +05:30
xfs_inode_item.c xfs: Replace xfs_isilocked with xfs_assert_ilocked 2024-02-19 21:19:33 +05:30
xfs_inode_item.h
xfs_inode.c xfs: remember sick inodes that get inactivated 2024-02-22 12:33:03 -08:00
xfs_inode.h xfs: track directory entry updates during live nlinks fsck 2024-02-22 12:30:59 -08:00
xfs_ioctl32.c
xfs_ioctl32.h
xfs_ioctl.c xfs: use kvfree in xfs_ioc_getfsmap() 2024-02-20 10:38:14 +05:30
xfs_ioctl.h
xfs_iomap.c xfs: report block map corruption errors to the health tracking system 2024-02-22 12:31:51 -08:00
xfs_iomap.h
xfs_iops.c xfs: Remove mrlock wrapper 2024-02-19 21:19:33 +05:30
xfs_iops.h
xfs_itable.c xfs: convert remaining kmem_free() to kfree() 2024-02-13 18:07:34 +05:30
xfs_itable.h
xfs_iunlink_item.c
xfs_iunlink_item.h
xfs_iwalk.c xfs: remove xfs_btree_reada_bufs 2024-02-22 12:41:01 -08:00
xfs_iwalk.h
xfs_linux.h xfs: allow scrub to hook metadata updates in other writers 2024-02-22 12:30:45 -08:00
xfs_log_cil.c xfs: place the CIL under nofs allocation context 2024-02-13 18:07:35 +05:30
xfs_log_priv.h
xfs_log_recover.c xfs: ensure submit buffers on LSN boundaries in error handlers 2024-02-17 09:34:52 +05:30
xfs_log.c xfs: remove the xfs_buftarg_t typedef 2024-02-22 12:42:44 -08:00
xfs_log.h
xfs_message.c
xfs_message.h
xfs_mount.c xfs: convert remaining kmem_free() to kfree() 2024-02-13 18:07:34 +05:30
xfs_mount.h xfs: teach buftargs to maintain their own buffer hashtable 2024-02-22 12:42:58 -08:00
xfs_mru_cache.c xfs: use GFP_KERNEL in pure transaction contexts 2024-02-13 18:07:35 +05:30
xfs_mru_cache.h
xfs_notify_failure.c
xfs_pnfs.c
xfs_pnfs.h
xfs_pwork.c
xfs_pwork.h
xfs_qm_bhv.c xfs: track quota updates during live quotacheck 2024-02-22 12:30:55 -08:00
xfs_qm_syscalls.c
xfs_qm.c xfs: report quota block corruption errors to the health system 2024-02-22 12:32:44 -08:00
xfs_qm.h xfs: track quota updates during live quotacheck 2024-02-22 12:30:55 -08:00
xfs_quota.h xfs: track quota updates during live quotacheck 2024-02-22 12:30:55 -08:00
xfs_quotaops.c
xfs_refcount_item.c xfs: place intent recovery under NOFS allocation context 2024-02-13 18:07:35 +05:30
xfs_refcount_item.h
xfs_reflink.c xfs: report block map corruption errors to the health tracking system 2024-02-22 12:31:51 -08:00
xfs_reflink.h
xfs_rmap_item.c xfs: place intent recovery under NOFS allocation context 2024-02-13 18:07:35 +05:30
xfs_rmap_item.h
xfs_rtalloc.c xfs: report realtime metadata corruption errors to the health system 2024-02-22 12:32:44 -08:00
xfs_rtalloc.h xfs: move xfs_bmap_rtalloc to xfs_rtalloc.c 2023-12-22 11:18:11 +05:30
xfs_stats.c xfs: define an in-memory btree for storing refcount bag info during repairs 2024-02-22 12:43:40 -08:00
xfs_stats.h xfs: define an in-memory btree for storing refcount bag info during repairs 2024-02-22 12:43:40 -08:00
xfs_super.c xfs: track directory entry updates during live nlinks fsck 2024-02-22 12:30:59 -08:00
xfs_super.h
xfs_symlink.c xfs: report symlink block corruption errors to the health system 2024-02-22 12:32:42 -08:00
xfs_symlink.h
xfs_sysctl.c fs: Remove the now superfluous sentinel elements from ctl_table array 2023-12-28 04:57:57 -08:00
xfs_sysctl.h
xfs_sysfs.c xfs: remove duplicate ifdefs 2024-02-17 09:32:32 +05:30
xfs_sysfs.h
xfs_trace.c xfs: support in-memory btrees 2024-02-22 12:43:35 -08:00
xfs_trace.h xfs: launder in-memory btree buffers before transaction commit 2024-02-22 12:43:36 -08:00
xfs_trans_ail.c xfs: convert remaining kmem_free() to kfree() 2024-02-13 18:07:34 +05:30
xfs_trans_buf.c xfs: launder in-memory btree buffers before transaction commit 2024-02-22 12:43:36 -08:00
xfs_trans_dquot.c xfs: track quota updates during live quotacheck 2024-02-22 12:30:55 -08:00
xfs_trans_priv.h
xfs_trans.c xfs: Replace xfs_isilocked with xfs_assert_ilocked 2024-02-19 21:19:33 +05:30
xfs_trans.h xfs: launder in-memory btree buffers before transaction commit 2024-02-22 12:43:36 -08:00
xfs_xattr.c xfs: set inode sick state flags when we zap either ondisk fork 2023-12-15 10:03:35 -08:00
xfs_xattr.h
xfs.h