iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
8837c70d53
linux/lib
History
Matthew Wilcox fa290cda10 radix tree: use GFP_ZONEMASK bits of gfp_t for flags 
Patch series "XArray", v9.  (First part thereof).

This patchset is, I believe, appropriate for merging for 4.17.  It
contains the XArray implementation, to eventually replace the radix
tree, and converts the page cache to use it.

This conversion keeps the radix tree and XArray data structures in sync
at all times.  That allows us to convert the page cache one function at
a time and should allow for easier bisection.  Other than renaming some
elements of the structures, the data structures are fundamentally
unchanged; a radix tree walk and an XArray walk will touch the same
number of cachelines.  I have changes planned to the XArray data
structure, but those will happen in future patches.

Improvements the XArray has over the radix tree:

 - The radix tree provides operations like other trees do; 'insert' and
   'delete'. But what most users really want is an automatically
   resizing array, and so it makes more sense to give users an API that
   is like an array -- 'load' and 'store'. We still have an 'insert'
   operation for users that really want that semantic.

 - The XArray considers locking as part of its API. This simplifies a
   lot of users who formerly had to manage their own locking just for
   the radix tree. It also improves code generation as we can now tell
   RCU that we're holding a lock and it doesn't need to generate as much
   fencing code. The other advantage is that tree nodes can be moved
   (not yet implemented).

 - GFP flags are now parameters to calls which may need to allocate
   memory. The radix tree forced users to decide what the allocation
   flags would be at creation time. It's much clearer to specify them at
   allocation time.

 - Memory is not preloaded; we don't tie up dozens of pages on the off
   chance that the slab allocator fails. Instead, we drop the lock,
   allocate a new node and retry the operation. We have to convert all
   the radix tree, IDA and IDR preload users before we can realise this
   benefit, but I have not yet found a user which cannot be converted.

 - The XArray provides a cmpxchg operation. The radix tree forces users
   to roll their own (and at least four have).

 - Iterators take a 'max' parameter. That simplifies many users and will
   reduce the amount of iteration done.

 - Iteration can proceed backwards. We only have one user for this, but
   since it's called as part of the pagefault readahead algorithm, that
   seemed worth mentioning.

 - RCU-protected pointers are not exposed as part of the API. There are
   some fun bugs where the page cache forgets to use rcu_dereference()
   in the current codebase.

 - Value entries gain an extra bit compared to radix tree exceptional
   entries. That gives us the extra bit we need to put huge page swap
   entries in the page cache.

 - Some iterators now take a 'filter' argument instead of having
   separate iterators for tagged/untagged iterations.

The page cache is improved by this:

 - Shorter, easier to read code

 - More efficient iterations

 - Reduction in size of struct address_space

 - Fewer walks from the top of the data structure; the XArray API
   encourages staying at the leaf node and conducting operations there.

This patch (of 8):

None of these bits may be used for slab allocations, so we can use them
as radix tree flags as long as we mask them off before passing them to
the slab allocator. Move the IDR flag from the high bits to the
GFP_ZONEMASK bits.

Link: http://lkml.kernel.org/r/20180313132639.17387-3-willy@infradead.org
Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
Acked-by: Jeff Layton <jlayton@kernel.org>
Cc: Darrick J. Wong <darrick.wong@oracle.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-04-11 10:28:39 -07:00
..
842 License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fonts License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
lz4 lib/lz4: make arrays static const, reduces object code size 2017-10-03 17:54:25 -07:00
lzo License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
mpi lib/mpi: Fix umul_ppmm() for MIPS64r6 2017-12-22 19:39:09 +11:00
raid6 powerpc updates for 4.17 2018-04-07 12:08:19 -07:00
reed_solomon
xz lib/xz: Add fall-through comments to a switch statement 2017-10-12 15:10:12 +02:00
zlib_deflate
zlib_inflate lib/zlib_inflate/inftrees.c: fix potential buffer overflow 2017-05-08 17:15:12 -07:00
zstd lib: zstd: clean up Makefile for simpler composite object handling 2018-03-26 02:01:27 +09:00
.gitignore
argv_split.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ashldi3.c move libgcc.h to include/linux 2017-12-01 13:09:40 -08:00
ashrdi3.c move libgcc.h to include/linux 2017-12-01 13:09:40 -08:00
asn1_decoder.c ASN.1: check for error from ASN1_OP_END__ACT actions 2017-12-08 15:13:27 +00:00
assoc_array.c lib/assoc_array: Remove smp_read_barrier_depends() 2017-12-04 10:52:56 -08:00
atomic64_test.c lib/atomic64_test.c: add a test that atomic64_inc_not_zero() returns an int 2017-07-14 15:05:13 -07:00
atomic64.c
audit.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
bcd.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
bch.c
bitmap.c lib: fix stall in __bitmap_parselist() 2018-04-05 21:36:21 -07:00
bitrev.c
bsearch.c lib/bsearch.c: micro-optimize pivot position calculation 2017-07-10 16:32:35 -07:00
btree.c btree: avoid variable-length allocations 2018-03-14 16:55:29 -07:00
bucket_locks.c spinlock: Add library function to allocate spinlock buckets array 2017-12-11 09:58:39 -05:00
bug.c lib/bug.c: exclude non-BUG/WARN exceptions from report_bug() 2018-03-09 16:40:01 -08:00
build_OID_registry
bust_spinlocks.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
chacha20.c crypto: chacha20 - use rol32() macro from bitops.h 2018-01-12 23:03:01 +11:00
check_signature.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
checksum.c
clz_ctz.c
clz_tab.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cmdline.c lib/cmdline.c: remove meaningless comment 2017-09-08 18:26:49 -07:00
cmpdi2.c move libgcc.h to include/linux 2017-12-01 13:09:40 -08:00
compat_audit.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cordic.c
cpu_rmap.c
cpumask.c lib: optimize cpumask_next_and() 2018-02-06 18:32:44 -08:00
crc4.c docs: clean up and add rest of CRC functions to kernel-api.rst 2017-09-26 15:01:20 -06:00
crc7.c
crc8.c docs: clean up and add rest of CRC functions to kernel-api.rst 2017-09-26 15:01:20 -06:00
crc16.c
crc32.c docs: clean up and add rest of CRC functions to kernel-api.rst 2017-09-26 15:01:20 -06:00
crc32defs.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
crc32test.c
crc-ccitt.c lib/crc-ccitt: Add CCITT-FALSE CRC16 variant 2018-01-08 10:08:33 +00:00
crc-itu-t.c
crc-t10dif.c
ctype.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
debug_info.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
debug_locks.c
debugobjects.c debugobjects: Avoid another unused variable warning 2018-03-14 20:20:01 +01:00
dec_and_lock.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
decompress_bunzip2.c
decompress_inflate.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
decompress_unlz4.c
decompress_unlzma.c
decompress_unlzo.c
decompress_unxz.c
decompress.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
devres.c devres: combine function devm_ioremap* 2018-03-15 18:08:55 +01:00
digsig.c lib/digsig: fix dereference of NULL user_key_payload 2017-10-12 17:16:40 +01:00
div64.c A relatively calm cycle for the docs tree again. 2017-11-13 08:25:06 -08:00
dma-debug.c dma-debug: fix memory leak in debug_dma_alloc_coherent 2018-02-22 15:02:33 -08:00
dma-direct.c dma-mapping: Don't clear GFP_ZERO in dma_alloc_attrs 2018-03-28 17:34:23 +02:00
dma-virt.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
dump_stack.c printk: move dump stack related code to lib/dump_stack.c 2018-03-15 13:25:36 +01:00
dynamic_debug.c dynamic-debug-howto: fix optional/omitted ending line number to be LARGE instead of 0 2017-11-17 16:10:01 -08:00
dynamic_queue_limits.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2017-11-15 11:56:19 -08:00
earlycpio.c
error-inject.c error-injection: Add injectable error types 2018-01-12 17:33:38 -08:00
errseq.c errseq: Add to documentation tree 2018-01-01 12:40:27 -07:00
extable.c lib/extable.c: use bsearch() library function in search_extable() 2017-07-10 16:32:35 -07:00
fault-inject.c fault-inject: fix wrong should_fail() decision in task context 2017-08-10 15:54:06 -07:00
fdt_empty_tree.c
fdt_ro.c
fdt_rw.c
fdt_strerror.c
fdt_sw.c
fdt_wip.c
fdt.c
find_bit_benchmark.c lib: optimize cpumask_next_and() 2018-02-06 18:32:44 -08:00
find_bit.c lib: optimize cpumask_next_and() 2018-02-06 18:32:44 -08:00
flex_array.c
flex_proportions.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
gcd.c lib/gcd: add kernel-doc notation 2017-10-07 10:45:14 -06:00
gen_crc32table.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
genalloc.c lib/genalloc.c: make the avail variable an atomic_long_t 2017-11-17 16:10:02 -08:00
glob.c
globtest.c
hexdump.c lib/hexdump.c: return -EINVAL in case of error in hex2bin() 2017-09-08 18:26:49 -07:00
hweight.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
idr.c idr: Fix handling of IDs above INT_MAX 2018-02-26 14:39:30 -05:00
inflate.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
int_sqrt.c lib: Add strongly typed 64bit int_sqrt 2018-02-04 10:17:21 +00:00
interval_tree_test.c lib/rbtree-test: lower default params 2017-11-17 16:10:02 -08:00
interval_tree.c
iomap_copy.c
iomap.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
iommu-common.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
iommu-helper.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ioremap.c mm/vmalloc: add interfaces to free unmapped page table 2018-03-22 17:07:01 -07:00
iov_iter.c new primitive: iov_iter_for_each_range() 2017-10-11 22:36:54 -04:00
irq_poll.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
irq_regs.c
is_single_threaded.c
jedec_ddr_data.c
kasprintf.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
Kconfig lib: Add generic PIO mapping method 2018-03-21 17:18:34 -05:00
Kconfig.debug lib/Kconfig.debug: Debug Lockups and Hangs: keep SOFTLOCKUP options together 2018-04-11 10:28:35 -07:00
Kconfig.kasan kasan: rework Kconfig settings 2018-02-06 18:32:47 -08:00
Kconfig.kgdb lib: update location of kgdb documentation 2017-05-16 08:44:22 -03:00
Kconfig.ubsan lib: add testing module for UBSAN 2018-04-11 10:28:35 -07:00
kfifo.c kfifo: fix inaccurate comment 2018-03-27 11:15:42 +02:00
klist.c
kobject_uevent.c net: Drop pernet_operations::async 2018-03-27 13:18:09 -04:00
kobject.c lib/kobject: Join string literals back 2018-03-15 14:38:55 +01:00
kstrtox.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
kstrtox.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
lcm.c
libcrc32c.c libcrc32c: Add crc32c_impl function 2018-03-26 15:09:38 +02:00
list_debug.c lib/list_debug.c: print unmangled addresses 2018-04-11 10:28:35 -07:00
list_sort.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
llist.c locking/atomics: COCCINELLE/treewide: Convert trivial ACCESS_ONCE() patterns to READ_ONCE()/WRITE_ONCE() 2017-10-25 11:01:08 +02:00
locking-selftest-hardirq.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
locking-selftest-mutex.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
locking-selftest-rsem.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
locking-selftest-rtmutex.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
locking-selftest-softirq.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
locking-selftest-wsem.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
locking-selftest.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
lockref.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
logic_pio.c lib: Add generic PIO mapping method 2018-03-21 17:18:34 -05:00
lru_cache.c
lshrdi3.c move libgcc.h to include/linux 2017-12-01 13:09:40 -08:00
Makefile lib: add testing module for UBSAN 2018-04-11 10:28:35 -07:00
memory-notifier-error-inject.c
memweight.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
muldi3.c move libgcc.h to include/linux 2017-12-01 13:09:40 -08:00
net_utils.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
netdev-notifier-error-inject.c
nlattr.c netlink: Relax attr validation for fixed length types 2017-12-07 14:00:57 -05:00
nmi_backtrace.c lib/nmi_backtrace.c: fix kernel text address leak 2017-11-17 16:10:02 -08:00
nodemask.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
notifier-error-inject.c
notifier-error-inject.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
of-reconfig-notifier-error-inject.c
oid_registry.c 509: fix printing uninitialized stack memory when OID is empty 2017-12-08 15:13:28 +00:00
once.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2017-11-04 09:26:51 +09:00
parman.c
parser.c
pci_iomap.c PCI: Add SPDX GPL-2.0 when no license was specified 2018-01-26 11:45:16 -06:00
percpu_counter.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
percpu_ida.c
percpu_test.c
percpu-refcount.c percpu_ref: Update doc to dissuade users from depending on internal RCU grace periods 2018-03-19 10:09:44 -07:00
plist.c
pm-notifier-error-inject.c
prime_numbers.c
radix-tree.c radix tree: use GFP_ZONEMASK bits of gfp_t for flags 2018-04-11 10:28:39 -07:00
random32.c treewide: Switch DEFINE_TIMER callbacks to struct timer_list * 2017-11-21 15:57:05 -08:00
ratelimit.c lib/ratelimit.c: use deferred printk() version 2017-10-03 17:54:26 -07:00
rational.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
rbtree_test.c lib/rbtree-test: lower default params 2017-11-17 16:10:02 -08:00
rbtree.c lib/rbtree,drm/mm: add rbtree_replace_node_cached() 2017-12-14 16:00:48 -08:00
reciprocal_div.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
refcount.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
rhashtable.c rhashtable: add schedule points 2018-03-31 23:25:39 -04:00
sbitmap.c sbitmap: use test_and_set_bit_lock()/clear_bit_unlock() 2018-02-28 12:23:35 -07:00
scatterlist.c lib/scatterlist: add sg_init_marker() helper 2018-03-30 22:50:15 +02:00
seq_buf.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sg_pool.c
sg_split.c
sha1.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
show_mem.c
siphash.c
smp_processor_id.c lib: do not use print_symbol() 2018-01-05 15:24:00 +01:00
sort.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
stackdepot.c lib/stackdepot.c: use a non-instrumented version of memcmp() 2018-02-06 18:32:44 -08:00
stmp_device.c
string_helpers.c mm: treewide: remove GFP_TEMPORARY allocation flag 2017-09-13 18:53:16 -07:00
string.c lib/strscpy: Shut up KASAN false-positives in strscpy() 2018-02-01 12:20:21 -08:00
strncpy_from_user.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
strnlen_user.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
swiotlb.c swiotlb: Make swiotlb_{alloc,free}_buffer depend on CONFIG_DMA_DIRECT_OPS 2018-03-23 20:15:38 +01:00
syscall.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
test_bitmap.c lib/test_bitmap.c: do not accidentally use stack VLA 2018-04-11 10:28:35 -07:00
test_bpf.c test_bpf: Fix NULL vs IS_ERR() check in test_skb_segment() 2018-03-29 14:33:29 -04:00
test_debug_virtual.c lib: add test module for CONFIG_DEBUG_VIRTUAL 2017-09-08 18:26:49 -07:00
test_firmware.c headers: untangle kmemleak.h from mm.h 2018-04-05 21:36:27 -07:00
test_hash.c
test_hexdump.c
test_kasan.c kasan: fix invalid-free test crashing the kernel 2018-04-11 10:28:32 -07:00
test_kmod.c lib/test_kmod.c: fix limit check on number of test devices created 2018-03-09 16:40:02 -08:00
test_list_sort.c lib/test: delete five error messages for failed memory allocations 2017-11-17 16:10:01 -08:00
test_module.c
test_parman.c
test_printf.c printk: hash addresses printed with %p 2017-11-29 12:09:02 +11:00
test_rhashtable.c test_rhashtable: add test case for rhltable with duplicate objects 2018-03-07 10:44:03 -05:00
test_siphash.c
test_sort.c lib/test_sort.c: add module unload support 2018-02-06 18:32:45 -08:00
test_static_key_base.c
test_static_keys.c
test_string.c lib: add module support to string tests 2017-11-17 16:10:01 -08:00
test_sysctl.c test_sysctl: test against int proc_dointvec() array support 2017-07-12 16:26:00 -07:00
test_ubsan.c lib/test_ubsan.c: make test_ubsan_misaligned_access() static 2018-04-11 10:28:35 -07:00
test_user_copy.c treewide: simplify Kconfig dependencies for removed archs 2018-03-26 15:55:57 +02:00
test_uuid.c uuid: fix incorrect uuid_equal conversion in test_uuid_test 2017-07-21 09:38:30 +02:00
test-kstrtox.c
test-string_helpers.c
textsearch.c
timerqueue.c timerqueue: Document return values of timerqueue_add/del() 2017-12-29 23:13:10 +01:00
ts_bm.c
ts_fsm.c textsearch: fix typos in library helpers 2017-10-22 03:14:07 +01:00
ts_kmp.c textsearch: fix typos in library helpers 2017-10-22 03:14:07 +01:00
ubsan.c lib/ubsan: remove returns-nonnull-attribute checks 2018-02-06 18:32:46 -08:00
ubsan.h lib/ubsan: remove returns-nonnull-attribute checks 2018-02-06 18:32:46 -08:00
ucmpdi2.c move libgcc.h to include/linux 2017-12-01 13:09:40 -08:00
ucs2_string.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
usercopy.c Fix misannotated out-of-line _copy_to_user() 2017-12-11 09:35:11 -05:00
uuid.c Documentation: add UUID/GUID to kernel-api 2017-12-11 15:03:08 -07:00
vsprintf.c proc: add seq_put_decimal_ull_width to speed up /proc/pid/smaps 2018-04-11 10:28:33 -07:00
win_minmax.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xxhash.c lib: Add xxhash module 2017-08-15 09:02:07 -07:00