linux/lib
Liam Howlett 541e06b772 maple_tree: remove GFP_ZERO from kmem_cache_alloc() and kmem_cache_alloc_bulk()
Preallocations are common in the VMA code to avoid allocating under
certain locking conditions.  The preallocations must also cover the
worst-case scenario.  Removing the GFP_ZERO flag from the
kmem_cache_alloc() (and bulk variant) calls will reduce the amount of time
spent zeroing memory that may not be used.  Only zero out the necessary
area to keep track of the allocations in the maple state.  Zero the entire
node prior to using it in the tree.

This required internal changes to node counting on allocation, so the test
code is also updated.

This restores some micro-benchmark performance: up to +9% in mmtests mmap1
by my testing +10% to +20% in mmap, mmapaddr, mmapmany tests reported by
Red Hat

Link: https://bugzilla.redhat.com/show_bug.cgi?id=2149636
Link: https://lkml.kernel.org/r/20230105160427.2988454-1-Liam.Howlett@oracle.com
Signed-off-by: Liam Howlett <Liam.Howlett@oracle.com>
Reported-by: Jirka Hladky <jhladky@redhat.com>
Suggested-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-01-18 17:12:54 -08:00
..
842
crypto crypto: lib/aesgcm - Provide minimal library implementation 2022-11-11 18:14:59 +08:00
dim dim: initialize all struct fields 2022-05-09 17:20:37 -07:00
fonts lib/fonts: fix undefined behavior in bit shift for get_default_font 2022-11-18 13:55:09 -08:00
kunit kunit: alloc_string_stream_fragment error handling bug fix 2022-12-26 16:01:36 -07:00
livepatch selftests/livepatch: better synchronize test_klp_callbacks_busy 2022-06-15 10:29:10 +02:00
lz4 lib: make LZ4_decompress_safe_forceExtDict() static 2022-07-17 17:31:39 -07:00
lzo lib/lzo/lzo1x_compress.c: replace ternary operator with min() and min_t() 2022-07-29 18:12:34 -07:00
math math64: favor kernel-doc from header files 2022-11-21 14:30:53 -07:00
mpi Updates to various subsystems which I help look after. lib, ocfs2, 2022-08-07 10:03:24 -07:00
pldmfw lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
raid6 for-6.2/block-2022-12-08 2022-12-13 10:43:59 -08:00
reed_solomon treewide: use get_random_u32_below() instead of deprecated function 2022-11-18 02:15:15 +01:00
test_fortify fortify: Detect struct member overflows in memset() at compile-time 2022-02-13 16:50:06 -08:00
vdso lib/vdso: use "grep -E" instead of "egrep" 2022-11-23 19:50:15 +01:00
xz lib/xz, lib/decompress_unxz.c: Fix spelling in comments 2021-10-19 23:44:30 +08:00
zlib_deflate
zlib_dfltcc zlib: move EXPORT_SYMBOL() and MODULE_LICENSE() out of dfltcc_syms.c 2020-12-29 15:36:49 -08:00
zlib_inflate lib/zlib_inflate/inffast: check config in C to avoid unused function warning 2021-09-24 16:13:35 -07:00
zstd zstd: import usptream v1.5.2 2022-10-24 12:12:32 -07:00
.gitignore bootconfig: Support embedding a bootconfig file in kernel 2022-04-26 17:58:51 -04:00
argv_split.c
ashldi3.c
ashrdi3.c
asn1_decoder.c
asn1_encoder.c lib: remove redundant assignment to variable ret 2022-01-20 08:52:55 +02:00
assoc_array.c assoc_array: Fix BUG_ON during garbage collect 2022-06-01 18:29:06 -07:00
atomic64_test.c
atomic64.c locking/atomic: atomic64: Remove unusable atomic ops 2021-12-13 10:56:09 +01:00
audit.c audit: add support for the openat2 syscall 2021-10-01 16:52:48 -04:00
base64.c lib/base64: RFC4648-compliant base64 encoding 2022-08-02 17:14:47 -06:00
bcd.c
bch.c lib/bch.c: fix a typo in the file bch.c 2021-05-06 19:24:12 -07:00
bitfield_kunit.c
bitmap.c lib/bitmap: remove bitmap_ord_to_pos 2022-09-26 12:19:12 -07:00
bitrev.c
bootconfig-data.S bootconfig: Support embedding a bootconfig file in kernel 2022-04-26 17:58:51 -04:00
bootconfig.c bootconfig: Support embedding a bootconfig file in kernel 2022-04-26 17:58:51 -04:00
bsearch.c
btree.c lib/btree: simplify btree_{lookup|update} 2022-06-16 19:58:21 -07:00
bucket_locks.c
bug.c bug: Use normal relative pointers in 'struct bug_entry' 2022-05-19 23:46:10 +02:00
build_OID_registry
buildid.c kdump: use vmlinux_build_id to simplify 2021-07-08 11:48:22 -07:00
bust_spinlocks.c kernel/panic: Drop unblank_screen call 2022-09-01 16:55:35 +02:00
check_signature.c
checksum.c
clz_ctz.c
clz_tab.c
cmdline_kunit.c treewide: use get_random_{u8,u16}() when possible, part 1 2022-10-11 17:42:58 -06:00
cmdline.c lib/cmdline: avoid page fault in next_arg 2022-09-11 21:55:06 -07:00
cmpdi2.c
compat_audit.c audit: add support for the openat2 syscall 2021-10-01 16:52:48 -04:00
cpu_rmap.c
cpumask_kunit.c lib/test_cpumask: Add for_each_cpu_and(not) tests 2022-10-06 05:57:36 -07:00
cpumask.c lib/find_bit: add find_next{,_and}_bit_wrap 2022-10-01 10:22:57 -07:00
crc4.c
crc7.c lib/crc7: fix a kernel-doc markup 2021-01-21 14:06:00 -07:00
crc8.c lib: crc8: pointer to data block should be const 2021-05-06 19:24:12 -07:00
crc16.c
crc32.c lib/crc32: Make crc32_be weak for arch override 2022-01-31 11:21:43 +11:00
crc32defs.h
crc32test.c lib/crc32test: correct printed bytes count 2022-01-31 11:21:43 +11:00
crc64-rocksoft.c crypto: add rocksoft 64b crc guard tag framework 2022-03-07 12:48:35 -07:00
crc64.c lib: add rocksoft model crc64 2022-03-07 12:48:35 -07:00
crc-ccitt.c
crc-itu-t.c crc-itu-t: fix typo in CRC ITU-T polynomial comment 2022-06-07 10:27:38 +02:00
crc-t10dif.c
ctype.c
debug_info.c isystem: ship and use stdarg.h 2021-08-19 09:02:55 +09:00
debug_locks.c locking/lockdep: Improve noinstr vs errors 2021-06-22 13:56:43 +02:00
debugobjects.c Non-MM patches for 6.2-rc1. 2022-12-12 17:28:58 -08:00
dec_and_lock.c
decompress_bunzip2.c lib/decompressors: fix spelling mistakes 2021-07-01 11:06:05 -07:00
decompress_inflate.c
decompress_unlz4.c lib/decompress_unlz4.c: correctly handle zero-padding around initrds. 2021-07-01 11:06:06 -07:00
decompress_unlzma.c lib: fix inconsistent indenting in process_bit1() 2021-05-06 19:24:12 -07:00
decompress_unlzo.c lib/decompressors: remove set but not used variabled 'level' 2021-07-01 11:06:06 -07:00
decompress_unxz.c lib/xz, lib/decompress_unxz.c: Fix spelling in comments 2021-10-19 23:44:30 +08:00
decompress_unzstd.c lib: zstd: Add decompress_sources.h for decompress_unzstd 2021-11-08 16:55:26 -08:00
decompress.c
devmem_is_allowed.c lib: use PFN_PHYS() in devmem_is_allowed() 2021-08-13 14:09:32 -10:00
devres.c devres: remove devm_ioremap_np 2022-09-01 18:04:43 +02:00
digsig.c
dump_stack.c printk: rename cpulock functions 2022-04-22 21:30:57 +02:00
dynamic_debug.c dyndbg: add drm.debug style (drm/parameters/debug) bitmap support 2022-09-07 17:04:49 +02:00
dynamic_queue_limits.c
earlycpio.c lib: move from strlcpy with unused retval to strscpy 2022-09-11 21:55:10 -07:00
errname.c
error-inject.c lib/error-inject: traverse list with mutex 2022-07-17 17:31:38 -07:00
errseq.c
extable.c sparc32: switch to generic extables 2021-01-03 20:05:18 -05:00
fault-inject-usercopy.c
fault-inject.c fault-injection: make stacktrace filter works as expected 2022-12-15 16:40:44 -08:00
fdt_addresses.c
fdt_empty_tree.c
fdt_ro.c
fdt_rw.c
fdt_strerror.c
fdt_sw.c
fdt_wip.c
fdt.c
find_bit_benchmark.c treewide: use get_random_u32_below() instead of deprecated function 2022-11-18 02:15:15 +01:00
find_bit.c lib/find_bit: Introduce find_next_andnot_bit() 2022-10-06 05:57:36 -07:00
flex_proportions.c flex_proportions: Disable preemption entering the write section. 2022-09-19 14:35:08 +02:00
fortify_kunit.c kunit/fortify: Validate __alloc_size attribute results 2022-11-22 21:08:28 -08:00
gen_crc32table.c
gen_crc64table.c lib: add rocksoft model crc64 2022-03-07 12:48:35 -07:00
genalloc.c all: replace find_next{,_zero}_bit with find_first{,_zero}_bit where appropriate 2022-01-15 08:47:31 -08:00
generic-radix-tree.c
glob.c lib: remove back_str initialization 2022-04-29 14:38:01 -07:00
globtest.c
hexdump.c hex2bin: fix access beyond string end 2022-04-27 10:57:33 -07:00
hweight.c
idr.c ida: don't use BUG_ON() for debugging 2022-07-10 13:55:49 -07:00
inflate.c
interval_tree_test.c
interval_tree.c interval-tree: Add a utility to iterate over spans in an interval tree 2022-11-29 16:34:15 -04:00
iomap_copy.c
iomap.c kmsan: add iomap support 2022-10-03 14:03:21 -07:00
iommu-helper.c
iov_iter.c for-6.2/block-2022-12-08 2022-12-13 10:43:59 -08:00
irq_poll.c lib/irq_poll: Prevent softirq pending leak in irq_poll_cpu_dead() 2022-04-13 21:32:21 +02:00
irq_regs.c
is_signed_type_kunit.c lib: assume char is unsigned 2022-11-19 00:56:15 +01:00
is_single_threaded.c
kasprintf.c isystem: ship and use stdarg.h 2021-08-19 09:02:55 +09:00
Kconfig iommufd for 6.2 2022-12-14 09:15:43 -08:00
Kconfig.debug Kbuild updates for v6.2 2022-12-19 12:33:32 -06:00
Kconfig.kasan MM patches for 6.2-rc1. 2022-12-13 19:29:45 -08:00
Kconfig.kcsan kcsan: Fix trivial typo in Kconfig help comments 2022-10-18 15:07:33 -07:00
Kconfig.kfence kfence: allow use of a deferrable timer 2022-03-22 15:57:11 -07:00
Kconfig.kgdb parisc: Convert PDC console to an early console 2022-10-11 12:01:24 +02:00
Kconfig.kmsan kmsan: make sure PREEMPT_RT is off 2022-11-08 15:57:24 -08:00
Kconfig.ubsan ubsan: disable UBSAN_DIV_ZERO for clang 2022-07-14 15:45:26 -07:00
kfifo.c lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
klist.c
kobject_uevent.c kobject: remove kset from struct kset_uevent_ops callbacks 2021-12-28 11:26:18 +01:00
kobject.c Driver Core changes for 6.2-rc1 2022-12-16 03:54:54 -08:00
kstrtox.c lib/kstrtox.c: add "false"/"true" support to kstrtobool() 2022-05-13 07:20:13 -07:00
kstrtox.h lib: vsprintf: Fix handling of number field widths in vsscanf 2021-05-19 15:05:11 +02:00
libcrc32c.c
linear_ranges.c lib: add linear range get selector within 2021-08-13 18:37:38 +02:00
list_debug.c lib/list_debug.c: Detect uninitialized lists 2022-06-16 19:58:20 -07:00
list_sort.c lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
list-test.c list: test: Test the hlist structure 2022-04-05 13:32:27 -06:00
llist.c llist: avoid extra memory read in llist_add_batch 2022-11-18 13:55:06 -08:00
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-rtmutex.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c lockdep/selftests: Adapt ww-tests for PREEMPT_RT 2021-12-04 10:56:24 +01:00
lockref.c lockref: stop doing cpu_relax in the cmpxchg loop 2023-01-13 14:35:38 -06:00
logic_iomem.c lib/logic_iomem: correct fallback config references 2022-03-11 10:42:56 +01:00
logic_pio.c PCI: Fix pci_register_io_range() memory leak 2021-02-17 17:31:06 -06:00
lru_cache.c lru_cache: remove unused lc_private, lc_set, lc_index_of 2022-11-22 19:38:39 -07:00
lshrdi3.c
Makefile hardening updates for v6.2-rc1 2022-12-14 12:20:00 -08:00
maple_tree.c maple_tree: remove GFP_ZERO from kmem_cache_alloc() and kmem_cache_alloc_bulk() 2023-01-18 17:12:54 -08:00
memcat_p.c
memcpy_kunit.c hardening updates for v6.2-rc1 2022-12-14 12:20:00 -08:00
memory-notifier-error-inject.c
memregion.c
memweight.c
muldi3.c
net_utils.c mac_pton: Don't access memory over expected length 2022-11-09 19:28:02 -08:00
netdev-notifier-error-inject.c
nlattr.c lib: Fix some kernel-doc comments 2022-11-08 15:06:56 -08:00
nmi_backtrace.c printk: rename cpulock functions 2022-04-22 21:30:57 +02:00
notifier-error-inject.c lib/notifier-error-inject: fix error when writing -errno to debugfs file 2022-11-30 16:13:16 -08:00
notifier-error-inject.h
objagg.c lib: objagg: Use the bitmap API when applicable 2021-12-24 14:54:29 -08:00
of-reconfig-notifier-error-inject.c
oid_registry.c lib/oid_registry.c: remove redundant assignment to variable num 2022-11-18 13:55:06 -08:00
once.c once: rename _SLOW to _SLEEPABLE 2022-10-03 17:34:32 -07:00
overflow_kunit.c overflow: Introduce overflows_type() and castable_to_type() 2022-11-02 12:39:27 -07:00
packing.c lib: packing: replace bit_reverse() with bitrev8() 2022-12-12 15:06:30 -08:00
parman.c lib: remove "expecting prototype" kernel-doc warnings 2021-04-16 16:10:37 -07:00
parser.c kernel.h: split out kstrtox() and simple_strtox() to a separate header 2021-07-01 11:06:05 -07:00
pci_iomap.c pci_iounmap'2: Electric Boogaloo: try to make sense of it all 2021-09-19 17:13:35 -07:00
percpu_counter.c percpu_counter: add percpu_counter_sum_all interface 2022-11-30 15:58:40 -08:00
percpu_test.c
percpu-refcount.c percpu-refcount: Use call_rcu_hurry() for atomic switch 2022-11-30 13:16:40 -08:00
plist.c
pm-notifier-error-inject.c
polynomial.c lib: add generic polynomial calculation 2022-05-22 11:32:30 -07:00
radix-tree.c lib/radix-tree.c: fix uninitialized variable compilation warning 2022-11-30 16:13:17 -08:00
random32.c treewide: use get_random_bytes() when possible 2022-10-11 17:42:58 -06:00
ratelimit.c ratelimit: Fix data-races in ___ratelimit(). 2022-08-24 13:46:57 +01:00
rbtree_test.c
rbtree.c
ref_tracker.c ref_tracker: remove filter_irq_stacks() call 2022-02-06 11:05:28 +00:00
refcount.c lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
rhashtable.c rhashtable: Allow rhashtable to be used from irq-safe contexts 2022-12-09 10:42:56 +00:00
sbitmap.c for-6.2/block-2022-12-08 2022-12-13 10:43:59 -08:00
scatterlist.c lib/scatterlist: Fix to merge contiguous pages into the last SG properly 2023-01-05 16:01:05 -04:00
seq_buf.c lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
sg_pool.c lib/sg_pool: change module_init(sg_pool_init) to subsys_initcall 2022-09-23 16:46:19 +02:00
sg_split.c
show_mem.c mm: reduce noise in show_mem for lowmem allocations 2022-09-26 19:46:29 -07:00
siphash_kunit.c siphash: Convert selftest to KUnit 2022-11-01 10:04:52 -07:00
siphash.c SPDX changes for 5.19-rc1 2022-06-03 10:34:34 -07:00
slub_kunit.c linux-kselftest-kunit-next-6.2-rc1 2022-12-12 16:42:57 -08:00
smp_processor_id.c lib/smp_processor_id: fix imbalanced instrumentation_end() call 2022-07-17 17:31:41 -07:00
sort.c lib/sort: Add priv pointer to swap function 2022-03-17 20:17:18 -07:00
stackdepot.c stackdepot: reserve 5 extra bits in depot_stack_handle_t 2022-10-03 14:03:18 -07:00
stackinit_kunit.c lib: stackinit: update reference to kunit-tool 2022-09-30 13:21:22 -06:00
stmp_device.c
string_helpers.c lib/string_helpers: Introduce parse_int_array_user() 2022-09-05 14:51:46 +01:00
string.c string: Rewrite and add more kern-doc for the str*() functions 2022-10-28 16:07:57 -07:00
strncpy_from_user.c lib/strn*,objtool: Enforce user_access_begin() rules 2022-04-19 21:58:47 +02:00
strnlen_user.c lib/strn*,objtool: Enforce user_access_begin() rules 2022-04-19 21:58:47 +02:00
strscpy_kunit.c fortify: Short-circuit known-safe calls to strscpy() 2022-11-01 10:04:52 -07:00
syscall.c sched: Change task_struct::state 2021-06-18 11:43:09 +02:00
test_bitmap.c lib/bitmap: add tests for for_each() loops 2022-10-01 10:22:58 -07:00
test_bitops.c lib/test: fix spelling mistakes 2021-07-08 11:48:20 -07:00
test_bits.c
test_blackhole_dev.c
test_bpf.c net: remove skb->vlan_present 2022-11-11 18:18:05 -08:00
test_debug_virtual.c
test_dynamic_debug.c dyndbg: test DECLARE_DYNDBG_CLASSMAP, sysfs nodes 2022-09-07 17:04:49 +02:00
test_firmware.c test_firmware: fix memory leak in test_firmware_init() 2022-11-23 19:49:13 +01:00
test_fprobe.c treewide: use get_random_u32_{above,below}() instead of manual loop 2022-11-18 02:15:22 +01:00
test_fpu.c selftests/fpu: Fix debugfs_simple_attr.cocci warning 2021-01-18 11:03:26 +01:00
test_free_pages.c lib/test_free_pages.c: pass a pointer to virt_to_page() 2022-07-17 17:14:36 -07:00
test_hash.c test_hash.c: refactor into kunit 2022-01-20 08:52:54 +02:00
test_hexdump.c treewide: use get_random_u32_inclusive() when possible 2022-11-18 02:18:02 +01:00
test_hmm_uapi.h hmm-tests: add test for migrate_device_range() 2022-10-12 18:51:50 -07:00
test_hmm.c hmm-tests: add test for migrate_device_range() 2022-10-12 18:51:50 -07:00
test_ida.c
test_kmod.c testing: use the copyleft-next-0.3.1 SPDX tag 2022-11-08 15:44:02 +01:00
test_kprobes.c treewide: use get_random_u32_{above,below}() instead of manual loop 2022-11-18 02:15:22 +01:00
test_linear_ranges.c lib/test_linear_ranges: Use LINEAR_RANGE() 2022-11-16 13:32:32 +00:00
test_list_sort.c treewide: use get_random_u32_below() instead of deprecated function 2022-11-18 02:15:15 +01:00
test_lockup.c lib/test_lockup: fix kernel pointer check for separate address spaces 2022-02-25 09:36:06 +01:00
test_maple_tree.c test_maple_tree: add test for mas_spanning_rebalance() on insufficient data 2022-12-21 14:31:52 -08:00
test_memcat_p.c
test_meminit.c lib/test_meminit: add checks for the allocation functions 2022-10-12 18:51:49 -07:00
test_min_heap.c treewide: use get_random_u32() when possible 2022-10-11 17:42:58 -06:00
test_module.c
test_objagg.c treewide: use get_random_bytes() when possible 2022-10-11 17:42:58 -06:00
test_parman.c
test_printf.c Random number generator updates for Linux 6.2-rc1. 2022-12-12 16:22:22 -08:00
test_ref_tracker.c lib: add tests for reference tracker 2021-12-06 16:04:44 -08:00
test_rhashtable.c Networking changes for 6.2. 2022-12-13 15:47:48 -08:00
test_scanf.c lib/test_scanf: split up number parsing test routines 2021-09-06 11:04:03 -07:00
test_sort.c lib/test: convert test_sort.c to use KUnit 2021-09-08 11:50:26 -07:00
test_static_key_base.c
test_static_keys.c
test_string.c lib/test_string.c: add strspn and strcspn tests 2022-04-29 14:38:00 -07:00
test_sysctl.c testing: use the copyleft-next-0.3.1 SPDX tag 2022-11-08 15:44:02 +01:00
test_ubsan.c ubsan: remove CONFIG_UBSAN_OBJECT_SIZE 2022-01-20 08:52:55 +02:00
test_user_copy.c
test_uuid.c
test_vmalloc.c lib/test_vmalloc.c: add parameter use_huge for fix_size_alloc_test 2023-01-18 17:12:41 -08:00
test_xarray.c XArray: Fix xas_create_range() when multi-order entry present 2022-03-28 19:25:11 -04:00
test-kstrtox.c
test-string_helpers.c treewide: use get_random_u32_below() instead of deprecated function 2022-11-18 02:15:15 +01:00
textsearch.c
timerqueue.c rbtree, timerqueue: Use rb_add_cached() 2021-02-17 14:08:01 +01:00
trace_readwrite.c asm-generic/io: Add _RET_IP_ to MMIO trace for more accurate debug info 2022-11-21 22:02:10 +01:00
ts_bm.c lib/ts_bm.c: remove redundant store to variable consumed after addition 2022-07-17 17:31:39 -07:00
ts_fsm.c
ts_kmp.c
ubsan.c panic: Consolidate open-coded panic_on_warn checks 2022-12-02 13:04:44 -08:00
ubsan.h ubsan: implement __ubsan_handle_alignment_assumption 2021-02-05 11:03:47 -08:00
ucmpdi2.c
ucs2_string.c
usercopy.c instrumented.h: allow instrumenting both sides of copy_from_user() 2022-10-03 14:03:18 -07:00
uuid.c treewide: use get_random_bytes() when possible 2022-10-11 17:42:58 -06:00
vsprintf.c Random number generator updates for Linux 6.2-rc1. 2022-12-12 16:22:22 -08:00
win_minmax.c lib/win_minmax: use /* notation for regular comments 2023-01-11 16:14:21 -08:00
xarray.c mm/huge_memory: Fix xarray node memory leak 2022-06-09 16:24:25 -04:00
xxhash.c