iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
bc08b449ee
linux/lib
History
Linus Torvalds bc08b449ee lockref: implement lockless reference count updates using cmpxchg() 
Instead of taking the spinlock, the lockless versions atomically check
that the lock is not taken, and do the reference count update using a
cmpxchg() loop.  This is semantically identical to doing the reference
count update protected by the lock, but avoids the "wait for lock"
contention that you get when accesses to the reference count are
contended.

Note that a "lockref" is absolutely _not_ equivalent to an atomic_t.
Even when the lockref reference counts are updated atomically with
cmpxchg, the fact that they also verify the state of the spinlock means
that the lockless updates can never happen while somebody else holds the
spinlock.

So while "lockref_put_or_lock()" looks a lot like just another name for
"atomic_dec_and_lock()", and both optimize to lockless updates, they are
fundamentally different: the decrement done by atomic_dec_and_lock() is
truly independent of any lock (as long as it doesn't decrement to zero),
so a locked region can still see the count change.

The lockref structure, in contrast, really is a *locked* reference
count.  If you hold the spinlock, the reference count will be stable and
you can modify the reference count without using atomics, because even
the lockless updates will see and respect the state of the lock.

In order to enable the cmpxchg lockless code, the architecture needs to
do three things:

 (1) Make sure that the "arch_spinlock_t" and an "unsigned int" can fit
     in an aligned u64, and have a "cmpxchg()" implementation that works
     on such a u64 data type.

 (2) define a helper function to test for a spinlock being unlocked
     ("arch_spin_value_unlocked()")

 (3) select the "ARCH_USE_CMPXCHG_LOCKREF" config variable in its
     Kconfig file.

This enables it for x86-64 (but not 32-bit, we'd need to make sure
cmpxchg() turns into the proper cmpxchg8b in order to enable it for
32-bit mode).

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-09-02 12:12:15 -07:00
..
fonts lib: Move fonts from drivers/video/console/ to lib/fonts/ 2013-06-28 10:28:22 +02:00
lz4 lib/lz4: correct the LZ4 license 2013-08-23 09:51:22 -07:00
lzo lib/lzo: Update LZO compression to current upstream version 2013-02-20 19:36:01 +01:00
mpi alpha: Modernize lib/mpi/longlong.h 2013-07-19 13:54:23 -07:00
raid6 lib/raid6: build proper files on corresponding arch 2012-12-13 19:51:04 +11:00
reed_solomon
xz decompressors: fix typo "POWERPC" 2013-03-13 15:21:48 -07:00
zlib_deflate
zlib_inflate
.gitignore X.509: Implement simple static OID registry 2012-10-08 13:50:18 +10:30
argv_split.c argv_split(): teach it to handle mutable strings 2013-04-29 18:28:19 -07:00
asn1_decoder.c Nothing all that exciting; a new module-from-fd syscall for those who want 2012-12-19 07:55:08 -08:00
atomic64_test.c atomic64_test: simplify the #ifdef for atomic64_dec_if_positive() test 2012-07-30 17:25:16 -07:00
atomic64.c lib: atomic64: Initialize locks statically to fix early users 2012-12-20 13:50:16 -08:00
audit.c
average.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
bcd.c usb/core: use bin2bcd() for bcdDevice in RH 2012-09-10 11:13:16 -07:00
bch.c
bitmap.c propagate name change to comments in kernel source 2012-12-06 10:39:54 +01:00
bitrev.c
bsearch.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
btree.c btree: catch NULL value before it does harm 2012-06-07 14:43:55 -07:00
bug.c taint: add explicit flag to show whether lock dep is still OK. 2013-01-21 17:17:57 +10:30
build_OID_registry X.509: do not emit any informational output 2013-06-19 17:54:06 +02:00
bust_spinlocks.c printk: Provide a wake_up_klogd() off-case 2013-03-22 16:41:20 -07:00
check_signature.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
checksum.c asm-generic headers: Allow yet more arch overrides in checksum.h 2013-02-11 20:00:33 +05:30
clz_ctz.c lib: add weak clz/ctz functions 2013-07-09 10:33:30 -07:00
clz_tab.c lib: Fix multiple definitions of clz_tab 2012-02-02 10:34:23 +11:00
cmdline.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
cordic.c
cpu_rmap.c lib: cpu_rmap: avoid flushing all workqueues 2013-01-11 14:54:54 -08:00
cpu-notifier-error-inject.c cpu: rewrite cpu-notifier-error-inject module 2012-07-30 17:25:22 -07:00
cpumask.c bootmem: fix wrong call parameter for free_bootmem() 2012-12-11 17:22:28 -08:00
crc7.c
crc8.c
crc16.c
crc32.c sections: fix const sections for crc32 table 2012-10-06 03:04:46 +09:00
crc32defs.h crc32: select an algorithm via Kconfig 2012-03-23 16:58:38 -07:00
crc-ccitt.c
crc-itu-t.c
crc-t10dif.c Revert "crypto: crct10dif - Wrap crc_t10dif function all to use crypto transform framework" 2013-07-24 17:04:16 +10:00
ctype.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
debug_locks.c mutex: Add support for wound/wait style locks 2013-06-26 12:10:56 +02:00
debugobjects.c hlist: drop the node parameter from iterators 2013-02-27 19:10:24 -08:00
dec_and_lock.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
decompress_bunzip2.c decompress_bunzip2: remove invalid vi modeline 2011-12-06 10:00:05 +01:00
decompress_inflate.c
decompress_unlz4.c lib: add support for LZ4-compressed kernel 2013-07-09 10:33:30 -07:00
decompress_unlzma.c treewide: Fix comment and string typo 'bufer' 2011-12-06 09:53:40 +01:00
decompress_unlzo.c lib/lzo: Rename lzo1x_decompress.c to lzo1x_decompress_safe.c 2013-02-20 19:36:00 +01:00
decompress_unxz.c
decompress.c lib: add support for LZ4-compressed kernel 2013-07-09 10:33:30 -07:00
devres.c lib/devres.c: fix misplaced #endif 2013-02-27 19:10:09 -08:00
digsig.c Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security 2013-02-21 08:18:12 -08:00
div64.c Revert "math64: New div64_u64_rem helper" 2013-04-30 19:13:05 +02:00
dma-debug.c dma-debug: update DMA debug API to better handle multiple mappings of a buffer 2013-03-22 16:41:20 -07:00
dump_stack.c dump_stack: serialize the output from dump_stack() 2013-07-03 16:07:41 -07:00
dynamic_debug.c dynamic_debug: reuse generic string_unescape function 2013-04-30 17:04:03 -07:00
dynamic_queue_limits.c bql: Avoid possible inconsistent calculation. 2012-05-31 18:18:17 -04:00
earlycpio.c kernel: delete __cpuinit usage from all core kernel files 2013-07-14 19:36:59 -04:00
extable.c
fault-inject.c debugfs: add get/set for atomic types 2013-06-03 13:55:01 -07:00
fdt_ro.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_rw.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_strerror.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_sw.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_wip.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
find_last_bit.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
find_next_bit.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
flex_array.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
flex_proportions.c lib/flex_proportions.c: fix corruption of denominator in flexible proportions 2012-09-25 08:59:21 -07:00
gcd.c lib/gcd.c: prevent possible div by 0 2012-10-06 03:04:57 +09:00
gen_crc32table.c sections: fix const sections for crc32 table 2012-10-06 03:04:46 +09:00
genalloc.c genalloc: add devres support, allow to find a managed pool by device 2013-04-29 18:28:13 -07:00
halfmd4.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
hexdump.c dynamic_debug: dynamic hex dump 2013-01-17 12:19:09 -08:00
hweight.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
idr.c idr: print a stack dump after ida_remove warning 2013-07-03 16:08:04 -07:00
inflate.c
int_sqrt.c lib/int_sqrt.c: optimize square root algorithm 2013-04-29 18:28:19 -07:00
interval_tree_test_main.c random32: rename random32 to prandom 2012-12-17 17:15:26 -08:00
interval_tree.c mm: interval tree updates 2012-10-09 16:22:40 +09:00
iomap_copy.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
iomap.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
iommu-helper.c The following text was taken from the original review request: 2012-03-24 10:24:31 -07:00
ioremap.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
iovec.c Hoist memcpy_fromiovec/memcpy_toiovec into lib/ 2013-05-20 10:24:22 +09:30
irq_regs.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
is_single_threaded.c
jedec_ddr_data.c ddr: add LPDDR2 data from JESD209-2 2012-05-02 00:04:06 -07:00
kasprintf.c lib/kasprintf.c: use kmalloc_track_caller() to get accurate traces for kvasprintf 2012-10-11 08:50:15 +09:00
Kconfig lockref: implement lockless reference count updates using cmpxchg() 2013-09-02 12:12:15 -07:00
Kconfig.debug kernel: delete __cpuinit usage from all core kernel files 2013-07-14 19:36:59 -04:00
Kconfig.kgdb treewide: Fix typo in printk 2013-06-18 13:48:45 +02:00
Kconfig.kmemcheck
kfifo.c kfifo: fix kfifo_alloc() and kfifo_init() 2013-02-27 19:10:23 -08:00
klist.c klist: del waiter from klist_remove_waiters before wakeup waitting process 2013-05-21 10:16:39 -07:00
kobject_uevent.c netlink: hide struct module parameter in netlink_kernel_create 2012-09-08 18:46:30 -04:00
kobject.c kobject: sanitize argument for format string 2013-06-07 16:05:50 -07:00
kstrtox.c kstrto*: add documentation 2012-12-17 17:15:22 -08:00
kstrtox.h
lcm.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
libcrc32c.c
list_debug.c rcu: Fix broken strings in RCU's source code. 2012-07-06 06:01:49 -07:00
list_sort.c lib/: rename random32() to prandom_u32() 2013-04-29 18:28:42 -07:00
llist.c llist: fix/simplify llist_add() and llist_add_batch() 2013-07-13 13:29:24 +04:00
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c mutex: Move ww_mutex definitions to ww_mutex.h 2013-07-12 12:07:46 +02:00
lockref.c lockref: implement lockless reference count updates using cmpxchg() 2013-09-02 12:12:15 -07:00
lru_cache.c lru_cache: introduce lc_get_cumulative() 2013-03-22 22:17:36 -06:00
Makefile lockref: uninline lockref helper functions 2013-09-02 11:58:20 -07:00
md5.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
memory-notifier-error-inject.c memory: memory notifier error injection module 2012-07-30 17:25:22 -07:00
memweight.c string: introduce memweight() 2012-07-30 17:25:16 -07:00
net_utils.c net: core: move mac_pton() to lib/net_utils.c 2013-06-05 12:00:27 -07:00
nlattr.c netlink: add minlen validation for the new signed types 2012-08-30 13:11:46 -04:00
notifier-error-inject.c mode_t, whack-a-mole at 11... 2013-04-09 14:13:05 -04:00
notifier-error-inject.h fault-injection: notifier error injection 2012-07-30 17:25:22 -07:00
of-reconfig-notifier-error-inject.c powerpc+of: Rename and fix OF reconfig notifier error inject module 2012-12-14 10:32:52 +11:00
oid_registry.c Give the OID registry file module info to avoid kernel tainting 2013-05-05 14:38:00 -07:00
parser.c lib/parser.c: fix up comments for valid return values from match_number 2013-02-21 17:22:25 -08:00
pci_iomap.c lib: add NO_GENERIC_PCI_IOPORT_MAP 2012-01-31 23:19:47 +02:00
percpu_counter.c kernel: delete __cpuinit usage from all core kernel files 2013-07-14 19:36:59 -04:00
percpu-refcount.c percpu-refcount: use RCU-sched insted of normal RCU 2013-06-16 16:12:26 -07:00
percpu-rwsem.c percpu_rw_semaphore: add lockdep annotations 2012-12-17 17:15:18 -08:00
plist.c lib/plist.c: make plist test announcements KERN_DEBUG 2012-10-06 03:04:58 +09:00
pm-notifier-error-inject.c PM: PM notifier error injection module 2012-07-30 17:25:22 -07:00
prio_heap.c
proportions.c
radix-tree.c radix-tree: fix contiguous iterator 2012-06-05 10:46:40 -07:00
random32.c prandom: introduce prandom_bytes() and prandom_bytes_state() 2012-12-17 17:15:26 -08:00
ratelimit.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
rational.c lib: Change mail address of Oskar Schirmer 2012-05-17 15:18:37 +02:00
rbtree_test.c rbtree_test: add __init/__exit annotations 2013-04-30 17:04:07 -07:00
rbtree.c lib/rbtree.c: avoid the use of non-static __always_inline 2013-01-11 14:54:56 -08:00
reciprocal_div.c sch_red: Adaptative RED AQM 2011-12-08 19:52:43 -05:00
rwsem-spinlock.c rwsem: simplify __rwsem_do_wake 2013-05-07 07:20:16 -07:00
rwsem.c rwsem: check counter to avoid cmpxchg calls 2013-05-07 16:11:51 -07:00
scatterlist.c lib/scatterlist: error handling in __sg_alloc_table() 2013-07-09 10:33:31 -07:00
sha1.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
show_mem.c mm, show_mem: suppress page counts in non-blockable contexts 2013-04-29 15:54:28 -07:00
smp_processor_id.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
sort.c
spinlock_debug.c lib/spinlock_debug: avoid livelock in do_raw_spin_lock() 2012-10-06 03:04:57 +09:00
stmp_device.c lib: add support for stmp-style devices 2012-04-20 23:27:08 +02:00
string_helpers.c lib/string_helpers: introduce generic string_unescape 2013-04-30 17:04:03 -07:00
string.c The following text was taken from the original review request: 2012-03-24 10:24:31 -07:00
strncpy_from_user.c word-at-a-time: make the interfaces truly generic 2012-05-26 11:33:40 -07:00
strnlen_user.c lib: Fix generic strnlen_user for 32-bit big-endian machines 2012-05-27 20:59:46 -07:00
swiotlb.c x86, kdump: Set crashkernel_low automatically 2013-04-17 12:35:32 -07:00
syscall.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
test-kstrtox.c lib/test-kstrtox.c: mark const init data with __initconst instead of __initdata 2012-05-29 16:22:32 -07:00
test-string_helpers.c lib/string_helpers: introduce generic string_unescape 2013-04-30 17:04:03 -07:00
textsearch.c
timerqueue.c The following text was taken from the original review request: 2012-03-24 10:24:31 -07:00
ts_bm.c
ts_fsm.c
ts_kmp.c
ucs2_string.c Move utf16 functions to kernel core and rename 2013-04-15 21:23:03 +01:00
usercopy.c Kconfig: consolidate CONFIG_DEBUG_STRICT_USER_COPY_CHECKS 2013-04-30 17:04:09 -07:00
uuid.c uuid: use prandom_bytes() 2013-04-29 18:28:42 -07:00
vsprintf.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2013-07-09 18:24:39 -07:00