Nick Piggin e286781d5f mm: speculative page references
If we can be sure that elevating the page_count on a pagecache page will
pin it, we can speculatively run this operation, and subsequently check to
see if we hit the right page rather than relying on holding a lock or
otherwise pinning a reference to the page.

This can be done if get_page/put_page behaves consistently throughout the
whole tree (ie.  if we "get" the page after it has been used for something
else, we must be able to free it with a put_page).

Actually, there is a period where the count behaves differently: when the
page is free or if it is a constituent page of a compound page.  We need
an atomic_inc_not_zero operation to ensure we don't try to grab the page
in either case.

This patch introduces the core locking protocol to the pagecache (ie.
adds page_cache_get_speculative, and tweaks some update-side code to make
it work).

Thanks to Hugh for pointing out an improvement to the algorithm setting
page_count to zero when we have control of all references, in order to
hold off speculative getters.

[kamezawa.hiroyu@jp.fujitsu.com: fix migration_entry_wait()]
[hugh@veritas.com: fix add_to_page_cache]
[akpm@linux-foundation.org: repair a comment]
Signed-off-by: Nick Piggin <npiggin@suse.de>
Cc: Jeff Garzik <jeff@garzik.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Hugh Dickins <hugh@veritas.com>
Cc: "Paul E. McKenney" <paulmck@us.ibm.com>
Reviewed-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Acked-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-07-26 12:00:06 -07:00
..
2008-07-22 19:39:14 -04:00
2008-05-22 14:12:39 -04:00
2008-07-22 16:09:13 -04:00
2008-03-26 00:18:44 -04:00
2008-07-21 21:55:03 -07:00
2008-03-17 08:11:03 -04:00
2006-09-13 13:24:59 -04:00
2008-01-28 15:04:05 -08:00
2006-09-13 13:24:59 -04:00
2006-09-13 13:24:59 -04:00
2008-06-10 18:20:17 -04:00
2006-09-13 13:24:59 -04:00
2008-03-26 00:18:43 -04:00
2007-10-10 16:51:43 -07:00
2008-04-29 01:56:14 -04:00
2006-09-13 13:24:59 -04:00
2008-07-18 17:54:17 -07:00
2008-05-16 22:20:27 -07:00
2008-07-18 17:58:57 -07:00
2008-07-18 17:55:11 -07:00
2008-06-23 20:33:36 -07:00
2008-06-23 20:27:26 -07:00
2008-06-23 20:33:01 -07:00
2008-06-23 20:34:36 -07:00
2008-07-26 12:00:06 -07:00
2008-01-04 00:47:04 -08:00
2006-09-13 13:24:59 -04:00
2008-03-26 00:18:43 -04:00
2007-02-09 16:23:17 +00:00
2006-09-13 13:24:59 -04:00
2008-02-01 20:54:03 +11:00
2008-07-22 17:54:07 -04:00
2007-10-10 16:51:13 -07:00
2007-10-10 16:51:13 -07:00
2008-01-28 15:07:59 -08:00
2008-04-29 01:56:01 -04:00
2008-06-17 23:07:29 -04:00
2007-10-19 11:53:42 -07:00
2006-09-13 13:24:59 -04:00
2008-07-22 17:12:37 +10:00
2008-07-22 17:12:37 +10:00
2008-05-22 14:12:40 -04:00
2008-01-28 15:07:12 -08:00
2006-09-13 13:24:59 -04:00
2008-07-22 17:50:44 -04:00
2008-07-22 17:50:44 -04:00
2008-02-03 04:26:12 -08:00
2008-07-25 15:44:44 +10:00
2008-07-17 19:21:30 -07:00
2008-03-17 08:11:41 -04:00
2008-04-25 02:08:51 -04:00
2008-06-11 22:06:31 -04:00
2008-01-28 15:04:05 -08:00
2008-01-28 15:07:12 -08:00
2008-07-22 19:34:21 -04:00
2007-10-19 11:53:42 -07:00
2007-04-28 11:01:05 -04:00
2008-02-03 04:28:10 -08:00
2008-07-22 19:56:37 -04:00
2008-07-22 17:50:44 -04:00
2008-03-17 07:56:33 -04:00
2008-07-22 17:50:44 -04:00
2008-03-26 00:18:04 -04:00
2008-03-26 00:18:00 -04:00
2006-09-13 13:24:59 -04:00
2007-10-23 21:28:40 -04:00
2008-07-17 19:21:17 -07:00
2008-05-30 22:19:22 -04:00
2008-03-05 16:34:41 -06:00
2008-07-20 17:12:34 -07:00
2008-01-28 15:11:27 -08:00
2008-07-20 17:12:34 -07:00
2008-06-20 21:58:02 -07:00
2008-01-28 15:08:00 -08:00
2008-07-22 20:00:01 -04:00
2008-01-28 15:03:59 -08:00
2008-07-11 01:11:40 -04:00
2008-04-16 20:41:44 -04:00
2008-05-30 22:07:17 -04:00
2008-06-28 10:23:33 -04:00
2006-09-13 13:24:59 -04:00
2007-10-16 21:10:29 -04:00
2008-07-22 21:53:40 -07:00
2008-06-17 23:59:39 -04:00
2008-01-28 15:11:27 -08:00
2008-07-20 17:12:34 -07:00
2006-09-13 13:24:59 -04:00
2008-01-28 15:03:59 -08:00
2007-05-04 17:59:08 -07:00
2007-07-16 15:02:47 +02:00
2008-04-16 20:06:50 -04:00
2007-10-10 16:51:13 -07:00
2008-02-03 04:26:15 -08:00
2008-01-28 15:03:46 -08:00
2008-03-23 03:35:12 -07:00
2008-01-28 15:07:10 -08:00
2008-01-28 15:07:09 -08:00
2008-02-03 04:26:14 -08:00
2007-10-10 16:51:34 -07:00
2008-05-29 01:38:46 -07:00
2008-05-30 22:18:02 -04:00
2008-05-30 22:18:02 -04:00
2008-07-22 17:54:17 -04:00