Joel Fernandes (Google) 6bc3358280 rcu/tree: Make rcu_do_batch count how many callbacks were executed
The rcu_do_batch() function extracts the ready-to-invoke callbacks
from the rcu_segcblist located in the ->cblist field of the current
CPU's rcu_data structure.  These callbacks are first moved to a local
(unsegmented) rcu_cblist.  The rcu_do_batch() function then uses this
rcu_cblist's ->len field to count how many CBs it has invoked, but it
does so by counting that field down from zero.  Finally, this function
negates the value in this ->len field (resulting in a positive number)
and subtracts the result from the ->len field of the current CPU's
->cblist field.

Except that it is sometimes necessary for rcu_do_batch() to stop invoking
callbacks mid-stream, despite there being more ready to invoke, for
example, if a high-priority task wakes up.  In this case the remaining
not-yet-invoked callbacks are requeued back onto the CPU's ->cblist,
but remain in the ready-to-invoke segment of that list.  As above, the
negative of the local rcu_cblist's ->len field is still subtracted from
the ->len field of the current CPU's ->cblist field.

The design of counting down from 0 is confusing and error-prone, plus
use of a positive count will make it easier to provide a uniform and
consistent API to deal with the per-segment counts that are added
later in this series.  For example, rcu_segcblist_extract_done_cbs()
can unconditionally populate the resulting unsegmented list's ->len
field during extraction.

This commit therefore explicitly counts how many callbacks were executed
in rcu_do_batch() itself, counting up from zero, and then uses that
to update the per-CPU segcb list's ->len field, without relying on the
downcounting of rcl->len from zero.

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Reviewed-by: Neeraj Upadhyay <neeraju@codeaurora.org>
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
2021-01-04 13:22:12 -08:00
..
2020-12-09 17:47:36 -08:00
2020-11-24 16:47:49 +01:00
2020-12-22 13:19:43 -08:00
2020-12-15 22:46:18 -08:00
2020-12-16 10:45:11 -08:00
2020-12-17 13:22:17 -08:00
2020-12-02 22:52:16 -05:00
2020-12-16 10:54:03 -08:00
2020-12-14 18:29:11 -08:00
2020-12-09 17:08:45 +01:00
2020-12-15 22:46:19 -08:00
2020-12-17 13:01:31 -08:00
2020-12-14 16:40:27 -08:00
2020-11-14 11:26:04 -08:00
2020-12-17 13:01:31 -08:00
2020-12-14 16:40:27 -08:00
2020-07-27 14:31:12 -04:00
2020-12-01 10:30:28 +00:00
2020-12-16 11:30:10 -08:00
2020-12-16 12:33:35 -08:00
2020-12-27 09:06:10 -08:00
2020-12-19 11:18:38 -08:00
2020-12-14 16:40:27 -08:00
2020-08-19 14:14:12 +02:00
2020-08-19 14:13:20 +02:00