iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

rcu: Make cond_resched_rcu_qs() supply RCU-sched expedited QS

Browse Source 
Although cond_resched_rcu_qs() supplies quiescent states to all flavors
of normal RCU grace periods, it does nothing for expedited RCU-sched
grace periods.  This commit therefore adds a check for a need for a
quiescent state from the current CPU by an expedited RCU-sched grace
period, and invokes rcu_sched_qs() to supply that quiescent state if so.

Note that the check is racy in that we might be migrated to some other
CPU just after checking the per-CPU variable.  This is OK because the
act of migration will do a context switch, which will supply the needed
quiescent state.  The only downside is that we might do an unnecessary
call to rcu_sched_qs(), but the probability is low and the overhead
is small.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit is contained in:
Paul E. McKenney 2016-01-13 13:57:54 -08:00
parent 251c617c75
commit a1e1224849
1 changed files with 15 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
kernel/rcu/tree.c
View File

@ -370,6 +370,21 @@ void rcu_all_qs(void)
rcu_momentary_dyntick_idle();
local_irq_restore(flags);
}
if (unlikely(raw_cpu_read(rcu_sched_data.cpu_no_qs.b.exp))) {
/*
* Yes, we just checked a per-CPU variable with preemption
* enabled, so we might be migrated to some other CPU at
* this point. That is OK because in that case, the
* migration will supply the needed quiescent state.
* We might end up needlessly disabling preemption and
* invoking rcu_sched_qs() on the destination CPU, but
* the probability and cost are both quite low, so this
* should not be a problem in practice.
*/
preempt_disable();
rcu_sched_qs();
preempt_enable();
}
this_cpu_inc(rcu_qs_ctr);
barrier(); /* Avoid RCU read-side critical sections leaking up. */
}