linux/kernel/rcu
Frederic Weisbecker e787644caf rcu: Defer RCU kthreads wakeup when CPU is dying
When the CPU goes idle for the last time during the CPU down hotplug
process, RCU reports a final quiescent state for the current CPU. If
this quiescent state propagates up to the top, some tasks may then be
woken up to complete the grace period: the main grace period kthread
and/or the expedited main workqueue (or kworker).

If those kthreads have a SCHED_FIFO policy, the wake up can indirectly
arm the RT bandwith timer to the local offline CPU. Since this happens
after hrtimers have been migrated at CPUHP_AP_HRTIMERS_DYING stage, the
timer gets ignored. Therefore if the RCU kthreads are waiting for RT
bandwidth to be available, they may never be actually scheduled.

This triggers TREE03 rcutorture hangs:

	 rcu: INFO: rcu_preempt self-detected stall on CPU
	 rcu:     4-...!: (1 GPs behind) idle=9874/1/0x4000000000000000 softirq=0/0 fqs=20 rcuc=21071 jiffies(starved)
	 rcu:     (t=21035 jiffies g=938281 q=40787 ncpus=6)
	 rcu: rcu_preempt kthread starved for 20964 jiffies! g938281 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
	 rcu:     Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
	 rcu: RCU grace-period kthread stack dump:
	 task:rcu_preempt     state:R  running task     stack:14896 pid:14    tgid:14    ppid:2      flags:0x00004000
	 Call Trace:
	  <TASK>
	  __schedule+0x2eb/0xa80
	  schedule+0x1f/0x90
	  schedule_timeout+0x163/0x270
	  ? __pfx_process_timeout+0x10/0x10
	  rcu_gp_fqs_loop+0x37c/0x5b0
	  ? __pfx_rcu_gp_kthread+0x10/0x10
	  rcu_gp_kthread+0x17c/0x200
	  kthread+0xde/0x110
	  ? __pfx_kthread+0x10/0x10
	  ret_from_fork+0x2b/0x40
	  ? __pfx_kthread+0x10/0x10
	  ret_from_fork_asm+0x1b/0x30
	  </TASK>

The situation can't be solved with just unpinning the timer. The hrtimer
infrastructure and the nohz heuristics involved in finding the best
remote target for an unpinned timer would then also need to handle
enqueues from an offline CPU in the most horrendous way.

So fix this on the RCU side instead and defer the wake up to an online
CPU if it's too late for the local one.

Reported-by: Paul E. McKenney <paulmck@kernel.org>
Fixes: 5c0930ccaa ("hrtimers: Push pending hrtimers away from outgoing CPU earlier")
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Neeraj Upadhyay (AMD) <neeraj.iitr10@gmail.com>
2024-01-24 22:46:17 +05:30
..
Kconfig rcu: Employ jiffies-based backstop to callback time limit 2023-05-11 13:42:39 -07:00
Kconfig.debug rcu: Restrict access to RCU CPU stall notifiers 2023-12-12 02:31:22 +05:30
Makefile
rcu_segcblist.c rcu: Use rcu_segcblist_segempty() instead of open coding it 2023-10-04 17:33:18 +02:00
rcu_segcblist.h rcu: Throttle callback invocation based on number of ready callbacks 2023-01-03 17:28:34 -08:00
rcu.h rcu: Restrict access to RCU CPU stall notifiers 2023-12-12 02:31:22 +05:30
rcuscale.c rcuscale: Move rcu_scale_writer() schedule_timeout_uninterruptible() to _idle() 2023-07-14 15:01:49 -07:00
rcutorture.c Merge branches 'doc.2023.12.13a', 'torture.2023.11.23a', 'fixes.2023.12.13a', 'rcu-tasks.2023.12.12b' and 'srcu.2023.12.13a' into rcu-merge.2023.12.13a 2023-12-14 01:21:31 +05:30
refscale.c refscale: Print out additional module parameters 2023-09-11 23:02:18 +02:00
srcutiny.c rcu: Dump memory object info if callback function is invalid 2023-09-13 22:29:12 +02:00
srcutree.c srcu: Explain why callbacks invocations can't run concurrently 2023-12-12 02:41:17 +05:30
sync.c rcu/sync: Use call_rcu_hurry() instead of call_rcu 2022-11-29 14:04:33 -08:00
tasks.h rcu-tasks: Mark RCU Tasks accesses to current->rcu_tasks_idle_cpu 2023-12-12 02:22:47 +05:30
tiny.c rcu: Dump memory object info if callback function is invalid 2023-09-13 22:29:12 +02:00
tree_exp.h rcu: Defer RCU kthreads wakeup when CPU is dying 2024-01-24 22:46:17 +05:30
tree_nocb.h rcu: dynamically allocate the rcu-lazy shrinker 2023-10-04 10:32:24 -07:00
tree_plugin.h rcu: Mark additional concurrent load from ->cpu_no_qs.b.exp 2023-05-11 13:42:39 -07:00
tree_stall.h rcu: Restrict access to RCU CPU stall notifiers 2023-12-12 02:31:22 +05:30
tree.c rcu: Defer RCU kthreads wakeup when CPU is dying 2024-01-24 22:46:17 +05:30
tree.h rcu/tree: Defer setting of jiffies during stall reset 2023-09-11 22:36:40 +02:00
update.c rcu: Restrict access to RCU CPU stall notifiers 2023-12-12 02:31:22 +05:30