linux/arch
Steven Rostedt 40d6a14662 Kick CPUS that might be sleeping in cpus_idle_wait
Sometimes cpu_idle_wait gets stuck because it might miss CPUS that are
already in idle, have no tasks waiting to run and have no interrupts going
to them.  This is common on bootup when switching cpu idle governors.

This patch gives those CPUS that don't check in an IPI kick.

 Background:
 -----------
I notice this while developing the mcount patches, that every once in a
while the system would hang. Looking deeper, the hang was always at boot
up when registering init_menu of the cpu_idle menu governor. Talking
with Thomas Gliexner, we discovered that one of the CPUS had no timer
events scheduled for it and it was in idle (running with NO_HZ). So the
CPU would not set the cpu_idle_state bit.

Hitting sysrq-t a few times would eventually route the interrupt to the
stuck CPU and the system would continue.

Note, I would have used the PDA isidle but that is set after the
cpu_idle_state bit is cleared, and would leave a window open where we
may miss being kicked.

hmm, looking closer at this, we still have a small race window between
clearing the cpu_idle_state and disabling interrupts (hence the RFC).

    CPU0:                          CPU 1:
  ---------                       ---------
 cpu_idle_wait():                 cpu_idle():
      |                           __cpu_cpu_var(is_idle) = 1;
      |                           if (__get_cpu_var(cpu_idle_state)) /* == 0 */
 per_cpu(cpu_idle_state, 1) = 1;         |
 if (per_cpu(is_idle, 1)) /* == 1 */     |
 smp_call_function(1)                    |
      |                             receives ipi and runs do_nothing.
 wait on map == empty               idle();
   /* waits forever */

So really we need interrupts off for most of this then. One might think
that we could simply clear the cpu_idle_state from do_nothing, but I'm
assuming that cpu_idle governors can be removed, and this might cause a
race that a governor might be used after the module was removed.

Venki said:

  I think your RFC patch is the right solution here.  As I see it, there is
  no race with your RFC patch.  As long as you call a dummy smp_call_function
  on all CPUs, we should be OK.  We can get rid of cpu_idle_state and the
  current wait forever logic altogether with dummy smp_call_function.  And so
  there wont be any wait forever scenario.

  The whole point of cpu_idle_wait() is to make all CPUs come out of idle
  loop atleast once.  The caller will use cpu_idle_wait something like this.

  // Want to change idle handler

  - Switch global idle handler to always present default_idle

  - call cpu_idle_wait so that all cpus come out of idle for an instant
    and stop using old idle pointer and start using default idle

  - Change the idle handler to a new handler

  - optional cpu_idle_wait if you want all cpus to start using the new
    handler immediately.

Maybe the below 1s patch is safe bet for .24.  But for .25, I would say we
just replace all complicated logic by simple dummy smp_call_function and
remove cpu_idle_state altogether.

Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Cc: Venkatesh Pallipadi <venkatesh.pallipadi@intel.com>
Acked-by: Ingo Molnar <mingo@elte.hu>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andi Kleen <ak@suse.de>
Cc: Len Brown <lenb@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-01-14 08:52:22 -08:00
..
alpha alpha: build fixes 2007-12-17 19:28:16 -08:00
arm [ARM] vfp: fix fuitod/fsitod instructions 2008-01-10 17:07:08 +00:00
avr32 [AVR32] Fix wrong pt_regs in critical exception handler 2007-12-07 14:54:48 +01:00
blackfin Blackfin arch: fix bug when enable uart1 with uart0 disabled => no initial console 2007-11-21 17:04:41 +08:00
cris ide: move CONFIG_IDE_ETRAX to drivers/ide/Kconfig 2007-11-27 21:35:55 +01:00
frv FRV: arrange things such that BRA can reach from the trap table 2007-11-29 09:24:54 -08:00
h8300 Kbuild/doc: fix links to Documentation files 2007-10-30 14:26:30 -07:00
ia64 [IA64] Update Altix nofault code 2008-01-03 13:22:54 -08:00
m32r m32r: Update sys_rt_sigsuspend 2007-11-28 01:24:04 +09:00
m68k m68k: export atari_keyb_init 2007-11-26 19:15:31 -08:00
m68knommu m68knommu: mark mem init functions as __init 2007-10-23 20:45:44 -07:00
mips [MIPS] Replace 40c7869b69 kludge 2008-01-11 17:05:42 +00:00
parisc [PARISC] print more than one character at a time for pdc console 2007-12-06 09:32:15 -08:00
powerpc [POWERPC] Fix CPU hotplug when using the SLB shadow buffer 2008-01-11 16:33:55 +11:00
ppc [POWERPC] virtex bug fix: Use canonical value for AC97 interrupt xparams 2007-12-07 11:43:55 +11:00
s390 [S390] Make sure the restore psw masks are initialized. 2007-12-04 16:09:58 +01:00
sh sh: Support PCI IO access of SH7780 base boards. 2007-11-30 12:36:13 +09:00
sh64 sh64: Kill off duplicate includes. 2007-11-05 12:18:17 +09:00
sparc [SPARC32]: Spelling fixes 2007-12-20 13:55:45 -08:00
sparc64 [SPARC64]: Implement pci_resource_to_user() 2007-12-26 19:33:46 -08:00
um uml: user of helper_wait() got missed when it got extra arguments 2007-12-23 12:54:37 -08:00
v850 spelling fixes: arch/v850/ 2007-10-20 01:24:05 +02:00
x86 Kick CPUS that might be sleeping in cpus_idle_wait 2008-01-14 08:52:22 -08:00
xtensa [XTENSA]: Fix use of skb after netif_rx 2007-12-11 02:45:28 -08:00