Alex Shi 313c8c16ee PM / CPU: replace raw_notifier with atomic_notifier ...

This patch replaces an rwlock and raw notifier by an atomic notifier
protected by a spin_lock and RCU.

The main reason for this change is due to a 'scheduling while atomic'
bug with RT kernels on ARM/ARM64. On ARM/ARM64, the rwlock
cpu_pm_notifier_lock in cpu_pm_enter/exit() causes a potential
schedule after IRQ disable in the idle call chain:

cpu_startup_entry
  cpu_idle_loop
    local_irq_disable()
    cpuidle_idle_call
      call_cpuidle
        cpuidle_enter
          cpuidle_enter_state
            ->enter :arm_enter_idle_state
              cpu_pm_enter/exit
                CPU_PM_CPU_IDLE_ENTER
                  read_lock(&cpu_pm_notifier_lock); <-- sleep in idle
                     __rt_spin_lock();
                        schedule();

The kernel panic is here:
[    4.609601] BUG: scheduling while atomic: swapper/1/0/0x00000002
[    4.609608] [<ffff0000086fae70>] arm_enter_idle_state+0x18/0x70
[    4.609614] Modules linked in:
[    4.609615] [<ffff0000086f9298>] cpuidle_enter_state+0xf0/0x218
[    4.609620] [<ffff0000086f93f8>] cpuidle_enter+0x18/0x20
[    4.609626] Preemption disabled at:
[    4.609627] [<ffff0000080fa234>] call_cpuidle+0x24/0x40
[    4.609635] [<ffff000008882fa4>] schedule_preempt_disabled+0x1c/0x28
[    4.609639] [<ffff0000080fa49c>] cpu_startup_entry+0x154/0x1f8
[    4.609645] [<ffff00000808e004>] secondary_start_kernel+0x15c/0x1a0

Daniel Lezcano said this notification is needed on arm/arm64 platforms.
Sebastian suggested using atomic_notifier instead of rwlock, which is not
only removing the sleeping in idle, but also improving latency.

Tony Lindgren found a miss use that rcu_read_lock used after rcu_idle_enter
Paul McKenney suggested trying RCU_NONIDLE.

Signed-off-by: Alex Shi <alex.shi@linaro.org>
Tested-by: Tony Lindgren <tony@atomide.com>
Acked-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
[ rjw: Subject & changelog ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

2017-07-31 13:09:49 +02:00

..

bpf

Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net

2017-07-20 16:33:39 -07:00

cgroup

mm, cpuset: always use seqlock when changing task's nodemask

2017-07-06 16:24:34 -07:00

configs

config: android-base: disable CONFIG_NFSD and CONFIG_NFS_FS

2017-06-09 11:47:38 +02:00

debug

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>

2017-03-02 08:42:34 +01:00

events

Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2017-07-21 11:12:48 -07:00

gcov

gcov: support GCC 7.1

2017-05-12 15:57:15 -07:00

irq

genirq/cpuhotplug: Revert "Set force affinity flag on hotplug migration"

2017-07-27 15:40:02 +02:00

livepatch

livepatch: Fix stacking of patches with respect to RCU

2017-06-20 10:42:19 +02:00

locking

Merge branch 'locking-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2017-07-21 11:11:23 -07:00

power

More power management updates for v4.13-rc1

2017-07-10 15:16:21 -07:00

printk

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/pmladek/printk

2017-07-05 11:11:26 -07:00

rcu

rcu: Remove RCU CPU stall warnings from Tiny RCU

2017-06-08 18:52:45 -07:00

sched

sched/core: Fix some documentation build warnings

2017-07-25 11:17:02 +02:00

time

Merge branch 'timers-compat' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2017-07-05 15:34:35 -07:00

trace

trace: fix the errors caused by incompatible type of RCU variables

2017-07-20 09:27:29 -04:00

.gitignore

certs: add .gitignore to stop git nagging about x509_certificate_list

2015-10-21 15:18:35 +01:00

acct.c

sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>

2017-03-02 08:42:39 +01:00

async.c

async: Adjust system_state checks

2017-05-23 10:01:37 +02:00

audit_fsnotify.c

Merge branch 'fsnotify' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs

2017-05-03 11:05:15 -07:00

audit_tree.c

Merge branch 'fsnotify' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs

2017-05-03 11:05:15 -07:00

audit_watch.c

Merge branch 'fsnotify' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs

2017-05-03 11:05:15 -07:00

audit.c

Merge branch 'stable-4.13' of git://git.infradead.org/users/pcmoore/audit

2017-07-20 10:22:26 -07:00

audit.h

audit: style fix

2017-06-12 18:07:43 -04:00

auditfilter.c

audit: kernel generated netlink traffic should have a portid of 0

2017-05-02 10:16:05 -04:00

auditsc.c

Merge branch 'stable-4.13' of git://git.infradead.org/users/pcmoore/audit

2017-07-05 11:24:05 -07:00

backtracetest.c

…

bounds.c

…

capability.c

capability: export has_capability

2017-01-12 07:01:56 -07:00

compat.c

Merge branch 'misc.compat' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2017-07-06 20:57:13 -07:00

configs.c

Replace <asm/uaccess.h> with <linux/uaccess.h> globally

2016-12-24 11:46:01 -08:00

context_tracking.c

context_tracking: Switch to new static_branch API

2015-11-24 09:56:43 +01:00

cpu_pm.c

PM / CPU: replace raw_notifier with atomic_notifier

2017-07-31 13:09:49 +02:00

cpu.c

smp/hotplug: Replace BUG_ON and react useful

2017-07-11 22:25:44 +02:00

crash_core.c

kdump: protect vmcoreinfo data under the crash memory

2017-07-12 16:26:00 -07:00

crash_dump.c

…

cred.c

doc: ReSTify credentials.txt

2017-05-18 10:30:19 -06:00

delayacct.c

sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>

2017-03-02 08:42:39 +01:00

dma.c

…

elfcore.c

…

exec_domain.c

…

exit.c

kernel/exit.c: avoid undefined behaviour when calling wait4()

2017-07-10 16:32:36 -07:00

extable.c

lib/extable.c: use bsearch() library function in search_extable()

2017-07-10 16:32:35 -07:00

fork.c

fork,random: use get_random_canary() to set tsk->stack_canary

2017-07-12 16:26:03 -07:00

freezer.c

freezer, oom: check TIF_MEMDIE on the correct task

2016-07-28 16:07:41 -07:00

futex_compat.c

Replace <asm/uaccess.h> with <linux/uaccess.h> globally

2016-12-24 11:46:01 -08:00

futex.c

Now that IPC and other changes have landed, enable manual markings for

2017-07-19 08:55:18 -07:00

groups.c

kernel/groups.c: use sort library function

2017-07-10 16:32:34 -07:00

hung_task.c

kernel/hung_task.c: defer showing held locks

2017-05-08 17:15:10 -07:00

irq_work.c

treewide: Remove old email address

2015-11-23 09:44:58 +01:00

jump_label.c

jump_label: Reorder hotplug lock and jump_label_lock

2017-05-26 10:10:45 +02:00

kallsyms.c

kernel/kallsyms.c: replace all_var with IS_ENABLED(CONFIG_KALLSYMS_ALL)

2017-07-10 16:32:34 -07:00

kcmp.c

kcmp: add KCMP_EPOLL_TFD mode to compare epoll target files

2017-07-12 16:26:01 -07:00

Kconfig.freezer

…

Kconfig.hz

…

Kconfig.locks

locking/mutex: Allow MUTEX_SPIN_ON_OWNER when DEBUG_MUTEXES

2016-10-25 11:31:51 +02:00

Kconfig.preempt

…

kcov.c

kcov: simplify interrupt check

2017-05-08 17:15:12 -07:00

kexec_core.c

kdump: protect vmcoreinfo data under the crash memory

2017-07-12 16:26:00 -07:00

kexec_file.c

kexec_file: adjust declaration of kexec_purgatory

2017-07-12 16:26:02 -07:00

kexec_internal.h

kexec_file: adjust declaration of kexec_purgatory

2017-07-12 16:26:02 -07:00

kexec.c

kdump: protect vmcoreinfo data under the crash memory

2017-07-12 16:26:00 -07:00

kmod.c

kmod: throttle kmod thread limit

2017-07-14 15:05:13 -07:00

kprobes.c

kprobes: Ensure that jprobe probepoints are at function entry

2017-07-08 11:05:35 +02:00

ksysfs.c

kexec: move vmcoreinfo out of the kernel's .bss section

2017-07-12 16:25:59 -07:00

kthread.c

cgroup, kthread: close race window where new kthreads can be migrated to non-root cgroups

2017-03-17 10:18:47 -04:00

latencytop.c

sched/headers: Prepare to move sched_info_on() and force_schedstat_enabled() from <linux/sched.h> to <linux/sched/stat.h>

2017-03-02 08:42:39 +01:00

Makefile

kernel/watchdog: split up config options

2017-07-12 16:26:02 -07:00

membarrier.c

Fix: Disable sys_membarrier when nohz_full is enabled

2017-01-23 11:32:16 -08:00

memremap.c

mm, memory_hotplug: replace for_device by want_memblock in arch_add_memory

2017-07-06 16:24:32 -07:00

module_signing.c

KEYS: Move the point of trust determination to __key_link()

2016-04-11 22:43:43 +01:00

module-internal.h

…

module.c

Modules updates for v4.13

2017-07-12 17:22:01 -07:00

notifier.c

kernel/notifier.c: simplify expression

2017-02-24 17:46:56 -08:00

nsproxy.c

perf: Add PERF_RECORD_NAMESPACES to include namespaces related info

2017-03-13 15:57:41 -03:00

padata.c

padata: Avoid nested calls to cpus_read_lock() in pcrypt_init_padata()

2017-05-26 10:10:37 +02:00

panic.c

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>

2017-03-02 08:42:34 +01:00

params.c

boot/param: Move next_arg() function to lib/cmdline.c for later reuse

2017-04-18 10:37:13 +02:00

pid_namespace.c

pid_ns: Sleep in TASK_INTERRUPTIBLE in zap_pid_ns_processes

2017-05-13 17:26:01 -05:00

pid.c

mm: update callers to use HASH_ZERO flag

2017-07-06 16:24:33 -07:00

profile.c

sched/headers: Prepare to move sched_info_on() and force_schedstat_enabled() from <linux/sched.h> to <linux/sched/stat.h>

2017-03-02 08:42:39 +01:00

ptrace.c

ptrace: Properly initialize ptracer_cred on fork

2017-05-23 07:40:44 -05:00

range.c

…

reboot.c

kexec: split kexec_load syscall from kexec core code

2015-09-10 13:29:01 -07:00

relay.c

Merge branch 'work.splice' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2017-05-02 11:38:06 -07:00

resource.c

/proc/iomem: only expose physical resource addresses to privileged users

2016-04-14 12:56:09 -07:00

seccomp.c

seccomp: Switch from atomic_t to recount_t

2017-06-26 09:24:00 -07:00

signal.c

kernel/signal.c: avoid undefined behaviour in kill_something_info

2017-07-10 16:32:36 -07:00

smp.c

smp, cpumask: Use non-atomic cpumask_{set,clear}_cpu()

2017-05-23 10:01:32 +02:00

smpboot.c

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>

2017-03-02 08:42:35 +01:00

smpboot.h

cpu/hotplug: Create hotplug threads

2016-03-01 20:36:56 +01:00

softirq.c

sched/core: Remove 'task' parameter and rename tsk_restore_flags() to current_restore_flags()

2017-04-11 09:06:32 +02:00

stacktrace.c

stacktrace/x86: add function for detecting reliable stack traces

2017-03-08 09:18:02 +01:00

stop_machine.c

stop_machine: Provide stop_machine_cpuslocked()

2017-05-26 10:10:36 +02:00

sys_ni.c

move aio compat to fs/aio.c

2016-12-22 22:58:37 -05:00

sys.c

fix a braino in compat_sys_getrlimit()

2017-07-12 09:15:00 -07:00

sysctl_binary.c

kernel/sysctl_binary.c: check name array length in deprecated_sysctl_warning()

2017-07-12 16:26:00 -07:00

sysctl.c

kernel/watchdog: split up config options

2017-07-12 16:26:02 -07:00

task_work.c

task_work: use READ_ONCE/lockless_dereference, avoid pi_lock if !task_works

2016-08-02 19:35:02 -04:00

taskstats.c

taskstats: add e/u/stime for TGID command

2017-05-08 17:15:12 -07:00

test_kprobes.c

…

torture.c

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/clock.h>

2017-03-02 08:42:27 +01:00

tracepoint.c

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>

2017-03-02 08:42:35 +01:00

tsacct.c

sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>

2017-03-02 08:42:39 +01:00

ucount.c

ucount: Remove the atomicity from ucount->count

2017-03-06 15:26:37 -06:00

uid16.c

sched/headers: Prepare to remove <linux/cred.h> inclusion from <linux/sched.h>

2017-03-02 08:42:31 +01:00

up.c

smp: Add function to execute a function synchronously on a CPU

2016-09-05 13:52:39 +02:00

user_namespace.c

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/signal.h>

2017-03-02 08:42:29 +01:00

user-return-notifier.c

…

user.c

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/user.h>

2017-03-02 08:42:29 +01:00

utsname_sysctl.c

sched/headers: Remove <linux/rwsem.h> from <linux/sched.h>

2017-03-03 01:45:36 +01:00

utsname.c

sched/headers: Prepare to move the task_lock()/unlock() APIs to <linux/sched/task.h>

2017-03-02 08:42:38 +01:00

watchdog_hld.c

kernel/watchdog: split up config options

2017-07-12 16:26:02 -07:00

watchdog.c

kernel/watchdog.c: use better pr_fmt prefix

2017-07-14 15:05:13 -07:00

workqueue_internal.h

sched/core: Get rid of 'cpu' argument in wq_worker_sleeping()

2016-03-02 10:28:47 -05:00

workqueue.c

sched/wait: Rename wait_queue_t => wait_queue_entry_t

2017-06-20 12:18:27 +02:00