f6cc0c5016
Patching a jump label involves patching a single instruction at a time,
swizzling between a branch and a NOP. The architecture treats these
instructions specially, so a concurrently executing CPU is guaranteed to
see either the NOP or the branch, rather than an amalgamation of the two
instruction encodings.
However, in order to guarantee that the new instruction is visible, it
is necessary to send an IPI to the concurrently executing CPU so that it
discards any previously fetched instructions from its pipeline. This
operation therefore cannot be completed from a context with IRQs
disabled, but this is exactly what happens on the jump label path where
the hotplug lock is held and irqs are subsequently disabled by
stop_machine_cpuslocked(). This results in a deadlock during boot on
Hikey-960.
Due to the architectural guarantees around patching NOPs and branches,
we don't actually need to stop_machine() at all on the jump label path,
so we can avoid the deadlock by using the "nosync" variant of our
instruction patching routine.
Fixes: 693350a79980 ("arm64: insn: Don't fallback on nosync path for general insn patching")
Reported-by: Tuomas Tynkkynen <tuomas.tynkkynen@iki.fi>
Reported-by: John Stultz <john.stultz@linaro.org>
Tested-by: Valentin Schneider <valentin.schneider@arm.com>
Tested-by: Tuomas Tynkkynen <tuomas@tuxera.com>
Tested-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
54 lines
1.6 KiB
C
54 lines
1.6 KiB
C
/*
|
|
* Copyright (C) 2013 Huawei Ltd.
|
|
* Author: Jiang Liu <liuj97@gmail.com>
|
|
*
|
|
* Based on arch/arm/kernel/jump_label.c
|
|
*
|
|
* This program is free software; you can redistribute it and/or modify
|
|
* it under the terms of the GNU General Public License version 2 as
|
|
* published by the Free Software Foundation.
|
|
*
|
|
* This program is distributed in the hope that it will be useful,
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
* GNU General Public License for more details.
|
|
*
|
|
* You should have received a copy of the GNU General Public License
|
|
* along with this program. If not, see <http://www.gnu.org/licenses/>.
|
|
*/
|
|
#include <linux/kernel.h>
|
|
#include <linux/jump_label.h>
|
|
#include <asm/insn.h>
|
|
|
|
#ifdef HAVE_JUMP_LABEL
|
|
|
|
void arch_jump_label_transform(struct jump_entry *entry,
|
|
enum jump_label_type type)
|
|
{
|
|
void *addr = (void *)entry->code;
|
|
u32 insn;
|
|
|
|
if (type == JUMP_LABEL_JMP) {
|
|
insn = aarch64_insn_gen_branch_imm(entry->code,
|
|
entry->target,
|
|
AARCH64_INSN_BRANCH_NOLINK);
|
|
} else {
|
|
insn = aarch64_insn_gen_nop();
|
|
}
|
|
|
|
aarch64_insn_patch_text_nosync(addr, insn);
|
|
}
|
|
|
|
void arch_jump_label_transform_static(struct jump_entry *entry,
|
|
enum jump_label_type type)
|
|
{
|
|
/*
|
|
* We use the architected A64 NOP in arch_static_branch, so there's no
|
|
* need to patch an identical A64 NOP over the top of it here. The core
|
|
* will call arch_jump_label_transform from a module notifier if the
|
|
* NOP needs to be replaced by a branch.
|
|
*/
|
|
}
|
|
|
|
#endif /* HAVE_JUMP_LABEL */
|