iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
linux/arch/riscv/kernel/entry.S

591 lines
14 KiB
ArmAsm
Raw Normal View History

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 286 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license as published by the free software foundation version 2 this program is distributed in the hope that it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 97 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Reviewed-by: Alexios Zavras <alexios.zavras@intel.com> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190529141901.025053186@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-29 07:18:00 -07:00
/* SPDX-License-Identifier: GPL-2.0-only */
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
/*
* Copyright (C) 2012 Regents of the University of California
* Copyright (C) 2017 SiFive
*/
#include <linux/init.h>
#include <linux/linkage.h>
#include <asm/asm.h>
#include <asm/csr.h>
#include <asm/unistd.h>
#include <asm/thread_info.h>
#include <asm/asm-offsets.h>
riscv: sifive: Apply errata "cip-453" patch Add sign extension to the $badaddr before addressing the instruction page fault and instruction access fault to workaround the issue "cip-453". To avoid affecting the existing code sequence, this patch will creates two trampolines to add sign extension to the $badaddr. By the "alternative" mechanism, these two trampolines will replace the original exception handler of instruction page fault and instruction access fault in the excp_vect_table. In this case, only the specific SiFive CPU core jumps to the do_page_fault and do_trap_insn_fault through these two trampolines. Other CPUs are not affected. Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:05 +08:00
#include <asm/errata_list.h>
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
RISC-V: Inline the assembly register save/restore macros These are only used once, and when reading the code I've always found them to be more of a headache than a benefit. While they were never worth removing before, LLVM's integrated assembler doesn't support LOCAL so rather that trying to figure out how to refactor the macros it seems saner to just inline them. Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-02-27 11:15:03 -08:00
#if !IS_ENABLED(CONFIG_PREEMPTION)
.set resume_kernel, restore_all
#endif
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
RISC-V: Inline the assembly register save/restore macros These are only used once, and when reading the code I've always found them to be more of a headache than a benefit. While they were never worth removing before, LLVM's integrated assembler doesn't support LOCAL so rather that trying to figure out how to refactor the macros it seems saner to just inline them. Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-02-27 11:15:03 -08:00
ENTRY(handle_exception)
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
/*
* If coming from userspace, preserve the user thread pointer and load
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
* the kernel thread pointer. If we came from the kernel, the scratch
* register will contain 0, and we should continue on the current TP.
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
*/
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
csrrw tp, CSR_SCRATCH, tp
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
bnez tp, _save_context
_restore_kernel_tpsp:
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
csrr tp, CSR_SCRATCH
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
REG_S sp, TASK_TI_KERNEL_SP(tp)
riscv: add VMAP_STACK overflow detection This patch adds stack overflow detection to riscv, usable when CONFIG_VMAP_STACK=y. Overflow is detected in kernel exception entry(kernel/entry.S), if the kernel stack is overflow and been detected, the overflow handler is invoked on a per-cpu overflow stack. This approach preserves GPRs and the original exception information. The overflow detect is performed before any attempt is made to access the stack and the principle of stack overflow detection: kernel stacks are aligned to double their size, enabling overflow to be detected with a single bit test. For example, a 16K stack is aligned to 32K, ensuring that bit 14 of the SP must be zero. On an overflow (or underflow), this bit is flipped. Thus, overflow (of less than the size of the stack) can be detected by testing whether this bit is set. This gives us a useful error message on stack overflow, as can be trigger with the LKDTM overflow test: [ 388.053267] lkdtm: Performing direct entry EXHAUST_STACK [ 388.053663] lkdtm: Calling function with 1024 frame size to depth 32 ... [ 388.054016] lkdtm: loop 32/32 ... [ 388.054186] lkdtm: loop 31/32 ... [ 388.054491] lkdtm: loop 30/32 ... [ 388.054672] lkdtm: loop 29/32 ... [ 388.054859] lkdtm: loop 28/32 ... [ 388.055010] lkdtm: loop 27/32 ... [ 388.055163] lkdtm: loop 26/32 ... [ 388.055309] lkdtm: loop 25/32 ... [ 388.055481] lkdtm: loop 24/32 ... [ 388.055653] lkdtm: loop 23/32 ... [ 388.055837] lkdtm: loop 22/32 ... [ 388.056015] lkdtm: loop 21/32 ... [ 388.056188] lkdtm: loop 20/32 ... [ 388.058145] Insufficient stack space to handle exception! [ 388.058153] Task stack: [0xffffffd014260000..0xffffffd014264000] [ 388.058160] Overflow stack: [0xffffffe1f8d2c220..0xffffffe1f8d2d220] [ 388.058168] CPU: 0 PID: 89 Comm: bash Not tainted 5.12.0-rc8-dirty #90 [ 388.058175] Hardware name: riscv-virtio,qemu (DT) [ 388.058187] epc : number+0x32/0x2c0 [ 388.058247] ra : vsnprintf+0x2ae/0x3f0 [ 388.058255] epc : ffffffe0002d38f6 ra : ffffffe0002d814e sp : ffffffd01425ffc0 [ 388.058263] gp : ffffffe0012e4010 tp : ffffffe08014da00 t0 : ffffffd0142606e8 [ 388.058271] t1 : 0000000000000000 t2 : 0000000000000000 s0 : ffffffd014260070 [ 388.058303] s1 : ffffffd014260158 a0 : ffffffd01426015e a1 : ffffffd014260158 [ 388.058311] a2 : 0000000000000013 a3 : ffff0a01ffffff10 a4 : ffffffe000c398e0 [ 388.058319] a5 : 511b02ec65f3e300 a6 : 0000000000a1749a a7 : 0000000000000000 [ 388.058327] s2 : ffffffff000000ff s3 : 00000000ffff0a01 s4 : ffffffe0012e50a8 [ 388.058335] s5 : 0000000000ffff0a s6 : ffffffe0012e50a8 s7 : ffffffe000da1cc0 [ 388.058343] s8 : ffffffffffffffff s9 : ffffffd0142602b0 s10: ffffffd0142602a8 [ 388.058351] s11: ffffffd01426015e t3 : 00000000000f0000 t4 : ffffffffffffffff [ 388.058359] t5 : 000000000000002f t6 : ffffffd014260158 [ 388.058366] status: 0000000000000100 badaddr: ffffffd01425fff8 cause: 000000000000000f [ 388.058374] Kernel panic - not syncing: Kernel stack overflow [ 388.058381] CPU: 0 PID: 89 Comm: bash Not tainted 5.12.0-rc8-dirty #90 [ 388.058387] Hardware name: riscv-virtio,qemu (DT) [ 388.058393] Call Trace: [ 388.058400] [<ffffffe000004944>] walk_stackframe+0x0/0xce [ 388.058406] [<ffffffe0006f0b28>] dump_backtrace+0x38/0x46 [ 388.058412] [<ffffffe0006f0b46>] show_stack+0x10/0x18 [ 388.058418] [<ffffffe0006f3690>] dump_stack+0x74/0x8e [ 388.058424] [<ffffffe0006f0d52>] panic+0xfc/0x2b2 [ 388.058430] [<ffffffe0006f0acc>] print_trace_address+0x0/0x24 [ 388.058436] [<ffffffe0002d814e>] vsnprintf+0x2ae/0x3f0 [ 388.058956] SMP: stopping secondary CPUs Signed-off-by: Tong Tiangen <tongtiangen@huawei.com> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-06-21 11:28:55 +08:00
#ifdef CONFIG_VMAP_STACK
addi sp, sp, -(PT_SIZE_ON_STACK)
srli sp, sp, THREAD_SHIFT
andi sp, sp, 0x1
bnez sp, handle_kernel_stack_overflow
REG_L sp, TASK_TI_KERNEL_SP(tp)
#endif
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
_save_context:
REG_S sp, TASK_TI_USER_SP(tp)
REG_L sp, TASK_TI_KERNEL_SP(tp)
addi sp, sp, -(PT_SIZE_ON_STACK)
REG_S x1, PT_RA(sp)
REG_S x3, PT_GP(sp)
REG_S x5, PT_T0(sp)
REG_S x6, PT_T1(sp)
REG_S x7, PT_T2(sp)
REG_S x8, PT_S0(sp)
REG_S x9, PT_S1(sp)
REG_S x10, PT_A0(sp)
REG_S x11, PT_A1(sp)
REG_S x12, PT_A2(sp)
REG_S x13, PT_A3(sp)
REG_S x14, PT_A4(sp)
REG_S x15, PT_A5(sp)
REG_S x16, PT_A6(sp)
REG_S x17, PT_A7(sp)
REG_S x18, PT_S2(sp)
REG_S x19, PT_S3(sp)
REG_S x20, PT_S4(sp)
REG_S x21, PT_S5(sp)
REG_S x22, PT_S6(sp)
REG_S x23, PT_S7(sp)
REG_S x24, PT_S8(sp)
REG_S x25, PT_S9(sp)
REG_S x26, PT_S10(sp)
REG_S x27, PT_S11(sp)
REG_S x28, PT_T3(sp)
REG_S x29, PT_T4(sp)
REG_S x30, PT_T5(sp)
REG_S x31, PT_T6(sp)
/*
riscv: disable SUM in the exception handler The SUM bit is enabled at the beginning of the copy_{to,from}_user and {get,put}_user routines, and cleared before they return. But these user copy helper can be interrupted by exceptions, in which case the SUM bit will remain set, which leads to elevated privileges for the code running in exception context, as that can now access userspace address space unconditionally. This frequently happens when the user copy routines access freshly allocated user memory that hasn't been faulted in, and a pagefault needs to be taken before the user copy routines can continue. Fix this by unconditionally clearing SUM when the exception handler is called - the restore code will automatically restore it based on the saved value. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-01-04 19:55:55 +01:00
* Disable user-mode memory access as it should only be set in the
* actual user copy routines.
*
* Disable the FPU to detect illegal usage of floating point in kernel
* space.
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
*/
riscv: disable SUM in the exception handler The SUM bit is enabled at the beginning of the copy_{to,from}_user and {get,put}_user routines, and cleared before they return. But these user copy helper can be interrupted by exceptions, in which case the SUM bit will remain set, which leads to elevated privileges for the code running in exception context, as that can now access userspace address space unconditionally. This frequently happens when the user copy routines access freshly allocated user memory that hasn't been faulted in, and a pagefault needs to be taken before the user copy routines can continue. Fix this by unconditionally clearing SUM when the exception handler is called - the restore code will automatically restore it based on the saved value. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-01-04 19:55:55 +01:00
li t0, SR_SUM | SR_FS
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
REG_L s0, TASK_TI_USER_SP(tp)
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
csrrc s1, CSR_STATUS, t0
csrr s2, CSR_EPC
csrr s3, CSR_TVAL
csrr s4, CSR_CAUSE
csrr s5, CSR_SCRATCH
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
REG_S s0, PT_SP(sp)
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
REG_S s1, PT_STATUS(sp)
REG_S s2, PT_EPC(sp)
REG_S s3, PT_BADADDR(sp)
REG_S s4, PT_CAUSE(sp)
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
REG_S s5, PT_TP(sp)
/*
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
* Set the scratch register to 0, so that if a recursive exception
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
* occurs, the exception vector knows it came from the kernel
*/
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
csrw CSR_SCRATCH, x0
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
/* Load the global pointer */
.option push
.option norelax
la gp, __global_pointer$
.option pop
riscv: Enable LOCKDEP_SUPPORT & fixup TRACE_IRQFLAGS_SUPPORT Lockdep is needed by proving the spinlocks and rwlocks. To suupport it, we need fixup TRACE_IRQFLAGS_SUPPORT in kernel/entry.S. This patch follow Documentation/irqflags-tracing.txt. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-27 13:57:08 +00:00
#ifdef CONFIG_TRACE_IRQFLAGS
call trace_hardirqs_off
#endif
riscv: Enable context tracking This patch implements and enables context tracking for riscv (which is a prerequisite for CONFIG_NO_HZ_FULL support) It adds checking for previous state in the entry that all excepttions and interrupts goes to and calls context_tracking_user_exit() if it comes from user space. It also calls context_tracking_user_enter() if it will return to user space before restore_all. This patch is tested with the dynticks-testing testcase in qemu-system-riscv64 virt machine and Unleashed board. git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git We can see the log here. The tick got mostly stopped during the execution of the user loop. _-----=> irqs-off / _----=> need-resched | / _---=> hardirq/softirq || / _--=> preempt-depth ||| / delay TASK-PID CPU# |||| TIMESTAMP FUNCTION | | | |||| | | <idle>-0 [001] d..2 604.183512: sched_switch: prev_comm=swapper/1 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=taskset next_pid=273 next_prio=120 user_loop-273 [001] d.h1 604.184788: hrtimer_expire_entry: hrtimer=000000002eda5fab function=tick_sched_timer now=604176096300 user_loop-273 [001] d.s2 604.184897: workqueue_queue_work: work struct=00000000383402c2 function=vmstat_update workqueue=00000000f36d35d4 req_cpu=1 cpu=1 user_loop-273 [001] dns2 604.185039: tick_stop: success=0 dependency=SCHED user_loop-273 [001] dn.1 604.185103: tick_stop: success=0 dependency=SCHED user_loop-273 [001] d..2 604.185154: sched_switch: prev_comm=taskset prev_pid=273 prev_prio=120 prev_state=R+ ==> next_comm=kworker/1:1 next_pid=46 next_prio=120 <...>-46 [001] .... 604.185194: workqueue_execute_start: work struct 00000000383402c2: function vmstat_update <...>-46 [001] d..2 604.185266: sched_switch: prev_comm=kworker/1:1 prev_pid=46 prev_prio=120 prev_state=I ==> next_comm=taskset next_pid=273 next_prio=120 user_loop-273 [001] d.h1 604.188812: hrtimer_expire_entry: hrtimer=000000002eda5fab function=tick_sched_timer now=604180133400 user_loop-273 [001] d..1 604.189050: tick_stop: success=1 dependency=NONE user_loop-273 [001] d..2 614.251386: sched_switch: prev_comm=user_loop prev_pid=273 prev_prio=120 prev_state=X ==> next_comm=swapper/1 next_pid=0 next_prio=120 <idle>-0 [001] d..2 614.315391: sched_switch: prev_comm=swapper/1 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=taskset next_pid=276 next_prio=120 Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-24 17:03:16 +08:00
#ifdef CONFIG_CONTEXT_TRACKING
/* If previous state is in user mode, call context_tracking_user_exit. */
li a0, SR_PP
and a0, s1, a0
bnez a0, skip_context_tracking
call context_tracking_user_exit
skip_context_tracking:
#endif
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
/*
* MSB of cause differentiates between
* interrupts and exceptions
*/
bge s4, zero, 1f
riscv: Enable LOCKDEP_SUPPORT & fixup TRACE_IRQFLAGS_SUPPORT Lockdep is needed by proving the spinlocks and rwlocks. To suupport it, we need fixup TRACE_IRQFLAGS_SUPPORT in kernel/entry.S. This patch follow Documentation/irqflags-tracing.txt. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-27 13:57:08 +00:00
la ra, ret_from_exception
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
/* Handle interrupts */
RISC-V: Move to the new GENERIC_IRQ_MULTI_HANDLER handler The existing mechanism for handling IRQs on RISC-V is pretty ugly: the irq entry code selects the handler via Kconfig dependencies. Use the new generic IRQ handling infastructure, which allows boot time registration of the low level entry handler. This does add an additional load to the interrupt latency, but there's a lot of tuning left to be done there on RISC-V so it's OK for now. Signed-off-by: Palmer Dabbelt <palmer@sifive.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Christoph Hellwig <hch@lst.de> Acked-by: Stafford Horne <shorne@gmail.com> Cc: jonas@southpole.se Cc: catalin.marinas@arm.com Cc: Will Deacon <will.deacon@arm.com> Cc: linux@armlinux.org.uk Cc: stefan.kristiansson@saunalahti.fi Cc: openrisc@lists.librecores.org Cc: linux-riscv@lists.infradead.org Cc: linux-arm-kernel@lists.infradead.org Link: https://lkml.kernel.org/r/20180307235731.22627-3-palmer@sifive.com
2018-03-07 15:57:28 -08:00
move a0, sp /* pt_regs */
RISC-V: Remove do_IRQ() function The only thing do_IRQ() does is call handle_arch_irq function pointer. We can very well call handle_arch_irq function pointer directly from assembly and remove do_IRQ() function hence this patch. Signed-off-by: Anup Patel <anup.patel@wdc.com> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-01 14:45:42 +05:30
la a1, handle_arch_irq
REG_L a1, (a1)
jr a1
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
1:
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
/*
* Exceptions run with interrupts enabled or disabled depending on the
* state of SR_PIE in m/sstatus.
*/
andi t0, s1, SR_PIE
riscv: Avoid interrupts being erroneously enabled in handle_exception() When the handle_exception function addresses an exception, the interrupts will be unconditionally enabled after finishing the context save. However, It may erroneously enable the interrupts if the interrupts are disabled before entering the handle_exception. For example, one of the WARN_ON() condition is satisfied in the scheduling where the interrupt is disabled and rq.lock is locked. The WARN_ON will trigger a break exception and the handle_exception function will enable the interrupts before entering do_trap_break function. During the procedure, if a timer interrupt is pending, it will be taken when interrupts are enabled. In this case, it may cause a deadlock problem if the rq.lock is locked again in the timer ISR. Hence, the handle_exception() can only enable interrupts when the state of sstatus.SPIE is 1. This patch is tested on HiFive Unleashed board. Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Reviewed-by: Palmer Dabbelt <palmer@sifive.com> [paul.walmsley@sifive.com: updated to apply] Fixes: bcae803a21317 ("RISC-V: Enable IRQ during exception handling") Cc: David Abdurachmanov <david.abdurachmanov@sifive.com> Cc: stable@vger.kernel.org Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-09-16 16:47:41 +08:00
beqz t0, 1f
riscv: keep interrupts disabled for BREAKPOINT exception Current riscv's kprobe handlers are run with both preemption and interrupt enabled, this violates kprobe requirements. Fix this issue by keeping interrupts disabled for BREAKPOINT exception. Fixes: c22b0bcb1dd0 ("riscv: Add kprobes supported") Cc: stable@vger.kernel.org Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> [Palmer: add a comment] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-30 02:16:24 +08:00
/* kprobes, entered via ebreak, must have interrupts disabled. */
li t0, EXC_BREAKPOINT
beq s4, t0, 1f
riscv: Trace irq on only interrupt is enabled We should call irq trace only if interrupt is going to be enabled during excecption handling. Otherwise, it results in following warning during boot with lock debugging enabled. [ 0.000000] ------------[ cut here ]------------ [ 0.000000] DEBUG_LOCKS_WARN_ON(early_boot_irqs_disabled) [ 0.000000] WARNING: CPU: 0 PID: 0 at kernel/locking/lockdep.c:4085 lockdep_hardirqs_on_prepare+0x22a/0x22e [ 0.000000] Modules linked in: [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.10.0-00022-ge20097fb37e2-dirty #548 [ 0.000000] epc: c005d5d4 ra : c005d5d4 sp : c1c01e80 [ 0.000000] gp : c1d456e0 tp : c1c0a980 t0 : 00000000 [ 0.000000] t1 : ffffffff t2 : 00000000 s0 : c1c01ea0 [ 0.000000] s1 : c100f360 a0 : 0000002d a1 : c00666ee [ 0.000000] a2 : 00000000 a3 : 00000000 a4 : 00000000 [ 0.000000] a5 : 00000000 a6 : c1c6b390 a7 : 3ffff00e [ 0.000000] s2 : c2384fe8 s3 : 00000000 s4 : 00000001 [ 0.000000] s5 : c1c0a980 s6 : c1d48000 s7 : c1613b4c [ 0.000000] s8 : 00000fff s9 : 80000200 s10: c1613b40 [ 0.000000] s11: 00000000 t3 : 00000000 t4 : 00000000 [ 0.000000] t5 : 00000001 t6 : 00000000 Fixes: 3c4697982982 ("riscv:Enable LOCKDEP_SUPPORT & fixup TRACE_IRQFLAGS_SUPPORT") Signed-off-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-18 16:20:51 -08:00
#ifdef CONFIG_TRACE_IRQFLAGS
call trace_hardirqs_on
#endif
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
csrs CSR_STATUS, SR_IE
RISC-V: Enable IRQ during exception handling Interrupt is allowed during exception handling. There are warning messages if the kernel enables the configuration 'CONFIG_DEBUG_ATOMIC_SLEEP=y'. BUG: sleeping function called from invalid context at kernel/locking/rwsem.c:23 in_atomic(): 0, irqs_disabled(): 1, pid: 43, name: ash CPU: 0 PID: 43 Comm: ash Tainted: G W 4.15.0-rc8-00089-g89ffdae-dirty #17 Call Trace: [<000000009abb1587>] walk_stackframe+0x0/0x7a [<00000000d4f3d088>] ___might_sleep+0x102/0x11a [<00000000b1fd792a>] down_read+0x18/0x28 [<000000000289ec01>] do_page_fault+0x86/0x2f6 [<00000000012441f6>] _do_fork+0x1b4/0x1e0 [<00000000f46c3e3b>] ret_from_syscall+0xa/0xe Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zong Li <zong@andestech.com> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-01-29 23:51:45 -08:00
riscv: Avoid interrupts being erroneously enabled in handle_exception() When the handle_exception function addresses an exception, the interrupts will be unconditionally enabled after finishing the context save. However, It may erroneously enable the interrupts if the interrupts are disabled before entering the handle_exception. For example, one of the WARN_ON() condition is satisfied in the scheduling where the interrupt is disabled and rq.lock is locked. The WARN_ON will trigger a break exception and the handle_exception function will enable the interrupts before entering do_trap_break function. During the procedure, if a timer interrupt is pending, it will be taken when interrupts are enabled. In this case, it may cause a deadlock problem if the rq.lock is locked again in the timer ISR. Hence, the handle_exception() can only enable interrupts when the state of sstatus.SPIE is 1. This patch is tested on HiFive Unleashed board. Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Reviewed-by: Palmer Dabbelt <palmer@sifive.com> [paul.walmsley@sifive.com: updated to apply] Fixes: bcae803a21317 ("RISC-V: Enable IRQ during exception handling") Cc: David Abdurachmanov <david.abdurachmanov@sifive.com> Cc: stable@vger.kernel.org Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-09-16 16:47:41 +08:00
1:
riscv: Enable LOCKDEP_SUPPORT & fixup TRACE_IRQFLAGS_SUPPORT Lockdep is needed by proving the spinlocks and rwlocks. To suupport it, we need fixup TRACE_IRQFLAGS_SUPPORT in kernel/entry.S. This patch follow Documentation/irqflags-tracing.txt. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-27 13:57:08 +00:00
la ra, ret_from_exception
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
/* Handle syscalls */
li t0, EXC_SYSCALL
beq s4, t0, handle_syscall
/* Handle other exceptions */
slli t0, s4, RISCV_LGPTR
la t1, excp_vect_table
la t2, excp_vect_table_end
move a0, sp /* pt_regs */
add t0, t1, t0
/* Check if exception code lies within bounds */
bgeu t0, t2, 1f
REG_L t0, 0(t0)
jr t0
1:
tail do_trap_unknown
handle_syscall:
riscv: Enable interrupts during syscalls with M-Mode When running is M-Mode (no MMU config), MPIE does not get set. This results in all syscalls being executed with interrupts disabled as handle_exception never sets SR_IE as it always sees SR_PIE being cleared. Fix this by always force enabling interrupts in handle_syscall when CONFIG_RISCV_M_MODE is enabled. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-13 22:50:36 +09:00
#ifdef CONFIG_RISCV_M_MODE
/*
* When running is M-Mode (no MMU config), MPIE does not get set.
* As a result, we need to force enable interrupts here because
* handle_exception did not do set SR_IE as it always sees SR_PIE
* being cleared.
*/
csrs CSR_STATUS, SR_IE
#endif
riscv: Enable context tracking This patch implements and enables context tracking for riscv (which is a prerequisite for CONFIG_NO_HZ_FULL support) It adds checking for previous state in the entry that all excepttions and interrupts goes to and calls context_tracking_user_exit() if it comes from user space. It also calls context_tracking_user_enter() if it will return to user space before restore_all. This patch is tested with the dynticks-testing testcase in qemu-system-riscv64 virt machine and Unleashed board. git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git We can see the log here. The tick got mostly stopped during the execution of the user loop. _-----=> irqs-off / _----=> need-resched | / _---=> hardirq/softirq || / _--=> preempt-depth ||| / delay TASK-PID CPU# |||| TIMESTAMP FUNCTION | | | |||| | | <idle>-0 [001] d..2 604.183512: sched_switch: prev_comm=swapper/1 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=taskset next_pid=273 next_prio=120 user_loop-273 [001] d.h1 604.184788: hrtimer_expire_entry: hrtimer=000000002eda5fab function=tick_sched_timer now=604176096300 user_loop-273 [001] d.s2 604.184897: workqueue_queue_work: work struct=00000000383402c2 function=vmstat_update workqueue=00000000f36d35d4 req_cpu=1 cpu=1 user_loop-273 [001] dns2 604.185039: tick_stop: success=0 dependency=SCHED user_loop-273 [001] dn.1 604.185103: tick_stop: success=0 dependency=SCHED user_loop-273 [001] d..2 604.185154: sched_switch: prev_comm=taskset prev_pid=273 prev_prio=120 prev_state=R+ ==> next_comm=kworker/1:1 next_pid=46 next_prio=120 <...>-46 [001] .... 604.185194: workqueue_execute_start: work struct 00000000383402c2: function vmstat_update <...>-46 [001] d..2 604.185266: sched_switch: prev_comm=kworker/1:1 prev_pid=46 prev_prio=120 prev_state=I ==> next_comm=taskset next_pid=273 next_prio=120 user_loop-273 [001] d.h1 604.188812: hrtimer_expire_entry: hrtimer=000000002eda5fab function=tick_sched_timer now=604180133400 user_loop-273 [001] d..1 604.189050: tick_stop: success=1 dependency=NONE user_loop-273 [001] d..2 614.251386: sched_switch: prev_comm=user_loop prev_pid=273 prev_prio=120 prev_state=X ==> next_comm=swapper/1 next_pid=0 next_prio=120 <idle>-0 [001] d..2 614.315391: sched_switch: prev_comm=swapper/1 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=taskset next_pid=276 next_prio=120 Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-24 17:03:16 +08:00
#if defined(CONFIG_TRACE_IRQFLAGS) || defined(CONFIG_CONTEXT_TRACKING)
riscv: Enable LOCKDEP_SUPPORT & fixup TRACE_IRQFLAGS_SUPPORT Lockdep is needed by proving the spinlocks and rwlocks. To suupport it, we need fixup TRACE_IRQFLAGS_SUPPORT in kernel/entry.S. This patch follow Documentation/irqflags-tracing.txt. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-27 13:57:08 +00:00
/* Recover a0 - a7 for system calls */
REG_L a0, PT_A0(sp)
REG_L a1, PT_A1(sp)
REG_L a2, PT_A2(sp)
REG_L a3, PT_A3(sp)
REG_L a4, PT_A4(sp)
REG_L a5, PT_A5(sp)
REG_L a6, PT_A6(sp)
REG_L a7, PT_A7(sp)
#endif
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
/* save the initial A0 value (needed in signal handlers) */
REG_S a0, PT_ORIG_A0(sp)
/*
* Advance SEPC to avoid executing the original
* scall instruction on sret
*/
addi s2, s2, 0x4
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
REG_S s2, PT_EPC(sp)
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
/* Trace syscalls, but only if requested by the user. */
REG_L t0, TASK_TI_FLAGS(tp)
riscv: add audit support On RISC-V (riscv) audit is supported through generic lib/audit.c. The patch adds required arch specific definitions. Signed-off-by: David Abdurachmanov <david.abdurachmanov@gmail.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-10-29 11:48:53 +01:00
andi t0, t0, _TIF_SYSCALL_WORK
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
bnez t0, handle_syscall_trace_enter
check_syscall_nr:
/* Check to make sure we don't jump to a bogus syscall number. */
li t0, __NR_syscalls
la s0, sys_ni_syscall
riscv: add support for SECCOMP and SECCOMP_FILTER This patch was extensively tested on Fedora/RISCV (applied by default on top of 5.2-rc7 kernel for <2 months). The patch was also tested with 5.3-rc on QEMU and SiFive Unleashed board. libseccomp (userspace) was rebased: https://github.com/seccomp/libseccomp/pull/134 Fully passes libseccomp regression testing (simulation and live). There is one failing kernel selftest: global.user_notification_signal v1 -> v2: - return immediately if secure_computing(NULL) returns -1 - fixed whitespace issues - add missing seccomp.h - remove patch #2 (solved now) - add riscv to seccomp kernel selftest Signed-off-by: David Abdurachmanov <david.abdurachmanov@sifive.com> Cc: keescook@chromium.org Cc: me@carlosedp.com Tested-by: Carlos de Paula <me@carlosedp.com> Reviewed-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/linux-riscv/CAEn-LTp=ss0Dfv6J00=rCAy+N78U2AmhqJNjfqjr2FDpPYjxEQ@mail.gmail.com/ Link: https://lore.kernel.org/linux-riscv/CAJr-aD=UnCN9E_mdVJ2H5nt=6juRSWikZnA5HxDLQxXLbsRz-w@mail.gmail.com/ [paul.walmsley@sifive.com: cleaned up Cc: lines; fixed spelling and checkpatch issues; updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-04 17:12:22 -07:00
/*
* Syscall number held in a7.
* If syscall number is above allowed value, redirect to ni_syscall.
*/
riscv: return -ENOSYS for syscall -1 Properly return -ENOSYS for syscall -1 instead of leaving the return value uninitialized. This fixes the strace teststuite. Fixes: 5340627e3fe0 ("riscv: add support for SECCOMP and SECCOMP_FILTER") Cc: stable@vger.kernel.org Signed-off-by: Andreas Schwab <schwab@suse.de> Reviewed-by: Tycho Andersen <tycho@tycho.pizza> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-21 23:52:00 +01:00
bgeu a7, t0, 1f
riscv: add support for SECCOMP and SECCOMP_FILTER This patch was extensively tested on Fedora/RISCV (applied by default on top of 5.2-rc7 kernel for <2 months). The patch was also tested with 5.3-rc on QEMU and SiFive Unleashed board. libseccomp (userspace) was rebased: https://github.com/seccomp/libseccomp/pull/134 Fully passes libseccomp regression testing (simulation and live). There is one failing kernel selftest: global.user_notification_signal v1 -> v2: - return immediately if secure_computing(NULL) returns -1 - fixed whitespace issues - add missing seccomp.h - remove patch #2 (solved now) - add riscv to seccomp kernel selftest Signed-off-by: David Abdurachmanov <david.abdurachmanov@sifive.com> Cc: keescook@chromium.org Cc: me@carlosedp.com Tested-by: Carlos de Paula <me@carlosedp.com> Reviewed-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/linux-riscv/CAEn-LTp=ss0Dfv6J00=rCAy+N78U2AmhqJNjfqjr2FDpPYjxEQ@mail.gmail.com/ Link: https://lore.kernel.org/linux-riscv/CAJr-aD=UnCN9E_mdVJ2H5nt=6juRSWikZnA5HxDLQxXLbsRz-w@mail.gmail.com/ [paul.walmsley@sifive.com: cleaned up Cc: lines; fixed spelling and checkpatch issues; updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-04 17:12:22 -07:00
/* Call syscall */
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
la s0, sys_call_table
slli t0, a7, RISCV_LGPTR
add s0, s0, t0
REG_L s0, 0(s0)
1:
jalr s0
ret_from_syscall:
/* Set user a0 to kernel a0 */
REG_S a0, PT_A0(sp)
riscv: add support for SECCOMP and SECCOMP_FILTER This patch was extensively tested on Fedora/RISCV (applied by default on top of 5.2-rc7 kernel for <2 months). The patch was also tested with 5.3-rc on QEMU and SiFive Unleashed board. libseccomp (userspace) was rebased: https://github.com/seccomp/libseccomp/pull/134 Fully passes libseccomp regression testing (simulation and live). There is one failing kernel selftest: global.user_notification_signal v1 -> v2: - return immediately if secure_computing(NULL) returns -1 - fixed whitespace issues - add missing seccomp.h - remove patch #2 (solved now) - add riscv to seccomp kernel selftest Signed-off-by: David Abdurachmanov <david.abdurachmanov@sifive.com> Cc: keescook@chromium.org Cc: me@carlosedp.com Tested-by: Carlos de Paula <me@carlosedp.com> Reviewed-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/linux-riscv/CAEn-LTp=ss0Dfv6J00=rCAy+N78U2AmhqJNjfqjr2FDpPYjxEQ@mail.gmail.com/ Link: https://lore.kernel.org/linux-riscv/CAJr-aD=UnCN9E_mdVJ2H5nt=6juRSWikZnA5HxDLQxXLbsRz-w@mail.gmail.com/ [paul.walmsley@sifive.com: cleaned up Cc: lines; fixed spelling and checkpatch issues; updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-04 17:12:22 -07:00
/*
* We didn't execute the actual syscall.
* Seccomp already set return value for the current task pt_regs.
* (If it was configured with SECCOMP_RET_ERRNO/TRACE)
*/
ret_from_syscall_rejected:
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
/* Trace syscalls, but only if requested by the user. */
REG_L t0, TASK_TI_FLAGS(tp)
riscv: add audit support On RISC-V (riscv) audit is supported through generic lib/audit.c. The patch adds required arch specific definitions. Signed-off-by: David Abdurachmanov <david.abdurachmanov@gmail.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-10-29 11:48:53 +01:00
andi t0, t0, _TIF_SYSCALL_WORK
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
bnez t0, handle_syscall_trace_exit
ret_from_exception:
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
REG_L s0, PT_STATUS(sp)
csrc CSR_STATUS, SR_IE
riscv: Enable LOCKDEP_SUPPORT & fixup TRACE_IRQFLAGS_SUPPORT Lockdep is needed by proving the spinlocks and rwlocks. To suupport it, we need fixup TRACE_IRQFLAGS_SUPPORT in kernel/entry.S. This patch follow Documentation/irqflags-tracing.txt. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-27 13:57:08 +00:00
#ifdef CONFIG_TRACE_IRQFLAGS
call trace_hardirqs_off
#endif
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
#ifdef CONFIG_RISCV_M_MODE
/* the MPP value is too large to be used as an immediate arg for addi */
li t0, SR_MPP
and s0, s0, t0
#else
riscv: rename SR_* constants to match the spec Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-01-04 18:35:03 +01:00
andi s0, s0, SR_SPP
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
#endif
RISC-V: Add _TIF_NEED_RESCHED check for kernel thread when CONFIG_PREEMPT=y The cond_resched() can be used to yield the CPU resource if CONFIG_PREEMPT is not defined. Otherwise, cond_resched() is a dummy function. In order to avoid kernel thread occupying entire CPU, when CONFIG_PREEMPT=y, the kernel thread needs to follow the rescheduling mechanism like a user thread. Signed-off-by: Vincent Chen <vincentc@andestech.com> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-01-03 11:32:33 +08:00
bnez s0, resume_kernel
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
resume_userspace:
/* Interrupts must be disabled here so flags are checked atomically */
REG_L s0, TASK_TI_FLAGS(tp) /* current_thread_info->flags */
andi s1, s0, _TIF_WORK_MASK
bnez s1, work_pending
riscv: Enable context tracking This patch implements and enables context tracking for riscv (which is a prerequisite for CONFIG_NO_HZ_FULL support) It adds checking for previous state in the entry that all excepttions and interrupts goes to and calls context_tracking_user_exit() if it comes from user space. It also calls context_tracking_user_enter() if it will return to user space before restore_all. This patch is tested with the dynticks-testing testcase in qemu-system-riscv64 virt machine and Unleashed board. git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git We can see the log here. The tick got mostly stopped during the execution of the user loop. _-----=> irqs-off / _----=> need-resched | / _---=> hardirq/softirq || / _--=> preempt-depth ||| / delay TASK-PID CPU# |||| TIMESTAMP FUNCTION | | | |||| | | <idle>-0 [001] d..2 604.183512: sched_switch: prev_comm=swapper/1 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=taskset next_pid=273 next_prio=120 user_loop-273 [001] d.h1 604.184788: hrtimer_expire_entry: hrtimer=000000002eda5fab function=tick_sched_timer now=604176096300 user_loop-273 [001] d.s2 604.184897: workqueue_queue_work: work struct=00000000383402c2 function=vmstat_update workqueue=00000000f36d35d4 req_cpu=1 cpu=1 user_loop-273 [001] dns2 604.185039: tick_stop: success=0 dependency=SCHED user_loop-273 [001] dn.1 604.185103: tick_stop: success=0 dependency=SCHED user_loop-273 [001] d..2 604.185154: sched_switch: prev_comm=taskset prev_pid=273 prev_prio=120 prev_state=R+ ==> next_comm=kworker/1:1 next_pid=46 next_prio=120 <...>-46 [001] .... 604.185194: workqueue_execute_start: work struct 00000000383402c2: function vmstat_update <...>-46 [001] d..2 604.185266: sched_switch: prev_comm=kworker/1:1 prev_pid=46 prev_prio=120 prev_state=I ==> next_comm=taskset next_pid=273 next_prio=120 user_loop-273 [001] d.h1 604.188812: hrtimer_expire_entry: hrtimer=000000002eda5fab function=tick_sched_timer now=604180133400 user_loop-273 [001] d..1 604.189050: tick_stop: success=1 dependency=NONE user_loop-273 [001] d..2 614.251386: sched_switch: prev_comm=user_loop prev_pid=273 prev_prio=120 prev_state=X ==> next_comm=swapper/1 next_pid=0 next_prio=120 <idle>-0 [001] d..2 614.315391: sched_switch: prev_comm=swapper/1 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=taskset next_pid=276 next_prio=120 Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-24 17:03:16 +08:00
#ifdef CONFIG_CONTEXT_TRACKING
call context_tracking_user_enter
#endif
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
/* Save unwound kernel stack pointer in thread_info */
addi s0, sp, PT_SIZE_ON_STACK
REG_S s0, TASK_TI_KERNEL_SP(tp)
/*
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
* Save TP into the scratch register , so we can find the kernel data
* structures again.
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
*/
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
csrw CSR_SCRATCH, tp
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
restore_all:
riscv: Enable LOCKDEP_SUPPORT & fixup TRACE_IRQFLAGS_SUPPORT Lockdep is needed by proving the spinlocks and rwlocks. To suupport it, we need fixup TRACE_IRQFLAGS_SUPPORT in kernel/entry.S. This patch follow Documentation/irqflags-tracing.txt. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-27 13:57:08 +00:00
#ifdef CONFIG_TRACE_IRQFLAGS
REG_L s1, PT_STATUS(sp)
andi t0, s1, SR_PIE
beqz t0, 1f
call trace_hardirqs_on
j 2f
1:
call trace_hardirqs_off
2:
#endif
RISC-V: Inline the assembly register save/restore macros These are only used once, and when reading the code I've always found them to be more of a headache than a benefit. While they were never worth removing before, LLVM's integrated assembler doesn't support LOCAL so rather that trying to figure out how to refactor the macros it seems saner to just inline them. Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-02-27 11:15:03 -08:00
REG_L a0, PT_STATUS(sp)
/*
* The current load reservation is effectively part of the processor's
* state, in the sense that load reservations cannot be shared between
* different hart contexts. We can't actually save and restore a load
* reservation, so instead here we clear any existing reservation --
* it's always legal for implementations to clear load reservations at
* any point (as long as the forward progress guarantee is kept, but
* we'll ignore that here).
*
* Dangling load reservations can be the result of taking a trap in the
* middle of an LR/SC sequence, but can also be the result of a taken
* forward branch around an SC -- which is how we implement CAS. As a
* result we need to clear reservations between the last CAS and the
* jump back to the new context. While it is unlikely the store
* completes, implementations are allowed to expand reservations to be
* arbitrarily large.
*/
REG_L a2, PT_EPC(sp)
REG_SC x0, a2, PT_EPC(sp)
csrw CSR_STATUS, a0
csrw CSR_EPC, a2
REG_L x1, PT_RA(sp)
REG_L x3, PT_GP(sp)
REG_L x4, PT_TP(sp)
REG_L x5, PT_T0(sp)
REG_L x6, PT_T1(sp)
REG_L x7, PT_T2(sp)
REG_L x8, PT_S0(sp)
REG_L x9, PT_S1(sp)
REG_L x10, PT_A0(sp)
REG_L x11, PT_A1(sp)
REG_L x12, PT_A2(sp)
REG_L x13, PT_A3(sp)
REG_L x14, PT_A4(sp)
REG_L x15, PT_A5(sp)
REG_L x16, PT_A6(sp)
REG_L x17, PT_A7(sp)
REG_L x18, PT_S2(sp)
REG_L x19, PT_S3(sp)
REG_L x20, PT_S4(sp)
REG_L x21, PT_S5(sp)
REG_L x22, PT_S6(sp)
REG_L x23, PT_S7(sp)
REG_L x24, PT_S8(sp)
REG_L x25, PT_S9(sp)
REG_L x26, PT_S10(sp)
REG_L x27, PT_S11(sp)
REG_L x28, PT_T3(sp)
REG_L x29, PT_T4(sp)
REG_L x30, PT_T5(sp)
REG_L x31, PT_T6(sp)
REG_L x2, PT_SP(sp)
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
#ifdef CONFIG_RISCV_M_MODE
mret
#else
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
sret
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
#endif
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
sched/rt, riscv: Use CONFIG_PREEMPTION CONFIG_PREEMPTION is selected by CONFIG_PREEMPT and by CONFIG_PREEMPT_RT. Both PREEMPT and PREEMPT_RT require the same functionality which today depends on CONFIG_PREEMPT. Switch the entry code over to use CONFIG_PREEMPTION. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Acked-by: Paul Walmsley <paul.walmsley@sifive.com> # for arch/riscv Cc: Albert Ou <aou@eecs.berkeley.edu> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Palmer Dabbelt <palmer@sifive.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: linux-riscv@lists.infradead.org Link: https://lore.kernel.org/r/20191015191821.11479-17-bigeasy@linutronix.de Signed-off-by: Ingo Molnar <mingo@kernel.org>
2019-10-15 21:18:03 +02:00
#if IS_ENABLED(CONFIG_PREEMPTION)
RISC-V: Add _TIF_NEED_RESCHED check for kernel thread when CONFIG_PREEMPT=y The cond_resched() can be used to yield the CPU resource if CONFIG_PREEMPT is not defined. Otherwise, cond_resched() is a dummy function. In order to avoid kernel thread occupying entire CPU, when CONFIG_PREEMPT=y, the kernel thread needs to follow the rescheduling mechanism like a user thread. Signed-off-by: Vincent Chen <vincentc@andestech.com> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-01-03 11:32:33 +08:00
resume_kernel:
REG_L s0, TASK_TI_PREEMPT_COUNT(tp)
bnez s0, restore_all
REG_L s0, TASK_TI_FLAGS(tp)
andi s0, s0, _TIF_NEED_RESCHED
beqz s0, restore_all
call preempt_schedule_irq
RISC-V: entry: Remove unneeded need_resched() loop Since the enabling and disabling of IRQs within preempt_schedule_irq() is contained in a need_resched() loop, we don't need the outer arch code loop. Reviewed-by: Palmer Dabbelt <palmer@sifive.com> Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> Cc: Albert Ou <aou@eecs.berkeley.edu> Cc: linux-riscv@lists.infradead.org Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-09-23 15:36:17 +01:00
j restore_all
RISC-V: Add _TIF_NEED_RESCHED check for kernel thread when CONFIG_PREEMPT=y The cond_resched() can be used to yield the CPU resource if CONFIG_PREEMPT is not defined. Otherwise, cond_resched() is a dummy function. In order to avoid kernel thread occupying entire CPU, when CONFIG_PREEMPT=y, the kernel thread needs to follow the rescheduling mechanism like a user thread. Signed-off-by: Vincent Chen <vincentc@andestech.com> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-01-03 11:32:33 +08:00
#endif
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
work_pending:
/* Enter slow path for supplementary processing */
la ra, ret_from_exception
andi s1, s0, _TIF_NEED_RESCHED
bnez s1, work_resched
work_notifysig:
/* Handle pending signals and notify-resume requests */
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
csrs CSR_STATUS, SR_IE /* Enable interrupts for do_notify_resume() */
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
move a0, sp /* pt_regs */
move a1, s0 /* current_thread_info->flags */
tail do_notify_resume
work_resched:
tail schedule
/* Slow paths for ptrace. */
handle_syscall_trace_enter:
move a0, sp
call do_syscall_trace_enter
riscv: fix seccomp reject syscall code path If secure_computing() rejected a system call, we were previously setting the system call number to -1, to indicate to later code that the syscall failed. However, if something (e.g. a user notification) was sleeping, and received a signal, we may set a0 to -ERESTARTSYS and re-try the system call again. In this case, seccomp "denies" the syscall (because of the signal), and we would set a7 to -1, thus losing the value of the system call we want to restart. Instead, let's return -1 from do_syscall_trace_enter() to indicate that the syscall was rejected, so we don't clobber the value in case of -ERESTARTSYS or whatever. This commit fixes the user_notification_signal seccomp selftest on riscv to no longer hang. That test expects the system call to be re-issued after the signal, and it wasn't due to the above bug. Now that it is, everything works normally. Note that in the ptrace (tracer) case, the tracer can set the register values to whatever they want, so we still need to keep the code that handles out-of-bounds syscalls. However, we can drop the comment. We can also drop syscall_set_nr(), since it is no longer used anywhere, and the code that re-loads the value in a7 because of it. Reported in: https://lore.kernel.org/bpf/CAEn-LTp=ss0Dfv6J00=rCAy+N78U2AmhqJNjfqjr2FDpPYjxEQ@mail.gmail.com/ Reported-by: David Abdurachmanov <david.abdurachmanov@gmail.com> Signed-off-by: Tycho Andersen <tycho@tycho.ws> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-02-08 08:18:17 -07:00
move t0, a0
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
REG_L a0, PT_A0(sp)
REG_L a1, PT_A1(sp)
REG_L a2, PT_A2(sp)
REG_L a3, PT_A3(sp)
REG_L a4, PT_A4(sp)
REG_L a5, PT_A5(sp)
REG_L a6, PT_A6(sp)
REG_L a7, PT_A7(sp)
riscv: fix seccomp reject syscall code path If secure_computing() rejected a system call, we were previously setting the system call number to -1, to indicate to later code that the syscall failed. However, if something (e.g. a user notification) was sleeping, and received a signal, we may set a0 to -ERESTARTSYS and re-try the system call again. In this case, seccomp "denies" the syscall (because of the signal), and we would set a7 to -1, thus losing the value of the system call we want to restart. Instead, let's return -1 from do_syscall_trace_enter() to indicate that the syscall was rejected, so we don't clobber the value in case of -ERESTARTSYS or whatever. This commit fixes the user_notification_signal seccomp selftest on riscv to no longer hang. That test expects the system call to be re-issued after the signal, and it wasn't due to the above bug. Now that it is, everything works normally. Note that in the ptrace (tracer) case, the tracer can set the register values to whatever they want, so we still need to keep the code that handles out-of-bounds syscalls. However, we can drop the comment. We can also drop syscall_set_nr(), since it is no longer used anywhere, and the code that re-loads the value in a7 because of it. Reported in: https://lore.kernel.org/bpf/CAEn-LTp=ss0Dfv6J00=rCAy+N78U2AmhqJNjfqjr2FDpPYjxEQ@mail.gmail.com/ Reported-by: David Abdurachmanov <david.abdurachmanov@gmail.com> Signed-off-by: Tycho Andersen <tycho@tycho.ws> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-02-08 08:18:17 -07:00
bnez t0, ret_from_syscall_rejected
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
j check_syscall_nr
handle_syscall_trace_exit:
move a0, sp
call do_syscall_trace_exit
j ret_from_exception
riscv: add VMAP_STACK overflow detection This patch adds stack overflow detection to riscv, usable when CONFIG_VMAP_STACK=y. Overflow is detected in kernel exception entry(kernel/entry.S), if the kernel stack is overflow and been detected, the overflow handler is invoked on a per-cpu overflow stack. This approach preserves GPRs and the original exception information. The overflow detect is performed before any attempt is made to access the stack and the principle of stack overflow detection: kernel stacks are aligned to double their size, enabling overflow to be detected with a single bit test. For example, a 16K stack is aligned to 32K, ensuring that bit 14 of the SP must be zero. On an overflow (or underflow), this bit is flipped. Thus, overflow (of less than the size of the stack) can be detected by testing whether this bit is set. This gives us a useful error message on stack overflow, as can be trigger with the LKDTM overflow test: [ 388.053267] lkdtm: Performing direct entry EXHAUST_STACK [ 388.053663] lkdtm: Calling function with 1024 frame size to depth 32 ... [ 388.054016] lkdtm: loop 32/32 ... [ 388.054186] lkdtm: loop 31/32 ... [ 388.054491] lkdtm: loop 30/32 ... [ 388.054672] lkdtm: loop 29/32 ... [ 388.054859] lkdtm: loop 28/32 ... [ 388.055010] lkdtm: loop 27/32 ... [ 388.055163] lkdtm: loop 26/32 ... [ 388.055309] lkdtm: loop 25/32 ... [ 388.055481] lkdtm: loop 24/32 ... [ 388.055653] lkdtm: loop 23/32 ... [ 388.055837] lkdtm: loop 22/32 ... [ 388.056015] lkdtm: loop 21/32 ... [ 388.056188] lkdtm: loop 20/32 ... [ 388.058145] Insufficient stack space to handle exception! [ 388.058153] Task stack: [0xffffffd014260000..0xffffffd014264000] [ 388.058160] Overflow stack: [0xffffffe1f8d2c220..0xffffffe1f8d2d220] [ 388.058168] CPU: 0 PID: 89 Comm: bash Not tainted 5.12.0-rc8-dirty #90 [ 388.058175] Hardware name: riscv-virtio,qemu (DT) [ 388.058187] epc : number+0x32/0x2c0 [ 388.058247] ra : vsnprintf+0x2ae/0x3f0 [ 388.058255] epc : ffffffe0002d38f6 ra : ffffffe0002d814e sp : ffffffd01425ffc0 [ 388.058263] gp : ffffffe0012e4010 tp : ffffffe08014da00 t0 : ffffffd0142606e8 [ 388.058271] t1 : 0000000000000000 t2 : 0000000000000000 s0 : ffffffd014260070 [ 388.058303] s1 : ffffffd014260158 a0 : ffffffd01426015e a1 : ffffffd014260158 [ 388.058311] a2 : 0000000000000013 a3 : ffff0a01ffffff10 a4 : ffffffe000c398e0 [ 388.058319] a5 : 511b02ec65f3e300 a6 : 0000000000a1749a a7 : 0000000000000000 [ 388.058327] s2 : ffffffff000000ff s3 : 00000000ffff0a01 s4 : ffffffe0012e50a8 [ 388.058335] s5 : 0000000000ffff0a s6 : ffffffe0012e50a8 s7 : ffffffe000da1cc0 [ 388.058343] s8 : ffffffffffffffff s9 : ffffffd0142602b0 s10: ffffffd0142602a8 [ 388.058351] s11: ffffffd01426015e t3 : 00000000000f0000 t4 : ffffffffffffffff [ 388.058359] t5 : 000000000000002f t6 : ffffffd014260158 [ 388.058366] status: 0000000000000100 badaddr: ffffffd01425fff8 cause: 000000000000000f [ 388.058374] Kernel panic - not syncing: Kernel stack overflow [ 388.058381] CPU: 0 PID: 89 Comm: bash Not tainted 5.12.0-rc8-dirty #90 [ 388.058387] Hardware name: riscv-virtio,qemu (DT) [ 388.058393] Call Trace: [ 388.058400] [<ffffffe000004944>] walk_stackframe+0x0/0xce [ 388.058406] [<ffffffe0006f0b28>] dump_backtrace+0x38/0x46 [ 388.058412] [<ffffffe0006f0b46>] show_stack+0x10/0x18 [ 388.058418] [<ffffffe0006f3690>] dump_stack+0x74/0x8e [ 388.058424] [<ffffffe0006f0d52>] panic+0xfc/0x2b2 [ 388.058430] [<ffffffe0006f0acc>] print_trace_address+0x0/0x24 [ 388.058436] [<ffffffe0002d814e>] vsnprintf+0x2ae/0x3f0 [ 388.058956] SMP: stopping secondary CPUs Signed-off-by: Tong Tiangen <tongtiangen@huawei.com> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-06-21 11:28:55 +08:00
#ifdef CONFIG_VMAP_STACK
handle_kernel_stack_overflow:
la sp, shadow_stack
addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
//save caller register to shadow stack
addi sp, sp, -(PT_SIZE_ON_STACK)
REG_S x1, PT_RA(sp)
REG_S x5, PT_T0(sp)
REG_S x6, PT_T1(sp)
REG_S x7, PT_T2(sp)
REG_S x10, PT_A0(sp)
REG_S x11, PT_A1(sp)
REG_S x12, PT_A2(sp)
REG_S x13, PT_A3(sp)
REG_S x14, PT_A4(sp)
REG_S x15, PT_A5(sp)
REG_S x16, PT_A6(sp)
REG_S x17, PT_A7(sp)
REG_S x28, PT_T3(sp)
REG_S x29, PT_T4(sp)
REG_S x30, PT_T5(sp)
REG_S x31, PT_T6(sp)
la ra, restore_caller_reg
tail get_overflow_stack
restore_caller_reg:
//save per-cpu overflow stack
REG_S a0, -8(sp)
//restore caller register from shadow_stack
REG_L x1, PT_RA(sp)
REG_L x5, PT_T0(sp)
REG_L x6, PT_T1(sp)
REG_L x7, PT_T2(sp)
REG_L x10, PT_A0(sp)
REG_L x11, PT_A1(sp)
REG_L x12, PT_A2(sp)
REG_L x13, PT_A3(sp)
REG_L x14, PT_A4(sp)
REG_L x15, PT_A5(sp)
REG_L x16, PT_A6(sp)
REG_L x17, PT_A7(sp)
REG_L x28, PT_T3(sp)
REG_L x29, PT_T4(sp)
REG_L x30, PT_T5(sp)
REG_L x31, PT_T6(sp)
//load per-cpu overflow stack
REG_L sp, -8(sp)
addi sp, sp, -(PT_SIZE_ON_STACK)
//save context to overflow stack
REG_S x1, PT_RA(sp)
REG_S x3, PT_GP(sp)
REG_S x5, PT_T0(sp)
REG_S x6, PT_T1(sp)
REG_S x7, PT_T2(sp)
REG_S x8, PT_S0(sp)
REG_S x9, PT_S1(sp)
REG_S x10, PT_A0(sp)
REG_S x11, PT_A1(sp)
REG_S x12, PT_A2(sp)
REG_S x13, PT_A3(sp)
REG_S x14, PT_A4(sp)
REG_S x15, PT_A5(sp)
REG_S x16, PT_A6(sp)
REG_S x17, PT_A7(sp)
REG_S x18, PT_S2(sp)
REG_S x19, PT_S3(sp)
REG_S x20, PT_S4(sp)
REG_S x21, PT_S5(sp)
REG_S x22, PT_S6(sp)
REG_S x23, PT_S7(sp)
REG_S x24, PT_S8(sp)
REG_S x25, PT_S9(sp)
REG_S x26, PT_S10(sp)
REG_S x27, PT_S11(sp)
REG_S x28, PT_T3(sp)
REG_S x29, PT_T4(sp)
REG_S x30, PT_T5(sp)
REG_S x31, PT_T6(sp)
REG_L s0, TASK_TI_KERNEL_SP(tp)
csrr s1, CSR_STATUS
csrr s2, CSR_EPC
csrr s3, CSR_TVAL
csrr s4, CSR_CAUSE
csrr s5, CSR_SCRATCH
REG_S s0, PT_SP(sp)
REG_S s1, PT_STATUS(sp)
REG_S s2, PT_EPC(sp)
REG_S s3, PT_BADADDR(sp)
REG_S s4, PT_CAUSE(sp)
REG_S s5, PT_TP(sp)
move a0, sp
tail handle_bad_stack
#endif
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
END(handle_exception)
ENTRY(ret_from_fork)
la ra, ret_from_exception
tail schedule_tail
ENDPROC(ret_from_fork)
ENTRY(ret_from_kernel_thread)
call schedule_tail
/* Call fn(arg) */
la ra, ret_from_exception
move a0, s1
jr s0
ENDPROC(ret_from_kernel_thread)
/*
* Integer register context switch
* The callee-saved registers must be saved and restored.
*
* a0: previous task_struct (must be preserved across the switch)
* a1: next task_struct
*
* The value of a0 and a1 must be preserved by this function, as that's how
* arguments are passed to schedule_tail.
*/
ENTRY(__switch_to)
/* Save context into prev->thread */
li a4, TASK_THREAD_RA
add a3, a0, a4
add a4, a1, a4
REG_S ra, TASK_THREAD_RA_RA(a3)
REG_S sp, TASK_THREAD_SP_RA(a3)
REG_S s0, TASK_THREAD_S0_RA(a3)
REG_S s1, TASK_THREAD_S1_RA(a3)
REG_S s2, TASK_THREAD_S2_RA(a3)
REG_S s3, TASK_THREAD_S3_RA(a3)
REG_S s4, TASK_THREAD_S4_RA(a3)
REG_S s5, TASK_THREAD_S5_RA(a3)
REG_S s6, TASK_THREAD_S6_RA(a3)
REG_S s7, TASK_THREAD_S7_RA(a3)
REG_S s8, TASK_THREAD_S8_RA(a3)
REG_S s9, TASK_THREAD_S9_RA(a3)
REG_S s10, TASK_THREAD_S10_RA(a3)
REG_S s11, TASK_THREAD_S11_RA(a3)
/* Restore context from next->thread */
REG_L ra, TASK_THREAD_RA_RA(a4)
REG_L sp, TASK_THREAD_SP_RA(a4)
REG_L s0, TASK_THREAD_S0_RA(a4)
REG_L s1, TASK_THREAD_S1_RA(a4)
REG_L s2, TASK_THREAD_S2_RA(a4)
REG_L s3, TASK_THREAD_S3_RA(a4)
REG_L s4, TASK_THREAD_S4_RA(a4)
REG_L s5, TASK_THREAD_S5_RA(a4)
REG_L s6, TASK_THREAD_S6_RA(a4)
REG_L s7, TASK_THREAD_S7_RA(a4)
REG_L s8, TASK_THREAD_S8_RA(a4)
REG_L s9, TASK_THREAD_S9_RA(a4)
REG_L s10, TASK_THREAD_S10_RA(a4)
REG_L s11, TASK_THREAD_S11_RA(a4)
/* Swap the CPU entry around. */
lw a3, TASK_TI_CPU(a0)
lw a4, TASK_TI_CPU(a1)
sw a3, TASK_TI_CPU(a1)
sw a4, TASK_TI_CPU(a0)
riscv: Cleanup unnecessary define in asm-offset.c - TASK_THREAD_SP is duplicated define - TASK_STACK is no use at all - Don't worry about thread_info's offset in task_struct, have a look on comment in include/linux/sched.h: struct task_struct { /* * For reasons of header soup (see current_thread_info()), this * must be the first element of task_struct. */ struct thread_info thread_info; Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-07-12 13:41:49 +00:00
/* The offset of thread_info in task_struct is zero. */
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
move tp, a1
ret
ENDPROC(__switch_to)
riscv: add nommu support The kernel runs in M-mode without using page tables, and thus can't run bare metal without help from additional firmware. Most of the patch is just stubbing out code not needed without page tables, but there is an interesting detail in the signals implementation: - The normal RISC-V syscall ABI only implements rt_sigreturn as VDSO entry point, but the ELF VDSO is not supported for nommu Linux. We instead copy the code to call the syscall onto the stack. In addition to enabling the nommu code a new defconfig for a small kernel image that can run in nommu mode on qemu is also provided, to run a kernel in qemu you can use the following command line: qemu-system-riscv64 -smp 2 -m 64 -machine virt -nographic \ -kernel arch/riscv/boot/loader \ -drive file=rootfs.ext2,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0 Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> [paul.walmsley@sifive.com: updated to apply; add CONFIG_MMU guards around PCI_IOBASE definition to fix build issues; fixed checkpatch issues; move the PCI_IO_* and VMEMMAP address space macros along with the others; resolve sparse warning] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:41 +01:00
#ifndef CONFIG_MMU
#define do_page_fault do_trap_unknown
#endif
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
.section ".rodata"
riscv,entry: fix misaligned base for excp_vect_table In RV64, the size of each entry in excp_vect_table is 8 bytes. If the base of the table is not 8-byte aligned, loading an entry in the table will raise a misaligned exception. Although such exception will be handled by opensbi/bbl, this still causes performance degradation. Signed-off-by: Zihao Yu <yuzihao@ict.ac.cn> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-17 16:17:25 +08:00
.align LGREG
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
/* Exception vector table */
ENTRY(excp_vect_table)
RISCV_PTR do_trap_insn_misaligned
riscv: sifive: Apply errata "cip-453" patch Add sign extension to the $badaddr before addressing the instruction page fault and instruction access fault to workaround the issue "cip-453". To avoid affecting the existing code sequence, this patch will creates two trampolines to add sign extension to the $badaddr. By the "alternative" mechanism, these two trampolines will replace the original exception handler of instruction page fault and instruction access fault in the excp_vect_table. In this case, only the specific SiFive CPU core jumps to the do_page_fault and do_trap_insn_fault through these two trampolines. Other CPUs are not affected. Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:05 +08:00
ALT_INSN_FAULT(RISCV_PTR do_trap_insn_fault)
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
RISCV_PTR do_trap_insn_illegal
RISCV_PTR do_trap_break
RISCV_PTR do_trap_load_misaligned
RISCV_PTR do_trap_load_fault
RISCV_PTR do_trap_store_misaligned
RISCV_PTR do_trap_store_fault
RISCV_PTR do_trap_ecall_u /* system call, gets intercepted */
RISCV_PTR do_trap_ecall_s
RISCV_PTR do_trap_unknown
RISCV_PTR do_trap_ecall_m
riscv: sifive: Apply errata "cip-453" patch Add sign extension to the $badaddr before addressing the instruction page fault and instruction access fault to workaround the issue "cip-453". To avoid affecting the existing code sequence, this patch will creates two trampolines to add sign extension to the $badaddr. By the "alternative" mechanism, these two trampolines will replace the original exception handler of instruction page fault and instruction access fault in the excp_vect_table. In this case, only the specific SiFive CPU core jumps to the do_page_fault and do_trap_insn_fault through these two trampolines. Other CPUs are not affected. Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:05 +08:00
/* instruciton page fault */
ALT_PAGE_FAULT(RISCV_PTR do_page_fault)
RISC-V: Task implementation This patch contains the implementation of tasks on RISC-V, most of which is involved in task switching. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:04:30 -07:00
RISCV_PTR do_page_fault /* load page fault */
RISCV_PTR do_trap_unknown
RISCV_PTR do_page_fault /* store page fault */
excp_vect_table_end:
END(excp_vect_table)
riscv: add nommu support The kernel runs in M-mode without using page tables, and thus can't run bare metal without help from additional firmware. Most of the patch is just stubbing out code not needed without page tables, but there is an interesting detail in the signals implementation: - The normal RISC-V syscall ABI only implements rt_sigreturn as VDSO entry point, but the ELF VDSO is not supported for nommu Linux. We instead copy the code to call the syscall onto the stack. In addition to enabling the nommu code a new defconfig for a small kernel image that can run in nommu mode on qemu is also provided, to run a kernel in qemu you can use the following command line: qemu-system-riscv64 -smp 2 -m 64 -machine virt -nographic \ -kernel arch/riscv/boot/loader \ -drive file=rootfs.ext2,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0 Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> [paul.walmsley@sifive.com: updated to apply; add CONFIG_MMU guards around PCI_IOBASE definition to fix build issues; fixed checkpatch issues; move the PCI_IO_* and VMEMMAP address space macros along with the others; resolve sparse warning] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:41 +01:00
#ifndef CONFIG_MMU
ENTRY(__user_rt_sigreturn)
li a7, __NR_rt_sigreturn
scall
END(__user_rt_sigreturn)
#endif
Reference in New Issue Copy Permalink