iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
linux/arch/riscv/kernel/head.S

456 lines
9.4 KiB
ArmAsm
Raw Normal View History

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 286 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license as published by the free software foundation version 2 this program is distributed in the hope that it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 97 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Reviewed-by: Alexios Zavras <alexios.zavras@intel.com> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190529141901.025053186@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-29 07:18:00 -07:00
/* SPDX-License-Identifier: GPL-2.0-only */
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
/*
* Copyright (C) 2012 Regents of the University of California
*/
#include <asm/asm-offsets.h>
#include <asm/asm.h>
#include <linux/init.h>
#include <linux/linkage.h>
#include <asm/thread_info.h>
#include <asm/page.h>
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
#include <asm/pgtable.h>
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
#include <asm/csr.h>
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:40 +01:00
#include <asm/hwcap.h>
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 16:08:00 -07:00
#include <asm/image.h>
riscv: Implement Shadow Call Stack Implement CONFIG_SHADOW_CALL_STACK for RISC-V. When enabled, the compiler injects instructions to all non-leaf C functions to store the return address to the shadow stack and unconditionally load it again before returning, which makes it harder to corrupt the return address through a stack overflow, for example. The active shadow call stack pointer is stored in the gp register, which makes SCS incompatible with gp relaxation. Use --no-relax-gp to ensure gp relaxation is disabled and disable global pointer loading. Add SCS pointers to struct thread_info, implement SCS initialization, and task switching Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Tested-by: Nathan Chancellor <nathan@kernel.org> Link: https://lore.kernel.org/r/20230927224757.1154247-12-samitolvanen@google.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-27 22:48:02 +00:00
#include <asm/scs.h>
RISC-V: Split out the XIP fixups into their own file This was broken by the original refactoring (as the XIP definitions depend on <asm/pgtable.h>) and then more broken by the merge (as I accidentally took the old version). This fixes both breakages, while also pulling this out of <asm/asm.h> to avoid polluting most assembly files with the XIP fixups. Fixes: bee7fbc38579 ("RISC-V CPU Idle Support") Fixes: 63b13e64a829 ("RISC-V: Add arch functions for non-retentive suspend entry/exit") Link: https://lore.kernel.org/r/20220420184056.7886-4-palmer@rivosinc.com Reviewed-by: Guo Ren <guoren@kernel.org> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-04-19 20:02:16 -07:00
#include <asm/xip_fixup.h>
RISC-V: Add PE/COFF header for EFI stub Linux kernel Image can appear as an EFI application With appropriate PE/COFF header fields in the beginning of the Image header. An EFI application loader can directly load a Linux kernel Image and an EFI stub residing in kernel can boot Linux kernel directly. Add the necessary PE/COFF header. Signed-off-by: Atish Patra <atish.patra@wdc.com> Link: https://lore.kernel.org/r/20200421033336.9663-3-atish.patra@wdc.com [ardb: - use C prefix for c.li to ensure the expected opcode is emitted - align all image sections according to PE/COFF section alignment ] Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-09-17 15:37:13 -07:00
#include "efi-header.S"
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
__HEAD
riscv: Use SYM_*() assembly macros instead of deprecated ones ENTRY()/END()/WEAK() macros are deprecated and we should make use of the new SYM_*() macros [1] for better annotation of symbols. Replace the deprecated ones with the new ones and fix wrong usage of END()/ENDPROC() to correctly describe the symbols. [1] https://docs.kernel.org/core-api/asm-annotations.html Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-3-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:52 +02:00
SYM_CODE_START(_start)
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 16:08:00 -07:00
/*
* Image header expected by Linux boot-loaders. The image header data
* structure is described in asm/image.h.
* Do not modify it without modifying the structure and all bootloaders
* that expects this header format!!
*/
RISC-V: Add PE/COFF header for EFI stub Linux kernel Image can appear as an EFI application With appropriate PE/COFF header fields in the beginning of the Image header. An EFI application loader can directly load a Linux kernel Image and an EFI stub residing in kernel can boot Linux kernel directly. Add the necessary PE/COFF header. Signed-off-by: Atish Patra <atish.patra@wdc.com> Link: https://lore.kernel.org/r/20200421033336.9663-3-atish.patra@wdc.com [ardb: - use C prefix for c.li to ensure the expected opcode is emitted - align all image sections according to PE/COFF section alignment ] Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-09-17 15:37:13 -07:00
#ifdef CONFIG_EFI
/*
* This instruction decodes to "MZ" ASCII required by UEFI.
*/
c.li s4,-13
j _start_kernel
#else
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 16:08:00 -07:00
/* jump to start kernel */
j _start_kernel
/* reserved */
.word 0
RISC-V: Add PE/COFF header for EFI stub Linux kernel Image can appear as an EFI application With appropriate PE/COFF header fields in the beginning of the Image header. An EFI application loader can directly load a Linux kernel Image and an EFI stub residing in kernel can boot Linux kernel directly. Add the necessary PE/COFF header. Signed-off-by: Atish Patra <atish.patra@wdc.com> Link: https://lore.kernel.org/r/20200421033336.9663-3-atish.patra@wdc.com [ardb: - use C prefix for c.li to ensure the expected opcode is emitted - align all image sections according to PE/COFF section alignment ] Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-09-17 15:37:13 -07:00
#endif
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 16:08:00 -07:00
.balign 8
riscv: Set text_offset correctly for M-Mode M-Mode Linux is loaded at the start of RAM, not 2MB later. Perhaps this should be calculated based on PAGE_OFFSET somehow? Even better would be to deprecate text_offset and instead introduce something absolute. Signed-off-by: Sean Anderson <seanga2@gmail.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-10-22 16:30:12 -04:00
#ifdef CONFIG_RISCV_M_MODE
/* Image load offset (0MB) from start of RAM for M-mode */
.dword 0
#else
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 16:08:00 -07:00
#if __riscv_xlen == 64
/* Image load offset(2MB) from start of RAM */
.dword 0x200000
#else
/* Image load offset(4MB) from start of RAM */
.dword 0x400000
riscv: Set text_offset correctly for M-Mode M-Mode Linux is loaded at the start of RAM, not 2MB later. Perhaps this should be calculated based on PAGE_OFFSET somehow? Even better would be to deprecate text_offset and instead introduce something absolute. Signed-off-by: Sean Anderson <seanga2@gmail.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-10-22 16:30:12 -04:00
#endif
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 16:08:00 -07:00
#endif
/* Effective size of kernel image */
.dword _end - _start
.dword __HEAD_FLAGS
.word RISCV_HEADER_VERSION
.word 0
.dword 0
riscv: modify the Image header to improve compatibility with the ARM64 header Part of the intention during the definition of the RISC-V kernel image header was to lay the groundwork for a future merge with the ARM64 image header. One error during my original review was not noticing that the RISC-V header's "magic" field was at a different size and position than the ARM64's "magic" field. If the existing ARM64 Image header parsing code were to attempt to parse an existing RISC-V kernel image header format, it would see a magic number 0. This is undesirable, since it's our intention to align as closely as possible with the ARM64 header format. Another problem was that the original "res3" field was not being initialized correctly to zero. Address these issues by creating a 32-bit "magic2" field in the RISC-V header which matches the ARM64 "magic" field. RISC-V binaries will store "RSC\x05" in this field. The intention is that the use of the existing 64-bit "magic" field in the RISC-V header will be deprecated over time. Increment the minor version number of the file format to indicate this change, and update the documentation accordingly. Fix the assembler directives in head.S to ensure that reserved fields are properly zero-initialized. Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com> Reported-by: Palmer Dabbelt <palmer@sifive.com> Reviewed-by: Palmer Dabbelt <palmer@sifive.com> Cc: Atish Patra <atish.patra@wdc.com> Cc: Karsten Merker <merker@debian.org> Link: https://lore.kernel.org/linux-riscv/194c2f10c9806720623430dbf0cc59a965e50448.camel@wdc.com/T/#u Link: https://lore.kernel.org/linux-riscv/mhng-755b14c4-8f35-4079-a7ff-e421fd1b02bc@palmer-si-x1e/T/#t
2019-09-13 18:35:50 -07:00
.ascii RISCV_IMAGE_MAGIC
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 16:08:00 -07:00
.balign 4
riscv: modify the Image header to improve compatibility with the ARM64 header Part of the intention during the definition of the RISC-V kernel image header was to lay the groundwork for a future merge with the ARM64 image header. One error during my original review was not noticing that the RISC-V header's "magic" field was at a different size and position than the ARM64's "magic" field. If the existing ARM64 Image header parsing code were to attempt to parse an existing RISC-V kernel image header format, it would see a magic number 0. This is undesirable, since it's our intention to align as closely as possible with the ARM64 header format. Another problem was that the original "res3" field was not being initialized correctly to zero. Address these issues by creating a 32-bit "magic2" field in the RISC-V header which matches the ARM64 "magic" field. RISC-V binaries will store "RSC\x05" in this field. The intention is that the use of the existing 64-bit "magic" field in the RISC-V header will be deprecated over time. Increment the minor version number of the file format to indicate this change, and update the documentation accordingly. Fix the assembler directives in head.S to ensure that reserved fields are properly zero-initialized. Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com> Reported-by: Palmer Dabbelt <palmer@sifive.com> Reviewed-by: Palmer Dabbelt <palmer@sifive.com> Cc: Atish Patra <atish.patra@wdc.com> Cc: Karsten Merker <merker@debian.org> Link: https://lore.kernel.org/linux-riscv/194c2f10c9806720623430dbf0cc59a965e50448.camel@wdc.com/T/#u Link: https://lore.kernel.org/linux-riscv/mhng-755b14c4-8f35-4079-a7ff-e421fd1b02bc@palmer-si-x1e/T/#t
2019-09-13 18:35:50 -07:00
.ascii RISCV_IMAGE_MAGIC2
RISC-V: Add PE/COFF header for EFI stub Linux kernel Image can appear as an EFI application With appropriate PE/COFF header fields in the beginning of the Image header. An EFI application loader can directly load a Linux kernel Image and an EFI stub residing in kernel can boot Linux kernel directly. Add the necessary PE/COFF header. Signed-off-by: Atish Patra <atish.patra@wdc.com> Link: https://lore.kernel.org/r/20200421033336.9663-3-atish.patra@wdc.com [ardb: - use C prefix for c.li to ensure the expected opcode is emitted - align all image sections according to PE/COFF section alignment ] Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-09-17 15:37:13 -07:00
#ifdef CONFIG_EFI
.word pe_head_start - _start
pe_head_start:
__EFI_PE_HEADER
#else
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 16:08:00 -07:00
.word 0
RISC-V: Add PE/COFF header for EFI stub Linux kernel Image can appear as an EFI application With appropriate PE/COFF header fields in the beginning of the Image header. An EFI application loader can directly load a Linux kernel Image and an EFI stub residing in kernel can boot Linux kernel directly. Add the necessary PE/COFF header. Signed-off-by: Atish Patra <atish.patra@wdc.com> Link: https://lore.kernel.org/r/20200421033336.9663-3-atish.patra@wdc.com [ardb: - use C prefix for c.li to ensure the expected opcode is emitted - align all image sections according to PE/COFF section alignment ] Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-09-17 15:37:13 -07:00
#endif
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 16:08:00 -07:00
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
.align 2
#ifdef CONFIG_MMU
RISC-V: Rename relocate() and make it global The low-level relocate() function enables mmu and relocates execution to link-time addresses. We rename relocate() function to relocate_enable_mmu() function which is more informative. Also, the relocate_enable_mmu() function will be used in the resume path when a CPU wakes-up from a non-retentive suspend so we make it global symbol. Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Anup Patel <apatel@ventanamicro.com> Reviewed-by: Guo Ren <guoren@kernel.org> Reviewed-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-02-10 11:19:41 +05:30
.global relocate_enable_mmu
relocate_enable_mmu:
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
/* Relocate return address */
riscv: Introduce structure that group all variables regarding kernel mapping We have a lot of variables that are used to hold kernel mapping addresses, offsets between physical and virtual mappings and some others used for XIP kernels: they are all defined at different places in mm/init.c, so group them into a single structure with, for some of them, more explicit and concise names. Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-06-17 15:53:07 +02:00
la a1, kernel_map
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
XIP_FIXUP_OFFSET a1
riscv: Introduce structure that group all variables regarding kernel mapping We have a lot of variables that are used to hold kernel mapping addresses, offsets between physical and virtual mappings and some others used for XIP kernels: they are all defined at different places in mm/init.c, so group them into a single structure with, for some of them, more explicit and concise names. Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-06-17 15:53:07 +02:00
REG_L a1, KERNEL_MAP_VIRT_ADDR(a1)
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
la a2, _start
sub a1, a1, a2
add ra, ra, a1
/* Point stvec to virtual address of intruction after satp write */
la a2, 1f
add a2, a2, a1
csrw CSR_TVEC, a2
/* Compute satp for kernel page tables, but don't load it yet */
srl a2, a0, PAGE_SHIFT
riscv: Implement sv48 support By adding a new 4th level of page table, give the possibility to 64bit kernel to address 2^48 bytes of virtual address: in practice, that offers 128TB of virtual address space to userspace and allows up to 64TB of physical memory. If the underlying hardware does not support sv48, we will automatically fallback to a standard 3-level page table by folding the new PUD level into PGDIR level. In order to detect HW capabilities at runtime, we use SATP feature that ignores writes with an unsupported mode. Signed-off-by: Alexandre Ghiti <alexandre.ghiti@canonical.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2021-12-06 11:46:51 +01:00
la a1, satp_mode
riscv: Make XIP bootable again Currently, the XIP kernel seems to fail to boot due to missing XIP_FIXUP and a wrong page_offset value. A superfluous XIP_FIXUP has also been removed. Signed-off-by: Frederik Haxel <haxel@fzi.de> Link: https://lore.kernel.org/r/20231212130116.848530-2-haxel@fzi.de Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-12-12 14:01:12 +01:00
XIP_FIXUP_OFFSET a1
riscv: Implement sv48 support By adding a new 4th level of page table, give the possibility to 64bit kernel to address 2^48 bytes of virtual address: in practice, that offers 128TB of virtual address space to userspace and allows up to 64TB of physical memory. If the underlying hardware does not support sv48, we will automatically fallback to a standard 3-level page table by folding the new PUD level into PGDIR level. In order to detect HW capabilities at runtime, we use SATP feature that ignores writes with an unsupported mode. Signed-off-by: Alexandre Ghiti <alexandre.ghiti@canonical.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2021-12-06 11:46:51 +01:00
REG_L a1, 0(a1)
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
or a2, a2, a1
/*
* Load trampoline page directory, which will cause us to trap to
* stvec if VA != PA, or simply fall through if VA == PA. We need a
* full fence here because setup_vm() just wrote these PTEs and we need
* to ensure the new translations are in use.
*/
la a0, trampoline_pg_dir
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
XIP_FIXUP_OFFSET a0
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
srl a0, a0, PAGE_SHIFT
or a0, a0, a1
sfence.vma
csrw CSR_SATP, a0
.align 2
1:
riscv: Setup exception vector for nommu platform Exception vector is missing on nommu platform and that is an issue. This patch is tested in Sipeed Maix Bit Dev Board. Fixes: 79b1feba5455 ("RISC-V: Setup exception vector early") Suggested-by: Anup Patel <anup@brainfault.org> Suggested-by: Atish Patra <atishp@atishpatra.org> Signed-off-by: Qiu Wenbo <qiuwenbo@phytium.com.cn> Reviewed-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-08-13 11:38:04 +08:00
/* Set trap vector to spin forever to help debug */
la a0, .Lsecondary_park
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
csrw CSR_TVEC, a0
/* Reload the global pointer */
riscv: Move global pointer loading to a macro In Clang 17, -fsanitize=shadow-call-stack uses the newly declared platform register gp for storing shadow call stack pointers. As this is obviously incompatible with gp relaxation, in preparation for CONFIG_SHADOW_CALL_STACK support, move global pointer loading to a single macro, which we can cleanly disable when SCS is used instead. Link: https://reviews.llvm.org/rGaa1d2693c256 Link: https://github.com/riscv-non-isa/riscv-elf-psabi-doc/commit/a484e843e6eeb51f0cb7b8819e50da6d2444d769 Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Tested-by: Nathan Chancellor <nathan@kernel.org> Link: https://lore.kernel.org/r/20230927224757.1154247-11-samitolvanen@google.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-27 22:48:01 +00:00
load_global_pointer
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
/*
* Switch to kernel page tables. A full fence is necessary in order to
* avoid using the trampoline translations, which are only correct for
riscv/head: fix misspelling of guaranteed Fixes misspelling of guaranteed in comment describing why fetching fence is guaranteed to work when switching to kernel page tables. Signed-off-by: hasheddan <georgedanielmangum@gmail.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2021-11-20 09:26:05 -05:00
* the first superpage. Fetching the fence is guaranteed to work
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
* because that first superpage is translated the same way.
*/
csrw CSR_SATP, a2
sfence.vma
ret
#endif /* CONFIG_MMU */
#ifdef CONFIG_SMP
RISC-V: Add supported for ordered booting method using HSM Currently, all harts have to jump Linux in RISC-V. This complicates the multi-stage boot process as every transient stage also has to ensure all harts enter to that stage and jump to Linux afterwards. It also obstructs a clean Kexec implementation. SBI HSM extension provides alternate solutions where only a single hart need to boot and enter Linux. The booting hart can bring up secondary harts one by one afterwards. Add SBI HSM based cpu_ops that implements an ordered booting method in RISC-V. This change is also backward compatible with older firmware not implementing HSM extension. If a latest kernel is used with older firmware, it will continue to use the default spinning booting method. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:43 -07:00
.global secondary_start_sbi
secondary_start_sbi:
/* Mask all interrupts */
csrw CSR_IE, zero
csrw CSR_IP, zero
/* Load the global pointer */
riscv: Move global pointer loading to a macro In Clang 17, -fsanitize=shadow-call-stack uses the newly declared platform register gp for storing shadow call stack pointers. As this is obviously incompatible with gp relaxation, in preparation for CONFIG_SHADOW_CALL_STACK support, move global pointer loading to a single macro, which we can cleanly disable when SCS is used instead. Link: https://reviews.llvm.org/rGaa1d2693c256 Link: https://github.com/riscv-non-isa/riscv-elf-psabi-doc/commit/a484e843e6eeb51f0cb7b8819e50da6d2444d769 Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Tested-by: Nathan Chancellor <nathan@kernel.org> Link: https://lore.kernel.org/r/20230927224757.1154247-11-samitolvanen@google.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-27 22:48:01 +00:00
load_global_pointer
RISC-V: Add supported for ordered booting method using HSM Currently, all harts have to jump Linux in RISC-V. This complicates the multi-stage boot process as every transient stage also has to ensure all harts enter to that stage and jump to Linux afterwards. It also obstructs a clean Kexec implementation. SBI HSM extension provides alternate solutions where only a single hart need to boot and enter Linux. The booting hart can bring up secondary harts one by one afterwards. Add SBI HSM based cpu_ops that implements an ordered booting method in RISC-V. This change is also backward compatible with older firmware not implementing HSM extension. If a latest kernel is used with older firmware, it will continue to use the default spinning booting method. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:43 -07:00
/*
riscv: Disable Vector Instructions for kernel itself Disable vector instructions execution for kernel mode at its entrances. This helps find illegal uses of vector in the kernel space, which is similar to the fpu. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Co-developed-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Co-developed-by: Han-Kuan Chen <hankuan.chen@sifive.com> Signed-off-by: Han-Kuan Chen <hankuan.chen@sifive.com> Co-developed-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Vineet Gupta <vineetg@rivosinc.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Tested-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Reviewed-by: Palmer Dabbelt <palmer@rivosinc.com> Link: https://lore.kernel.org/r/20230605110724.21391-7-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 11:07:03 +00:00
* Disable FPU & VECTOR to detect illegal usage of
* floating point or vector in kernel space
RISC-V: Add supported for ordered booting method using HSM Currently, all harts have to jump Linux in RISC-V. This complicates the multi-stage boot process as every transient stage also has to ensure all harts enter to that stage and jump to Linux afterwards. It also obstructs a clean Kexec implementation. SBI HSM extension provides alternate solutions where only a single hart need to boot and enter Linux. The booting hart can bring up secondary harts one by one afterwards. Add SBI HSM based cpu_ops that implements an ordered booting method in RISC-V. This change is also backward compatible with older firmware not implementing HSM extension. If a latest kernel is used with older firmware, it will continue to use the default spinning booting method. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:43 -07:00
*/
riscv: Disable Vector Instructions for kernel itself Disable vector instructions execution for kernel mode at its entrances. This helps find illegal uses of vector in the kernel space, which is similar to the fpu. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Co-developed-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Co-developed-by: Han-Kuan Chen <hankuan.chen@sifive.com> Signed-off-by: Han-Kuan Chen <hankuan.chen@sifive.com> Co-developed-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Vineet Gupta <vineetg@rivosinc.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Tested-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Reviewed-by: Palmer Dabbelt <palmer@rivosinc.com> Link: https://lore.kernel.org/r/20230605110724.21391-7-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 11:07:03 +00:00
li t0, SR_FS_VS
RISC-V: Add supported for ordered booting method using HSM Currently, all harts have to jump Linux in RISC-V. This complicates the multi-stage boot process as every transient stage also has to ensure all harts enter to that stage and jump to Linux afterwards. It also obstructs a clean Kexec implementation. SBI HSM extension provides alternate solutions where only a single hart need to boot and enter Linux. The booting hart can bring up secondary harts one by one afterwards. Add SBI HSM based cpu_ops that implements an ordered booting method in RISC-V. This change is also backward compatible with older firmware not implementing HSM extension. If a latest kernel is used with older firmware, it will continue to use the default spinning booting method. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:43 -07:00
csrc CSR_STATUS, t0
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
/* Set trap vector to spin forever to help debug */
la a3, .Lsecondary_park
csrw CSR_TVEC, a3
RISC-V: Avoid using per cpu array for ordered booting Currently both order booting and spinwait approach uses a per cpu array to update stack & task pointer. This approach will not work for the following cases. 1. If NR_CPUs are configured to be less than highest hart id. 2. A platform has sparse hartid. This issue can be fixed for ordered booting as the booting cpu brings up one cpu at a time using SBI HSM extension which has opaque parameter that is unused until now. Introduce a common secondary boot data structure that can store the stack and task pointer. Secondary harts will use this data while booting up to setup the sp & tp. Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-01-20 01:09:13 -08:00
/* a0 contains the hartid & a1 contains boot data */
li a2, SBI_HART_BOOT_TASK_PTR_OFFSET
XIP_FIXUP_OFFSET a2
add a2, a2, a1
REG_L tp, (a2)
li a3, SBI_HART_BOOT_STACK_PTR_OFFSET
XIP_FIXUP_OFFSET a3
add a3, a3, a1
REG_L sp, (a3)
RISC-V: Add supported for ordered booting method using HSM Currently, all harts have to jump Linux in RISC-V. This complicates the multi-stage boot process as every transient stage also has to ensure all harts enter to that stage and jump to Linux afterwards. It also obstructs a clean Kexec implementation. SBI HSM extension provides alternate solutions where only a single hart need to boot and enter Linux. The booting hart can bring up secondary harts one by one afterwards. Add SBI HSM based cpu_ops that implements an ordered booting method in RISC-V. This change is also backward compatible with older firmware not implementing HSM extension. If a latest kernel is used with older firmware, it will continue to use the default spinning booting method. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:43 -07:00
riscv: head: make secondary_start_common() static There are no users outside head.S so make secondary_start_common() static. Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2021-11-29 00:07:38 +08:00
.Lsecondary_start_common:
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
#ifdef CONFIG_MMU
/* Enable virtual memory and relocate to virtual address */
la a0, swapper_pg_dir
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
XIP_FIXUP_OFFSET a0
RISC-V: Rename relocate() and make it global The low-level relocate() function enables mmu and relocates execution to link-time addresses. We rename relocate() function to relocate_enable_mmu() function which is more informative. Also, the relocate_enable_mmu() function will be used in the resume path when a CPU wakes-up from a non-retentive suspend so we make it global symbol. Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Anup Patel <apatel@ventanamicro.com> Reviewed-by: Guo Ren <guoren@kernel.org> Reviewed-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-02-10 11:19:41 +05:30
call relocate_enable_mmu
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
#endif
riscv: use ".L" local labels in assembly when applicable For the sake of coherency, use local labels in assembly when applicable. This also avoid kprobes being confused when applying a kprobe since the size of function is computed by checking where the next visible symbol is located. This might end up in computing some function size to be way shorter than expected and thus failing to apply kprobes to the specified offset. Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-2-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:51 +02:00
call .Lsetup_trap_vector
riscv: Fix SMP when shadow call stacks are enabled This fixes two bugs in SCS initialization for secondary CPUs. First, the SCS was not initialized at all in the spinwait boot path. Second, the code for the SBI HSM path attempted to initialize the SCS before enabling the MMU. However, that involves dereferencing the thread pointer, which requires the MMU to be enabled. Fix both issues by setting up the SCS in the common secondary entry path, after enabling the MMU. Fixes: d1584d791a29 ("riscv: Implement Shadow Call Stack") Signed-off-by: Samuel Holland <samuel.holland@sifive.com> Reviewed-by: Sami Tolvanen <samitolvanen@google.com> Link: https://lore.kernel.org/r/20231121211958.3158576-1-samuel.holland@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-11-21 13:19:29 -08:00
scs_load_current
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
tail smp_callin
#endif /* CONFIG_SMP */
riscv: Setup exception vector for nommu platform Exception vector is missing on nommu platform and that is an issue. This patch is tested in Sipeed Maix Bit Dev Board. Fixes: 79b1feba5455 ("RISC-V: Setup exception vector early") Suggested-by: Anup Patel <anup@brainfault.org> Suggested-by: Atish Patra <atishp@atishpatra.org> Signed-off-by: Qiu Wenbo <qiuwenbo@phytium.com.cn> Reviewed-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-08-13 11:38:04 +08:00
.align 2
riscv: use ".L" local labels in assembly when applicable For the sake of coherency, use local labels in assembly when applicable. This also avoid kprobes being confused when applying a kprobe since the size of function is computed by checking where the next visible symbol is located. This might end up in computing some function size to be way shorter than expected and thus failing to apply kprobes to the specified offset. Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-2-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:51 +02:00
.Lsetup_trap_vector:
riscv: Setup exception vector for nommu platform Exception vector is missing on nommu platform and that is an issue. This patch is tested in Sipeed Maix Bit Dev Board. Fixes: 79b1feba5455 ("RISC-V: Setup exception vector early") Suggested-by: Anup Patel <anup@brainfault.org> Suggested-by: Atish Patra <atishp@atishpatra.org> Signed-off-by: Qiu Wenbo <qiuwenbo@phytium.com.cn> Reviewed-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-08-13 11:38:04 +08:00
/* Set trap vector to exception handler */
la a0, handle_exception
csrw CSR_TVEC, a0
/*
* Set sup0 scratch register to 0, indicating to exception vector that
* we are presently executing in kernel.
*/
csrw CSR_SCRATCH, zero
ret
riscv: fix misalgned trap vector base address The trap vector marked by label .Lsecondary_park must align on a 4-byte boundary, as the {m,s}tvec is defined to require 4-byte alignment. Signed-off-by: Chen Lu <181250012@smail.nju.edu.cn> Reviewed-by: Anup Patel <anup.patel@wdc.com> Fixes: e011995e826f ("RISC-V: Move relocate and few other functions out of __init") Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-10-18 13:22:38 +08:00
.align 2
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
.Lsecondary_park:
/* We lack SMP support or have too many harts, so park this hart */
wfi
j .Lsecondary_park
riscv: Use SYM_*() assembly macros instead of deprecated ones ENTRY()/END()/WEAK() macros are deprecated and we should make use of the new SYM_*() macros [1] for better annotation of symbols. Replace the deprecated ones with the new ones and fix wrong usage of END()/ENDPROC() to correctly describe the symbols. [1] https://docs.kernel.org/core-api/asm-annotations.html Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-3-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:52 +02:00
SYM_CODE_END(_start)
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-17 18:11:39 -07:00
riscv: Use SYM_*() assembly macros instead of deprecated ones ENTRY()/END()/WEAK() macros are deprecated and we should make use of the new SYM_*() macros [1] for better annotation of symbols. Replace the deprecated ones with the new ones and fix wrong usage of END()/ENDPROC() to correctly describe the symbols. [1] https://docs.kernel.org/core-api/asm-annotations.html Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-3-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:52 +02:00
SYM_CODE_START(_start_kernel)
RISC-V: Access CSRs using CSR numbers We should prefer accessing CSRs using their CSR numbers because: 1. It compiles fine with older toolchains. 2. We can use latest CSR names in #define macro names of CSR numbers as-per RISC-V spec. 3. We can access newly added CSRs even if toolchain does not recognize newly addes CSRs by name. Signed-off-by: Anup Patel <anup.patel@wdc.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-04-25 08:38:41 +00:00
/* Mask all interrupts */
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
csrw CSR_IE, zero
csrw CSR_IP, zero
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
riscv: read the hart ID from mhartid on boot When in M-Mode, we can use the mhartid CSR to get the ID of the running HART. Doing so, direct M-Mode boot without firmware is possible. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:39 +01:00
#ifdef CONFIG_RISCV_M_MODE
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:40 +01:00
/* flush the instruction cache */
fence.i
/* Reset all registers except ra, a0, a1 */
call reset_regs
RISC-V: Skip setting up PMPs on traps The RISC-V ISA manual says that PMPs are WARL, but it appears the K210 doesn't implement them and instead traps on the unsupported accesses. This patch handles those traps by just skipping the PMP initialization entirely, under the theory that machines that trap on PMP accesses must allow memory accesses as otherwise they're pretty useless. Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-07 11:33:40 -07:00
/*
* Setup a PMP to permit access to all of memory. Some machines may
* not implement PMPs, so we set up a quick trap handler to just skip
* touching the PMPs on any trap.
*/
riscv: use ".L" local labels in assembly when applicable For the sake of coherency, use local labels in assembly when applicable. This also avoid kprobes being confused when applying a kprobe since the size of function is computed by checking where the next visible symbol is located. This might end up in computing some function size to be way shorter than expected and thus failing to apply kprobes to the specified offset. Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-2-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:51 +02:00
la a0, .Lpmp_done
RISC-V: Skip setting up PMPs on traps The RISC-V ISA manual says that PMPs are WARL, but it appears the K210 doesn't implement them and instead traps on the unsupported accesses. This patch handles those traps by just skipping the PMP initialization entirely, under the theory that machines that trap on PMP accesses must allow memory accesses as otherwise they're pretty useless. Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-07 11:33:40 -07:00
csrw CSR_TVEC, a0
riscv: set pmp configuration if kernel is running in M-mode When the kernel is running in S-mode, the expectation is that the bootloader or SBI layer will configure the PMP to allow the kernel to access physical memory. But, when the kernel is running in M-mode and is started with the ELF "loader", there's probably no bootloader or SBI layer involved to configure the PMP. Thus, we need to configure the PMP ourselves to enable the kernel to access all regions. Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-01-09 11:17:40 +08:00
li a0, -1
csrw CSR_PMPADDR0, a0
li a0, (PMP_A_NAPOT | PMP_R | PMP_W | PMP_X)
csrw CSR_PMPCFG0, a0
RISC-V: Skip setting up PMPs on traps The RISC-V ISA manual says that PMPs are WARL, but it appears the K210 doesn't implement them and instead traps on the unsupported accesses. This patch handles those traps by just skipping the PMP initialization entirely, under the theory that machines that trap on PMP accesses must allow memory accesses as otherwise they're pretty useless. Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-07 11:33:40 -07:00
.align 2
riscv: use ".L" local labels in assembly when applicable For the sake of coherency, use local labels in assembly when applicable. This also avoid kprobes being confused when applying a kprobe since the size of function is computed by checking where the next visible symbol is located. This might end up in computing some function size to be way shorter than expected and thus failing to apply kprobes to the specified offset. Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-2-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:51 +02:00
.Lpmp_done:
riscv: set pmp configuration if kernel is running in M-mode When the kernel is running in S-mode, the expectation is that the bootloader or SBI layer will configure the PMP to allow the kernel to access physical memory. But, when the kernel is running in M-mode and is started with the ELF "loader", there's probably no bootloader or SBI layer involved to configure the PMP. Thus, we need to configure the PMP ourselves to enable the kernel to access all regions. Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-01-09 11:17:40 +08:00
riscv: read the hart ID from mhartid on boot When in M-Mode, we can use the mhartid CSR to get the ID of the running HART. Doing so, direct M-Mode boot without firmware is possible. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:39 +01:00
/*
* The hartid in a0 is expected later on, and we have no firmware
* to hand it to us.
*/
csrr a0, CSR_MHARTID
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:40 +01:00
#endif /* CONFIG_RISCV_M_MODE */
riscv: read the hart ID from mhartid on boot When in M-Mode, we can use the mhartid CSR to get the ID of the running HART. Doing so, direct M-Mode boot without firmware is possible. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:39 +01:00
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
/* Load the global pointer */
riscv: Move global pointer loading to a macro In Clang 17, -fsanitize=shadow-call-stack uses the newly declared platform register gp for storing shadow call stack pointers. As this is obviously incompatible with gp relaxation, in preparation for CONFIG_SHADOW_CALL_STACK support, move global pointer loading to a single macro, which we can cleanly disable when SCS is used instead. Link: https://reviews.llvm.org/rGaa1d2693c256 Link: https://github.com/riscv-non-isa/riscv-elf-psabi-doc/commit/a484e843e6eeb51f0cb7b8819e50da6d2444d769 Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Tested-by: Nathan Chancellor <nathan@kernel.org> Link: https://lore.kernel.org/r/20230927224757.1154247-11-samitolvanen@google.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-27 22:48:01 +00:00
load_global_pointer
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
/*
riscv: Disable Vector Instructions for kernel itself Disable vector instructions execution for kernel mode at its entrances. This helps find illegal uses of vector in the kernel space, which is similar to the fpu. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Co-developed-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Co-developed-by: Han-Kuan Chen <hankuan.chen@sifive.com> Signed-off-by: Han-Kuan Chen <hankuan.chen@sifive.com> Co-developed-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Vineet Gupta <vineetg@rivosinc.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Tested-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Reviewed-by: Palmer Dabbelt <palmer@rivosinc.com> Link: https://lore.kernel.org/r/20230605110724.21391-7-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 11:07:03 +00:00
* Disable FPU & VECTOR to detect illegal usage of
* floating point or vector in kernel space
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
*/
riscv: Disable Vector Instructions for kernel itself Disable vector instructions execution for kernel mode at its entrances. This helps find illegal uses of vector in the kernel space, which is similar to the fpu. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Co-developed-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Co-developed-by: Han-Kuan Chen <hankuan.chen@sifive.com> Signed-off-by: Han-Kuan Chen <hankuan.chen@sifive.com> Co-developed-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Vineet Gupta <vineetg@rivosinc.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Tested-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Reviewed-by: Palmer Dabbelt <palmer@rivosinc.com> Link: https://lore.kernel.org/r/20230605110724.21391-7-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 11:07:03 +00:00
li t0, SR_FS_VS
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
csrc CSR_STATUS, t0
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
RISC-V: Move spinwait booting method to its own config The spinwait booting method should only be used for platforms with older firmware without SBI HSM extension or M-mode firmware because spinwait method can't support cpu hotplug, kexec or sparse hartid. It is better to move the entire spinwait implementation to its own config which can be disabled if required. It is enabled by default to maintain backward compatibility and M-mode Linux. Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-01-20 01:09:17 -08:00
#ifdef CONFIG_RISCV_BOOT_SPINWAIT
arch/riscv: disable excess harts before picking main boot hart Harts with id greater than or equal to CONFIG_NR_CPUS need to be disabled. But the kernel can pick any hart as the main hart. So, before picking the main hart, the kernel must disable harts with ids greater than or equal to CONFIG_NR_CPUS. Signed-off-by: Xiang Wang <merle@hardenedlinux.org> Reviewed-by: Palmer Dabbelt <palmer@sifive.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> [paul.walmsley@sifive.com: updated to apply; cleaned up patch description] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-09-06 11:56:09 +08:00
li t0, CONFIG_NR_CPUS
riscv: make sure the cores stay looping in .Lsecondary_park The code in secondary_park is currently placed in the .init section. The kernel reclaims and clears this code when it finishes booting. That causes the cores parked in it to go to somewhere unpredictable, so we move this function out of init to make sure the cores stay looping there. The instruction bgeu a0, t0, .Lsecondary_park may have "a relocation truncated to fit" issue during linking time. It is because that sections are too far to jump. Let's use tail to jump to the .Lsecondary_park. Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Reviewed-by: Anup Patel <anup.patel@sifive.com> Cc: Andreas Schwab <schwab@suse.de> Cc: stable@vger.kernel.org Fixes: 76d2a0493a17d ("RISC-V: Init and Halt Code") Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2020-01-15 14:54:36 +08:00
blt a0, t0, .Lgood_cores
tail .Lsecondary_park
.Lgood_cores:
arch/riscv: disable excess harts before picking main boot hart Harts with id greater than or equal to CONFIG_NR_CPUS need to be disabled. But the kernel can pick any hart as the main hart. So, before picking the main hart, the kernel must disable harts with ids greater than or equal to CONFIG_NR_CPUS. Signed-off-by: Xiang Wang <merle@hardenedlinux.org> Reviewed-by: Palmer Dabbelt <palmer@sifive.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> [paul.walmsley@sifive.com: updated to apply; cleaned up patch description] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-09-06 11:56:09 +08:00
RISC-V: Move the entire hart selection via lottery to SMP The booting hart selection via lottery is only useful for SMP systems. Moreover, the lottery selection is only necessary for systems using spinwait booting method. It is better to keep the entire lottery selection together so that it can be disabled in future. Move the lottery selection code to under CONFIG_SMP. Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-01-20 01:09:16 -08:00
/* The lottery system is only required for spinwait booting method */
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
#ifndef CONFIG_XIP_KERNEL
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
/* Pick one hart to run the main boot sequence */
la a3, hart_lottery
li a2, 1
amoadd.w a3, a2, (a3)
bnez a3, .Lsecondary_start
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
#else
/* hart_lottery in flash contains a magic number */
la a3, hart_lottery
mv a2, a3
XIP_FIXUP_OFFSET a2
riscv: remove .text section size limitation for XIP Currently there's a limit of 8MB for the .text section of a RISC-V image in the XIP case. This breaks compilation of many automatic builds and is generally inconvenient. This patch removes that limitation and optimizes XIP image file size at the same time. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-10-11 11:14:14 +02:00
XIP_FIXUP_FLASH_OFFSET a3
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
lw t1, (a3)
amoswap.w t0, t1, (a2)
/* first time here if hart_lottery in RAM is not set */
beq t0, t1, .Lsecondary_start
RISC-V: Move the entire hart selection via lottery to SMP The booting hart selection via lottery is only useful for SMP systems. Moreover, the lottery selection is only necessary for systems using spinwait booting method. It is better to keep the entire lottery selection together so that it can be disabled in future. Move the lottery selection code to under CONFIG_SMP. Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-01-20 01:09:16 -08:00
#endif /* CONFIG_XIP */
RISC-V: Move spinwait booting method to its own config The spinwait booting method should only be used for platforms with older firmware without SBI HSM extension or M-mode firmware because spinwait method can't support cpu hotplug, kexec or sparse hartid. It is better to move the entire spinwait implementation to its own config which can be disabled if required. It is enabled by default to maintain backward compatibility and M-mode Linux. Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-01-20 01:09:17 -08:00
#endif /* CONFIG_RISCV_BOOT_SPINWAIT */
RISC-V: Move the entire hart selection via lottery to SMP The booting hart selection via lottery is only useful for SMP systems. Moreover, the lottery selection is only necessary for systems using spinwait booting method. It is better to keep the entire lottery selection together so that it can be disabled in future. Move the lottery selection code to under CONFIG_SMP. Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-01-20 01:09:16 -08:00
#ifdef CONFIG_XIP_KERNEL
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
la sp, _end + THREAD_SIZE
XIP_FIXUP_OFFSET sp
mv s0, a0
riscv: Allow disabling of BUILTIN_DTB for XIP This enables, among other things, testing with the QEMU virt machine. To build an XIP kernel for the QEMU virt machine, configure the the kernel as desired and apply the following configuration ``` CONFIG_NONPORTABLE=y CONFIG_XIP_KERNEL=y CONFIG_XIP_PHYS_ADDR=0x20000000 CONFIG_PHYS_RAM_BASE=0x80200000 CONFIG_BUILTIN_DTB=n ``` Since the QEMU virt flash memory expects a 32 MB file, the built image must be padded. For example, with `truncate -s 32M arch/riscv/boot/xipImage` The kernel can be started using the following command in QEMU (v8+) ``` qemu-system-riscv64 -M virt,pflash0=pflash0 \ -blockdev node-name=pflash0,driver=file,read-only=on,\ filename=arch/riscv/boot/xipImage <optional parameters> ``` Signed-off-by: Frederik Haxel <haxel@fzi.de> Link: https://lore.kernel.org/r/20231212130116.848530-4-haxel@fzi.de Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-12-12 14:01:14 +01:00
mv s1, a1
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
call __copy_data
riscv: Allow disabling of BUILTIN_DTB for XIP This enables, among other things, testing with the QEMU virt machine. To build an XIP kernel for the QEMU virt machine, configure the the kernel as desired and apply the following configuration ``` CONFIG_NONPORTABLE=y CONFIG_XIP_KERNEL=y CONFIG_XIP_PHYS_ADDR=0x20000000 CONFIG_PHYS_RAM_BASE=0x80200000 CONFIG_BUILTIN_DTB=n ``` Since the QEMU virt flash memory expects a 32 MB file, the built image must be padded. For example, with `truncate -s 32M arch/riscv/boot/xipImage` The kernel can be started using the following command in QEMU (v8+) ``` qemu-system-riscv64 -M virt,pflash0=pflash0 \ -blockdev node-name=pflash0,driver=file,read-only=on,\ filename=arch/riscv/boot/xipImage <optional parameters> ``` Signed-off-by: Frederik Haxel <haxel@fzi.de> Link: https://lore.kernel.org/r/20231212130116.848530-4-haxel@fzi.de Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-12-12 14:01:14 +01:00
/* Restore a0 & a1 copy */
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
mv a0, s0
riscv: Allow disabling of BUILTIN_DTB for XIP This enables, among other things, testing with the QEMU virt machine. To build an XIP kernel for the QEMU virt machine, configure the the kernel as desired and apply the following configuration ``` CONFIG_NONPORTABLE=y CONFIG_XIP_KERNEL=y CONFIG_XIP_PHYS_ADDR=0x20000000 CONFIG_PHYS_RAM_BASE=0x80200000 CONFIG_BUILTIN_DTB=n ``` Since the QEMU virt flash memory expects a 32 MB file, the built image must be padded. For example, with `truncate -s 32M arch/riscv/boot/xipImage` The kernel can be started using the following command in QEMU (v8+) ``` qemu-system-riscv64 -M virt,pflash0=pflash0 \ -blockdev node-name=pflash0,driver=file,read-only=on,\ filename=arch/riscv/boot/xipImage <optional parameters> ``` Signed-off-by: Frederik Haxel <haxel@fzi.de> Link: https://lore.kernel.org/r/20231212130116.848530-4-haxel@fzi.de Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-12-12 14:01:14 +01:00
mv a1, s1
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
#endif
#ifndef CONFIG_XIP_KERNEL
RISC-V: Build flat and compressed kernel images This patch extends Linux RISC-V build system to build and install: Image - Flat uncompressed kernel image Image.gz - Flat and GZip compressed kernel image Quiet a few bootloaders (such as Uboot, UEFI, etc) are capable of booting flat and compressed kernel images. In case of Uboot, booting Image or Image.gz is achieved using bootm command. The flat and uncompressed kernel image (i.e. Image) is very useful in pre-silicon developent and testing because we can create back-door HEX files for RAM on FPGAs from Image. Signed-off-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-11-12 11:25:15 +05:30
/* Clear BSS for flat non-ELF images */
la a3, __bss_start
la a4, __bss_stop
riscv: use ".L" local labels in assembly when applicable For the sake of coherency, use local labels in assembly when applicable. This also avoid kprobes being confused when applying a kprobe since the size of function is computed by checking where the next visible symbol is located. This might end up in computing some function size to be way shorter than expected and thus failing to apply kprobes to the specified offset. Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-2-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:51 +02:00
ble a4, a3, .Lclear_bss_done
.Lclear_bss:
RISC-V: Build flat and compressed kernel images This patch extends Linux RISC-V build system to build and install: Image - Flat uncompressed kernel image Image.gz - Flat and GZip compressed kernel image Quiet a few bootloaders (such as Uboot, UEFI, etc) are capable of booting flat and compressed kernel images. In case of Uboot, booting Image or Image.gz is achieved using bootm command. The flat and uncompressed kernel image (i.e. Image) is very useful in pre-silicon developent and testing because we can create back-door HEX files for RAM on FPGAs from Image. Signed-off-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-11-12 11:25:15 +05:30
REG_S zero, (a3)
add a3, a3, RISCV_SZPTR
riscv: use ".L" local labels in assembly when applicable For the sake of coherency, use local labels in assembly when applicable. This also avoid kprobes being confused when applying a kprobe since the size of function is computed by checking where the next visible symbol is located. This might end up in computing some function size to be way shorter than expected and thus failing to apply kprobes to the specified offset. Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-2-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:51 +02:00
blt a3, a4, .Lclear_bss
.Lclear_bss_done:
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
#endif
RISC-V: Use Linux logical CPU number instead of hartid Setup the cpu_logical_map during boot. Moreover, every SBI call and PLIC context are based on the physical hartid. Use the logical CPU to hartid mapping to pass correct hartid to respective functions. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-10-02 12:15:05 -07:00
la a2, boot_cpu_hartid
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
XIP_FIXUP_OFFSET a2
RISC-V: Use Linux logical CPU number instead of hartid Setup the cpu_logical_map during boot. Moreover, every SBI call and PLIC context are based on the physical hartid. Use the logical CPU to hartid mapping to pass correct hartid to respective functions. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-10-02 12:15:05 -07:00
REG_S a0, (a2)
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
/* Initialize page tables and relocate to virtual addresses */
riscv: Initialize thread pointer before calling C functions Because of the stack canary feature that reads from the current task structure the stack canary value, the thread pointer register "tp" must be set before calling any C function from head.S: by chance, setup_vm and all the functions that it calls does not seem to be part of the functions where the canary check is done, but in the following commits, some functions will. Fixes: f2c9699f65557a31 ("riscv: Add STACKPROTECTOR supported") Signed-off-by: Alexandre Ghiti <alexandre.ghiti@canonical.com> Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2021-12-06 11:46:56 +01:00
la tp, init_task
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
la sp, init_thread_union + THREAD_SIZE
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
XIP_FIXUP_OFFSET sp
riscv: prevent stack corruption by reserving task_pt_regs(p) early Early function calls, such as setup_vm(), relocate_enable_mmu(), soc_early_init() etc, are free to operate on stack. However, PT_SIZE_ON_STACK bytes at the head of the kernel stack are purposedly reserved for the placement of per-task register context pointed by task_pt_regs(p). Those functions may corrupt task_pt_regs if we overlap the $sp with it. In fact, we had accidentally corrupted sstatus.VS in some tests, treating the kernel to save V context before V was actually allocated, resulting in a kernel panic. Thus, we should skip PT_SIZE_ON_STACK for $sp before making C function calls from the top-level assembly. Co-developed-by: ShihPo Hung <shihpo.hung@sifive.com> Signed-off-by: ShihPo Hung <shihpo.hung@sifive.com> Co-developed-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Tested-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Link: https://lore.kernel.org/r/20230605110724.21391-18-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 11:07:14 +00:00
addi sp, sp, -PT_SIZE_ON_STACK
riscv: Implement Shadow Call Stack Implement CONFIG_SHADOW_CALL_STACK for RISC-V. When enabled, the compiler injects instructions to all non-leaf C functions to store the return address to the shadow stack and unconditionally load it again before returning, which makes it harder to corrupt the return address through a stack overflow, for example. The active shadow call stack pointer is stored in the gp register, which makes SCS incompatible with gp relaxation. Use --no-relax-gp to ensure gp relaxation is disabled and disable global pointer loading. Add SCS pointers to struct thread_info, implement SCS initialization, and task switching Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Tested-by: Nathan Chancellor <nathan@kernel.org> Link: https://lore.kernel.org/r/20230927224757.1154247-12-samitolvanen@google.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-27 22:48:02 +00:00
scs_load_init_stack
riscv: add BUILTIN_DTB support for MMU-enabled targets Sometimes, especially in a production system we may not want to use a "smart bootloader" like u-boot to load kernel, ramdisk and device tree from a filesystem on eMMC, but rather load the kernel from a NAND partition and just run it as soon as we can, and in this case it is convenient to have device tree compiled into the kernel binary. Since this case is not limited to MMU-less systems, let's support it for these which have MMU enabled too. While at it, provide __dtb_start as a parameter to setup_vm() in BUILTIN_DTB case, so we don't have to duplicate BUILTIN_DTB specific processing in MMU-enabled and MMU-disabled versions of setup_vm(). Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-01-16 01:49:48 +02:00
#ifdef CONFIG_BUILTIN_DTB
la a0, __dtb_start
riscv: remove .text section size limitation for XIP Currently there's a limit of 8MB for the .text section of a RISC-V image in the XIP case. This breaks compilation of many automatic builds and is generally inconvenient. This patch removes that limitation and optimizes XIP image file size at the same time. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-10-11 11:14:14 +02:00
XIP_FIXUP_OFFSET a0
riscv: add BUILTIN_DTB support for MMU-enabled targets Sometimes, especially in a production system we may not want to use a "smart bootloader" like u-boot to load kernel, ramdisk and device tree from a filesystem on eMMC, but rather load the kernel from a NAND partition and just run it as soon as we can, and in this case it is convenient to have device tree compiled into the kernel binary. Since this case is not limited to MMU-less systems, let's support it for these which have MMU enabled too. While at it, provide __dtb_start as a parameter to setup_vm() in BUILTIN_DTB case, so we don't have to duplicate BUILTIN_DTB specific processing in MMU-enabled and MMU-disabled versions of setup_vm(). Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-01-16 01:49:48 +02:00
#else
riscv: remove redundant mv instructions Some mv instructions were useful when first introduced to preserve a0 and a1 before function calls. However the code has changed and they are now redundant. Remove them. Signed-off-by: Nam Cao <namcaov@gmail.com> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Link: https://lore.kernel.org/r/20230725053835.138910-1-namcaov@gmail.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-07-25 07:38:35 +02:00
mv a0, a1
riscv: add BUILTIN_DTB support for MMU-enabled targets Sometimes, especially in a production system we may not want to use a "smart bootloader" like u-boot to load kernel, ramdisk and device tree from a filesystem on eMMC, but rather load the kernel from a NAND partition and just run it as soon as we can, and in this case it is convenient to have device tree compiled into the kernel binary. Since this case is not limited to MMU-less systems, let's support it for these which have MMU enabled too. While at it, provide __dtb_start as a parameter to setup_vm() in BUILTIN_DTB case, so we don't have to duplicate BUILTIN_DTB specific processing in MMU-enabled and MMU-disabled versions of setup_vm(). Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-01-16 01:49:48 +02:00
#endif /* CONFIG_BUILTIN_DTB */
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
call setup_vm
riscv: add nommu support The kernel runs in M-mode without using page tables, and thus can't run bare metal without help from additional firmware. Most of the patch is just stubbing out code not needed without page tables, but there is an interesting detail in the signals implementation: - The normal RISC-V syscall ABI only implements rt_sigreturn as VDSO entry point, but the ELF VDSO is not supported for nommu Linux. We instead copy the code to call the syscall onto the stack. In addition to enabling the nommu code a new defconfig for a small kernel image that can run in nommu mode on qemu is also provided, to run a kernel in qemu you can use the following command line: qemu-system-riscv64 -smp 2 -m 64 -machine virt -nographic \ -kernel arch/riscv/boot/loader \ -drive file=rootfs.ext2,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0 Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> [paul.walmsley@sifive.com: updated to apply; add CONFIG_MMU guards around PCI_IOBASE definition to fix build issues; fixed checkpatch issues; move the PCI_IO_* and VMEMMAP address space macros along with the others; resolve sparse warning] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:41 +01:00
#ifdef CONFIG_MMU
RISC-V: Setup initial page tables in two stages Currently, the setup_vm() does initial page table setup in one-shot very early before enabling MMU. Due to this, the setup_vm() has to map all possible kernel virtual addresses since it does not know size and location of RAM. This means we have kernel mappings for non-existent RAM and any buggy driver (or kernel) code doing out-of-bound access to RAM will not fault and cause underterministic behaviour. Further, the setup_vm() creates PMD mappings (i.e. 2M mappings) for RV64 systems. This means for PAGE_OFFSET=0xffffffe000000000 (i.e. MAXPHYSMEM_128GB=y), the setup_vm() will require 129 pages (i.e. 516 KB) of memory for initial page tables which is never freed. The memory required for initial page tables will further increase if we chose a lower value of PAGE_OFFSET (e.g. 0xffffff0000000000) This patch implements two-staged initial page table setup, as follows: 1. Early (i.e. setup_vm()): This stage maps kernel image and DTB in a early page table (i.e. early_pg_dir). The early_pg_dir will be used only by boot HART so it can be freed as-part of init memory free-up. 2. Final (i.e. setup_vm_final()): This stage maps all possible RAM banks in the final page table (i.e. swapper_pg_dir). The boot HART will start using swapper_pg_dir at the end of setup_vm_final(). All non-boot HARTs directly use the swapper_pg_dir created by boot HART. We have following advantages with this new approach: 1. Kernel mappings for non-existent RAM don't exists anymore. 2. Memory consumed by initial page tables is now indpendent of the chosen PAGE_OFFSET. 3. Memory consumed by initial page tables on RV64 system is 2 pages (i.e. 8 KB) which has significantly reduced and these pages will be freed as-part of the init memory free-up. The patch also provides a foundation for implementing strict kernel mappings where we protect kernel text and rodata using PTE permissions. Suggested-by: Mike Rapoport <rppt@linux.ibm.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> [paul.walmsley@sifive.com: updated to apply; fixed a checkpatch warning] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-28 13:36:21 -07:00
la a0, early_pg_dir
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
XIP_FIXUP_OFFSET a0
RISC-V: Rename relocate() and make it global The low-level relocate() function enables mmu and relocates execution to link-time addresses. We rename relocate() function to relocate_enable_mmu() function which is more informative. Also, the relocate_enable_mmu() function will be used in the resume path when a CPU wakes-up from a non-retentive suspend so we make it global symbol. Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Anup Patel <apatel@ventanamicro.com> Reviewed-by: Guo Ren <guoren@kernel.org> Reviewed-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-02-10 11:19:41 +05:30
call relocate_enable_mmu
riscv: add nommu support The kernel runs in M-mode without using page tables, and thus can't run bare metal without help from additional firmware. Most of the patch is just stubbing out code not needed without page tables, but there is an interesting detail in the signals implementation: - The normal RISC-V syscall ABI only implements rt_sigreturn as VDSO entry point, but the ELF VDSO is not supported for nommu Linux. We instead copy the code to call the syscall onto the stack. In addition to enabling the nommu code a new defconfig for a small kernel image that can run in nommu mode on qemu is also provided, to run a kernel in qemu you can use the following command line: qemu-system-riscv64 -smp 2 -m 64 -machine virt -nographic \ -kernel arch/riscv/boot/loader \ -drive file=rootfs.ext2,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0 Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> [paul.walmsley@sifive.com: updated to apply; add CONFIG_MMU guards around PCI_IOBASE definition to fix build issues; fixed checkpatch issues; move the PCI_IO_* and VMEMMAP address space macros along with the others; resolve sparse warning] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:41 +01:00
#endif /* CONFIG_MMU */
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
riscv: use ".L" local labels in assembly when applicable For the sake of coherency, use local labels in assembly when applicable. This also avoid kprobes being confused when applying a kprobe since the size of function is computed by checking where the next visible symbol is located. This might end up in computing some function size to be way shorter than expected and thus failing to apply kprobes to the specified offset. Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-2-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:51 +02:00
call .Lsetup_trap_vector
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
/* Restore C environment */
la tp, init_task
riscv: simplify the stack pointer setup in head.S We don't need THREAD_SIZE in asm-offsets.c as we can just calculate the value of init_thread_union + THREAD_SIZE using cpp, just like we do a few lines above. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-04-15 11:14:37 +02:00
la sp, init_thread_union + THREAD_SIZE
riscv: prevent stack corruption by reserving task_pt_regs(p) early Early function calls, such as setup_vm(), relocate_enable_mmu(), soc_early_init() etc, are free to operate on stack. However, PT_SIZE_ON_STACK bytes at the head of the kernel stack are purposedly reserved for the placement of per-task register context pointed by task_pt_regs(p). Those functions may corrupt task_pt_regs if we overlap the $sp with it. In fact, we had accidentally corrupted sstatus.VS in some tests, treating the kernel to save V context before V was actually allocated, resulting in a kernel panic. Thus, we should skip PT_SIZE_ON_STACK for $sp before making C function calls from the top-level assembly. Co-developed-by: ShihPo Hung <shihpo.hung@sifive.com> Signed-off-by: ShihPo Hung <shihpo.hung@sifive.com> Co-developed-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Tested-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Link: https://lore.kernel.org/r/20230605110724.21391-18-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 11:07:14 +00:00
addi sp, sp, -PT_SIZE_ON_STACK
riscv: Implement Shadow Call Stack Implement CONFIG_SHADOW_CALL_STACK for RISC-V. When enabled, the compiler injects instructions to all non-leaf C functions to store the return address to the shadow stack and unconditionally load it again before returning, which makes it harder to corrupt the return address through a stack overflow, for example. The active shadow call stack pointer is stored in the gp register, which makes SCS incompatible with gp relaxation. Use --no-relax-gp to ensure gp relaxation is disabled and disable global pointer loading. Add SCS pointers to struct thread_info, implement SCS initialization, and task switching Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Tested-by: Nathan Chancellor <nathan@kernel.org> Link: https://lore.kernel.org/r/20230927224757.1154247-12-samitolvanen@google.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-27 22:48:02 +00:00
scs_load_current
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
riscv: Add KASAN support This patch ports the feature Kernel Address SANitizer (KASAN). Note: The start address of shadow memory is at the beginning of kernel space, which is 2^64 - (2^39 / 2) in SV39. The size of the kernel space is 2^38 bytes so the size of shadow memory should be 2^38 / 8. Thus, the shadow memory would not overlap with the fixmap area. There are currently two limitations in this port, 1. RV64 only: KASAN need large address space for extra shadow memory region. 2. KASAN can't debug the modules since the modules are allocated in VMALLOC area. We mapped the shadow memory, which corresponding to VMALLOC area, to the kasan_early_shadow_page because we don't have enough physical space for all the shadow memory corresponding to VMALLOC area. Signed-off-by: Nick Hu <nickhu@andestech.com> Reported-by: Greentime Hu <green.hu@gmail.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-01-06 10:38:32 -08:00
#ifdef CONFIG_KASAN
call kasan_early_init
#endif
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
/* Start the kernel */
riscv: Add SOC early init support Add a mechanism for early SoC initialization for platforms that need additional hardware initialization not possible through the regular device tree and drivers mechanism. With this, a SoC specific initialization function can be called very early, before DTB parsing is done by parse_dtb() in Linux RISC-V kernel setup code. This can be very useful for early hardware initialization for No-MMU kernels booted directly in M-mode because it is quite likely that no other booting stage exist prior to the No-MMU kernel. Example use of a SoC early initialization is as follows: static void vendor_abc_early_init(const void *fdt) { /* * some early init code here that can use simple matches * against the flat device tree file. */ } SOC_EARLY_INIT_DECLARE("vendor,abc", abc_early_init); This early initialization function is executed only if the flat device tree for the board has a 'compatible = "vendor,abc"' entry; Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-16 09:47:38 +09:00
call soc_early_init
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
tail start_kernel
riscv: fix -Wundef warning for CONFIG_RISCV_BOOT_SPINWAIT Since commit 80b6093b55e3 ("kbuild: add -Wundef to KBUILD_CPPFLAGS for W=1 builds"), building with W=1 detects misuse of #if. $ make W=1 ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- arch/riscv/kernel/ [snip] AS arch/riscv/kernel/head.o arch/riscv/kernel/head.S:329:5: warning: "CONFIG_RISCV_BOOT_SPINWAIT" is not defined, evaluates to 0 [-Wundef] 329 | #if CONFIG_RISCV_BOOT_SPINWAIT | ^~~~~~~~~~~~~~~~~~~~~~~~~~ CONFIG_RISCV_BOOT_SPINWAIT is a bool option. #ifdef should be used. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Fixes: 2ffc48fc7071 ("RISC-V: Move spinwait booting method to its own config") Link: https://lore.kernel.org/r/20230106161213.2374093-1-masahiroy@kernel.org Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-01-07 01:12:13 +09:00
#ifdef CONFIG_RISCV_BOOT_SPINWAIT
RISC-V: Move the entire hart selection via lottery to SMP The booting hart selection via lottery is only useful for SMP systems. Moreover, the lottery selection is only necessary for systems using spinwait booting method. It is better to keep the entire lottery selection together so that it can be disabled in future. Move the lottery selection code to under CONFIG_SMP. Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-01-20 01:09:16 -08:00
.Lsecondary_start:
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
/* Set trap vector to spin forever to help debug */
la a3, .Lsecondary_park
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:32 +01:00
csrw CSR_TVEC, a3
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
slli a3, a0, LGREG
RISC-V: Use __cpu_up_stack/task_pointer only for spinwait method The __cpu_up_stack/task_pointer array is only used for spinwait method now. The per cpu array based lookup is also fragile for platforms with discontiguous/sparse hartids. The spinwait method is only used for M-mode Linux or older firmwares without SBI HSM extension. For general Linux systems, ordered booting method is preferred anyways to support cpu hotplug and kexec. Make sure that __cpu_up_stack/task_pointer is only used for spinwait method. Take this opportunity to rename it to __cpu_spinwait_stack/task_pointer to emphasize the purpose as well. Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-01-20 01:09:15 -08:00
la a1, __cpu_spinwait_stack_pointer
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
XIP_FIXUP_OFFSET a1
RISC-V: Use __cpu_up_stack/task_pointer only for spinwait method The __cpu_up_stack/task_pointer array is only used for spinwait method now. The per cpu array based lookup is also fragile for platforms with discontiguous/sparse hartids. The spinwait method is only used for M-mode Linux or older firmwares without SBI HSM extension. For general Linux systems, ordered booting method is preferred anyways to support cpu hotplug and kexec. Make sure that __cpu_up_stack/task_pointer is only used for spinwait method. Take this opportunity to rename it to __cpu_spinwait_stack/task_pointer to emphasize the purpose as well. Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-01-20 01:09:15 -08:00
la a2, __cpu_spinwait_task_pointer
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 02:35:14 -04:00
XIP_FIXUP_OFFSET a2
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
add a1, a3, a1
add a2, a3, a2
/*
* This hart didn't win the lottery, so we wait for the winning hart to
* get far enough along the boot process that it should continue.
*/
.Lwait_for_cpu_up:
/* FIXME: We should WFI to save some energy here. */
REG_L sp, (a1)
REG_L tp, (a2)
beqz sp, .Lwait_for_cpu_up
beqz tp, .Lwait_for_cpu_up
fence
riscv: head: make secondary_start_common() static There are no users outside head.S so make secondary_start_common() static. Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2021-11-29 00:07:38 +08:00
tail .Lsecondary_start_common
RISC-V: Move spinwait booting method to its own config The spinwait booting method should only be used for platforms with older firmware without SBI HSM extension or M-mode firmware because spinwait method can't support cpu hotplug, kexec or sparse hartid. It is better to move the entire spinwait implementation to its own config which can be disabled if required. It is enabled by default to maintain backward compatibility and M-mode Linux. Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-01-20 01:09:17 -08:00
#endif /* CONFIG_RISCV_BOOT_SPINWAIT */
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
riscv: Use SYM_*() assembly macros instead of deprecated ones ENTRY()/END()/WEAK() macros are deprecated and we should make use of the new SYM_*() macros [1] for better annotation of symbols. Replace the deprecated ones with the new ones and fix wrong usage of END()/ENDPROC() to correctly describe the symbols. [1] https://docs.kernel.org/core-api/asm-annotations.html Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-3-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:52 +02:00
SYM_CODE_END(_start_kernel)
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:40 +01:00
#ifdef CONFIG_RISCV_M_MODE
riscv: Use SYM_*() assembly macros instead of deprecated ones ENTRY()/END()/WEAK() macros are deprecated and we should make use of the new SYM_*() macros [1] for better annotation of symbols. Replace the deprecated ones with the new ones and fix wrong usage of END()/ENDPROC() to correctly describe the symbols. [1] https://docs.kernel.org/core-api/asm-annotations.html Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-3-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:52 +02:00
SYM_CODE_START_LOCAL(reset_regs)
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:40 +01:00
li sp, 0
li gp, 0
li tp, 0
li t0, 0
li t1, 0
li t2, 0
li s0, 0
li s1, 0
li a2, 0
li a3, 0
li a4, 0
li a5, 0
li a6, 0
li a7, 0
li s2, 0
li s3, 0
li s4, 0
li s5, 0
li s6, 0
li s7, 0
li s8, 0
li s9, 0
li s10, 0
li s11, 0
li t3, 0
li t4, 0
li t5, 0
li t6, 0
riscv: fix scratch register clearing in M-mode. This patch fixes that the sscratch register clearing in M-mode. It cleared sscratch register in M-mode, but it should clear mscratch register. That will cause kernel trap if the CPU core doesn't support S-mode when trying to access sscratch. Fixes: 9e80635619b5 ("riscv: clear the instruction cache and all registers when booting") Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-12-19 14:44:59 +08:00
csrw CSR_SCRATCH, 0
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:40 +01:00
#ifdef CONFIG_FPU
csrr t0, CSR_MISA
andi t0, t0, (COMPAT_HWCAP_ISA_F | COMPAT_HWCAP_ISA_D)
riscv: Clear vector regfile on bootup clear vector registers on boot if kernel supports V. Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Vineet Gupta <vineetg@rivosinc.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Acked-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Tested-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Reviewed-by: Palmer Dabbelt <palmer@rivosinc.com> Link: https://lore.kernel.org/r/20230605110724.21391-6-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 11:07:02 +00:00
beqz t0, .Lreset_regs_done_fpu
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:40 +01:00
li t1, SR_FS
csrs CSR_STATUS, t1
fmv.s.x f0, zero
fmv.s.x f1, zero
fmv.s.x f2, zero
fmv.s.x f3, zero
fmv.s.x f4, zero
fmv.s.x f5, zero
fmv.s.x f6, zero
fmv.s.x f7, zero
fmv.s.x f8, zero
fmv.s.x f9, zero
fmv.s.x f10, zero
fmv.s.x f11, zero
fmv.s.x f12, zero
fmv.s.x f13, zero
fmv.s.x f14, zero
fmv.s.x f15, zero
fmv.s.x f16, zero
fmv.s.x f17, zero
fmv.s.x f18, zero
fmv.s.x f19, zero
fmv.s.x f20, zero
fmv.s.x f21, zero
fmv.s.x f22, zero
fmv.s.x f23, zero
fmv.s.x f24, zero
fmv.s.x f25, zero
fmv.s.x f26, zero
fmv.s.x f27, zero
fmv.s.x f28, zero
fmv.s.x f29, zero
fmv.s.x f30, zero
fmv.s.x f31, zero
csrw fcsr, 0
/* note that the caller must clear SR_FS */
riscv: Clear vector regfile on bootup clear vector registers on boot if kernel supports V. Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Vineet Gupta <vineetg@rivosinc.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Acked-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Tested-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Reviewed-by: Palmer Dabbelt <palmer@rivosinc.com> Link: https://lore.kernel.org/r/20230605110724.21391-6-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 11:07:02 +00:00
.Lreset_regs_done_fpu:
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:40 +01:00
#endif /* CONFIG_FPU */
riscv: Clear vector regfile on bootup clear vector registers on boot if kernel supports V. Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Vineet Gupta <vineetg@rivosinc.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Acked-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Tested-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Reviewed-by: Palmer Dabbelt <palmer@rivosinc.com> Link: https://lore.kernel.org/r/20230605110724.21391-6-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 11:07:02 +00:00
#ifdef CONFIG_RISCV_ISA_V
csrr t0, CSR_MISA
li t1, COMPAT_HWCAP_ISA_V
and t0, t0, t1
beqz t0, .Lreset_regs_done_vector
/*
* Clear vector registers and reset vcsr
* VLMAX has a defined value, VLEN is a constant,
* and this form of vsetvli is defined to set vl to VLMAX.
*/
li t1, SR_VS
csrs CSR_STATUS, t1
csrs CSR_VCSR, x0
vsetvli t1, x0, e8, m8, ta, ma
vmv.v.i v0, 0
vmv.v.i v8, 0
vmv.v.i v16, 0
vmv.v.i v24, 0
/* note that the caller must clear SR_VS */
.Lreset_regs_done_vector:
#endif /* CONFIG_RISCV_ISA_V */
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:40 +01:00
ret
riscv: Use SYM_*() assembly macros instead of deprecated ones ENTRY()/END()/WEAK() macros are deprecated and we should make use of the new SYM_*() macros [1] for better annotation of symbols. Replace the deprecated ones with the new ones and fix wrong usage of END()/ENDPROC() to correctly describe the symbols. [1] https://docs.kernel.org/core-api/asm-annotations.html Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20231024132655.730417-3-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-24 15:26:52 +02:00
SYM_CODE_END(reset_regs)
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 13:10:40 +01:00
#endif /* CONFIG_RISCV_M_MODE */
Reference in New Issue Copy Permalink