iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
81291383ff
linux/arch/powerpc/kernel/align.c

365 lines
8.5 KiB
C
Raw Normal View History

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license as published by the free software foundation either version 2 of the license or at your option any later version extracted by the scancode license scanner the SPDX license identifier GPL-2.0-or-later has been chosen to replace the boilerplate/reference in 3029 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190527070032.746973796@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-27 09:55:01 +03:00
// SPDX-License-Identifier: GPL-2.0-or-later
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 02:20:36 +04:00
/* align.c - handle alignment exceptions for the Power PC.
*
* Copyright (c) 1996 Paul Mackerras <paulus@cs.anu.edu.au>
* Copyright (c) 1998-1999 TiVo, Inc.
* PowerPC 403GCX modifications.
* Copyright (c) 1999 Grant Erickson <grant@lcse.umn.edu>
* PowerPC 403GCX/405GP modifications.
* Copyright (c) 2001-2002 PPC64 team, IBM Corp
* 64-bit and Power4 support
[PATCH] powerpc: merge align.c This patch merges align.c, the result isn't quite what was in ppc64 nor what was in ppc32 :) It should implement all the functionalities of both though. Kumar, since you played with that in the past, I suppose you have some test cases for verifying that it works properly before I dig out the 601 machine ? :) Since it's likely that I won't be able to test all scenario, code inspection is much welcome. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-18 06:09:41 +03:00
* Copyright (c) 2005 Benjamin Herrenschmidt, IBM Corp
* <benh@kernel.crashing.org>
* Merge ppc32 and ppc64 implementations
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 02:20:36 +04:00
*/
#include <linux/kernel.h>
#include <linux/mm.h>
#include <asm/processor.h>
Replace <asm/uaccess.h> with <linux/uaccess.h> globally This was entirely automated, using the script by Al: PATT='^[[:blank:]]*#[[:blank:]]*include[[:blank:]]*<asm/uaccess.h>' sed -i -e "s!$PATT!#include <linux/uaccess.h>!" \ $(git grep -l "$PATT"|grep -v ^include/linux/uaccess.h) to do the replacement at the end of the merge window. Requested-by: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-12-24 22:46:01 +03:00
#include <linux/uaccess.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 02:20:36 +04:00
#include <asm/cache.h>
#include <asm/cputable.h>
powerpc: Keep track of emulated instructions If CONFIG_PPC_EMULATED_STATS is enabled, make available counters for the various classes of emulated instructions under /sys/kernel/debug/powerpc/emulated_instructions/ (assumed debugfs is mounted on /sys/kernel/debug). Optionally (controlled by /sys/kernel/debug/powerpc/emulated_instructions/do_warn), rate-limited warnings can be printed to the console when instructions are emulated. Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-05-18 06:10:05 +04:00
#include <asm/emulated_ops.h>
Disintegrate asm/system.h for PowerPC Disintegrate asm/system.h for PowerPC. Signed-off-by: David Howells <dhowells@redhat.com> Acked-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> cc: linuxppc-dev@lists.ozlabs.org
2012-03-28 21:30:02 +04:00
#include <asm/switch_to.h>
KVM: PPC: BOOK3S: Remove open coded make_dsisr in alignment handler Use make_dsisr instead of open coding it. This also have the added benefit of handling alignment interrupt on additional instructions. Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> Signed-off-by: Alexander Graf <agraf@suse.de>
2014-05-12 15:34:06 +04:00
#include <asm/disassemble.h>
powerpc: Move cpu_has_feature() to a separate file We plan to use jump label for cpu_has_feature(). In order to implement this we need to include the linux/jump_label.h in asm/cputable.h. Unfortunately if we do that it leads to an include loop. The root of the problem seems to be that reg.h needs cputable.h (for CPU_FTRs), and then cputable.h via jump_label.h eventually pulls in hw_irq.h which needs reg.h (for MSR_EE). So move cpu_has_feature() to a separate file on its own. Signed-off-by: Kevin Hao <haokexin@gmail.com> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> [mpe: Rename to cpu_has_feature.h and flesh out change log] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2016-07-23 12:12:40 +03:00
#include <asm/cpu_has_feature.h>
powerpc: Use instruction emulation infrastructure to handle alignment faults This replaces almost all of the instruction emulation code in fix_alignment() with calls to analyse_instr(), emulate_loadstore() and emulate_dcbz(). The only emulation code left is the SPE emulation code; analyse_instr() etc. do not handle SPE instructions at present. One result of this is that we can now handle alignment faults on all the new VSX load and store instructions that were added in POWER9. VSX loads/stores will take alignment faults for unaligned accesses to cache-inhibited memory. Another effect is that we no longer rely on the DAR and DSISR values set by the processor. With this, we now need to include the instruction emulation code unconditionally. Signed-off-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-08-30 07:12:40 +03:00
#include <asm/sstep.h>
powerpc: Use a macro for creating instructions from u32s In preparation for instructions having a more complex data type start using a macro, ppc_inst(), for making an instruction out of a u32. A macro is used so that instructions can be used as initializer elements. Currently this does nothing, but it will allow for creating a data type that can represent prefixed instructions. Signed-off-by: Jordan Niethe <jniethe5@gmail.com> [mpe: Change include guard to _ASM_POWERPC_INST_H] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Reviewed-by: Alistair Popple <alistair@popple.id.au> Link: https://lore.kernel.org/r/20200506034050.24806-7-jniethe5@gmail.com
2020-05-06 06:40:26 +03:00
#include <asm/inst.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 02:20:36 +04:00
struct aligninfo {
unsigned char len;
unsigned char flags;
};
#define INVALID { 0, 0 }
[PATCH] powerpc: Implement support for setting little-endian mode via prctl This adds the PowerPC part of the code to allow processes to change their endian mode via prctl. This also extends the alignment exception handler to be able to fix up alignment exceptions that occur in little-endian mode, both for "PowerPC" little-endian and true little-endian. We always enter signal handlers in big-endian mode -- the support for little-endian mode does not amount to the creation of a little-endian user/kernel ABI. If the signal handler returns, the endian mode is restored to what it was when the signal was delivered. We have two new kernel CPU feature bits, one for PPC little-endian and one for true little-endian. Most of the classic 32-bit processors support PPC little-endian, and this is reflected in the CPU feature table. There are two corresponding feature bits reported to userland in the AT_HWCAP aux vector entry. This is based on an earlier patch by Anton Blanchard. Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-06-07 10:14:40 +04:00
/* Bits in the flags field */
#define LD 0 /* load */
#define ST 1 /* store */
[POWERPC] Handle alignment faults on new FP load/store instructions This adds code to handle alignment traps generated by the following new floating-point load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): lfiwax load floating-point as integer word algebraic indexed stfiwx store floating-point as integer word indexed lfdp load floating-point double pair lfdpx load floating-point double pair indexed stfdp store floating-point double pair stfdpx store floating-point double pair indexed All these except stfiwx are new in POWER6. lfdp/lfdpx/stfdp/stfdpx load and store 16 bytes of memory into an even/odd FP register pair. In little-endian mode each 8-byte value is byte-reversed separately (i.e. not as a 16-byte unit). lfiwax/stfiwx load or store the lower 4 bytes of a floating-point register from/to memory; lfiwax sets the upper 4 bytes of the FP register to the sign extension of the value loaded. Signed-off-by: Paul Mackerras <paulus@samba.org>
2007-08-10 08:07:38 +04:00
#define SE 2 /* sign-extend value, or FP ld/st as word */
[PATCH] powerpc: Implement support for setting little-endian mode via prctl This adds the PowerPC part of the code to allow processes to change their endian mode via prctl. This also extends the alignment exception handler to be able to fix up alignment exceptions that occur in little-endian mode, both for "PowerPC" little-endian and true little-endian. We always enter signal handlers in big-endian mode -- the support for little-endian mode does not amount to the creation of a little-endian user/kernel ABI. If the signal handler returns, the endian mode is restored to what it was when the signal was delivered. We have two new kernel CPU feature bits, one for PPC little-endian and one for true little-endian. Most of the classic 32-bit processors support PPC little-endian, and this is reflected in the CPU feature table. There are two corresponding feature bits reported to userland in the AT_HWCAP aux vector entry. This is based on an earlier patch by Anton Blanchard. Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-06-07 10:14:40 +04:00
#define SW 0x20 /* byte swap */
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
#define E4 0x40 /* SPE endianness is word */
#define E8 0x80 /* SPE endianness is double word */
powerpc: Add lq/stq emulation Recent CPUs support quad word load and store instructions. Add support to the alignment handler for them. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-03-28 10:01:23 +04:00
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
#ifdef CONFIG_SPE
static struct aligninfo spe_aligninfo[32] = {
{ 8, LD+E8 }, /* 0 00 00: evldd[x] */
{ 8, LD+E4 }, /* 0 00 01: evldw[x] */
{ 8, LD }, /* 0 00 10: evldh[x] */
INVALID, /* 0 00 11 */
{ 2, LD }, /* 0 01 00: evlhhesplat[x] */
INVALID, /* 0 01 01 */
{ 2, LD }, /* 0 01 10: evlhhousplat[x] */
{ 2, LD+SE }, /* 0 01 11: evlhhossplat[x] */
{ 4, LD }, /* 0 10 00: evlwhe[x] */
INVALID, /* 0 10 01 */
{ 4, LD }, /* 0 10 10: evlwhou[x] */
{ 4, LD+SE }, /* 0 10 11: evlwhos[x] */
{ 4, LD+E4 }, /* 0 11 00: evlwwsplat[x] */
INVALID, /* 0 11 01 */
{ 4, LD }, /* 0 11 10: evlwhsplat[x] */
INVALID, /* 0 11 11 */
{ 8, ST+E8 }, /* 1 00 00: evstdd[x] */
{ 8, ST+E4 }, /* 1 00 01: evstdw[x] */
{ 8, ST }, /* 1 00 10: evstdh[x] */
INVALID, /* 1 00 11 */
INVALID, /* 1 01 00 */
INVALID, /* 1 01 01 */
INVALID, /* 1 01 10 */
INVALID, /* 1 01 11 */
{ 4, ST }, /* 1 10 00: evstwhe[x] */
INVALID, /* 1 10 01 */
{ 4, ST }, /* 1 10 10: evstwho[x] */
INVALID, /* 1 10 11 */
{ 4, ST+E4 }, /* 1 11 00: evstwwe[x] */
INVALID, /* 1 11 01 */
{ 4, ST+E4 }, /* 1 11 10: evstwwo[x] */
INVALID, /* 1 11 11 */
};
#define EVLDD 0x00
#define EVLDW 0x01
#define EVLDH 0x02
#define EVLHHESPLAT 0x04
#define EVLHHOUSPLAT 0x06
#define EVLHHOSSPLAT 0x07
#define EVLWHE 0x08
#define EVLWHOU 0x0A
#define EVLWHOS 0x0B
#define EVLWWSPLAT 0x0C
#define EVLWHSPLAT 0x0E
#define EVSTDD 0x10
#define EVSTDW 0x11
#define EVSTDH 0x12
#define EVSTWHE 0x18
#define EVSTWHO 0x1A
#define EVSTWWE 0x1C
#define EVSTWWO 0x1E
/*
* Emulate SPE loads and stores.
* Only Book-E has these instructions, and it does true little-endian,
* so we don't need the address swizzling.
*/
static int emulate_spe(struct pt_regs *regs, unsigned int reg,
powerpc: Use a datatype for instructions Currently unsigned ints are used to represent instructions on powerpc. This has worked well as instructions have always been 4 byte words. However, ISA v3.1 introduces some changes to instructions that mean this scheme will no longer work as well. This change is Prefixed Instructions. A prefixed instruction is made up of a word prefix followed by a word suffix to make an 8 byte double word instruction. No matter the endianness of the system the prefix always comes first. Prefixed instructions are only planned for powerpc64. Introduce a ppc_inst type to represent both prefixed and word instructions on powerpc64 while keeping it possible to exclusively have word instructions on powerpc32. Signed-off-by: Jordan Niethe <jniethe5@gmail.com> [mpe: Fix compile error in emulate_spe()] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20200506034050.24806-12-jniethe5@gmail.com
2020-05-06 06:40:31 +03:00
struct ppc_inst ppc_instr)
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
{
union {
u64 ll;
u32 w[2];
u16 h[4];
u8 v[8];
} data, temp;
unsigned char __user *p, *addr;
unsigned long *evr = &current->thread.evr[reg];
powerpc: Use a datatype for instructions Currently unsigned ints are used to represent instructions on powerpc. This has worked well as instructions have always been 4 byte words. However, ISA v3.1 introduces some changes to instructions that mean this scheme will no longer work as well. This change is Prefixed Instructions. A prefixed instruction is made up of a word prefix followed by a word suffix to make an 8 byte double word instruction. No matter the endianness of the system the prefix always comes first. Prefixed instructions are only planned for powerpc64. Introduce a ppc_inst type to represent both prefixed and word instructions on powerpc64 while keeping it possible to exclusively have word instructions on powerpc32. Signed-off-by: Jordan Niethe <jniethe5@gmail.com> [mpe: Fix compile error in emulate_spe()] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20200506034050.24806-12-jniethe5@gmail.com
2020-05-06 06:40:31 +03:00
unsigned int nb, flags, instr;
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
powerpc: Use a datatype for instructions Currently unsigned ints are used to represent instructions on powerpc. This has worked well as instructions have always been 4 byte words. However, ISA v3.1 introduces some changes to instructions that mean this scheme will no longer work as well. This change is Prefixed Instructions. A prefixed instruction is made up of a word prefix followed by a word suffix to make an 8 byte double word instruction. No matter the endianness of the system the prefix always comes first. Prefixed instructions are only planned for powerpc64. Introduce a ppc_inst type to represent both prefixed and word instructions on powerpc64 while keeping it possible to exclusively have word instructions on powerpc32. Signed-off-by: Jordan Niethe <jniethe5@gmail.com> [mpe: Fix compile error in emulate_spe()] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20200506034050.24806-12-jniethe5@gmail.com
2020-05-06 06:40:31 +03:00
instr = ppc_inst_val(ppc_instr);
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
instr = (instr >> 1) & 0x1f;
/* DAR has the operand effective address */
addr = (unsigned char __user *)regs->dar;
nb = spe_aligninfo[instr].len;
flags = spe_aligninfo[instr].flags;
/* userland only */
if (unlikely(!user_mode(regs)))
return 0;
flush_spe_to_thread(current);
/* If we are loading, get the data from user space, else
* get it from register values
*/
if (flags & ST) {
data.ll = 0;
switch (instr) {
case EVSTDD:
case EVSTDW:
case EVSTDH:
data.w[0] = *evr;
data.w[1] = regs->gpr[reg];
break;
case EVSTWHE:
data.h[2] = *evr >> 16;
data.h[3] = regs->gpr[reg] >> 16;
break;
case EVSTWHO:
data.h[2] = *evr & 0xffff;
data.h[3] = regs->gpr[reg] & 0xffff;
break;
case EVSTWWE:
data.w[1] = *evr;
break;
case EVSTWWO:
data.w[1] = regs->gpr[reg];
break;
default:
return -EINVAL;
}
} else {
temp.ll = data.ll = 0;
p = addr;
powerpc/align: Convert emulate_spe() to user_access_begin This patch converts emulate_spe() to using user_access_begin logic. Since commit 662bbcb2747c ("mm, sched: Allow uaccess in atomic with pagefault_disable()"), might_fault() doesn't fire when called from sections where pagefaults are disabled, which must be the case when using _inatomic variants of __get_user and __put_user. So the might_fault() in user_access_begin() is not a problem. There was a verification of user_mode() together with the access_ok(), but there is a second verification of user_mode() just after, that leads to immediate return. The access_ok() is now part of the user_access_begin which is called after that other user_mode() verification, so no need to check user_mode() again. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/c95a648fdf75992c9d88f3c73cc23e7537fcf2ad.1615555354.git.christophe.leroy@csgroup.eu
2021-03-12 16:25:11 +03:00
if (!user_read_access_begin(addr, nb))
return -EFAULT;
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
switch (nb) {
case 8:
powerpc/align: Convert emulate_spe() to user_access_begin This patch converts emulate_spe() to using user_access_begin logic. Since commit 662bbcb2747c ("mm, sched: Allow uaccess in atomic with pagefault_disable()"), might_fault() doesn't fire when called from sections where pagefaults are disabled, which must be the case when using _inatomic variants of __get_user and __put_user. So the might_fault() in user_access_begin() is not a problem. There was a verification of user_mode() together with the access_ok(), but there is a second verification of user_mode() just after, that leads to immediate return. The access_ok() is now part of the user_access_begin which is called after that other user_mode() verification, so no need to check user_mode() again. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/c95a648fdf75992c9d88f3c73cc23e7537fcf2ad.1615555354.git.christophe.leroy@csgroup.eu
2021-03-12 16:25:11 +03:00
unsafe_get_user(temp.v[0], p++, Efault_read);
unsafe_get_user(temp.v[1], p++, Efault_read);
unsafe_get_user(temp.v[2], p++, Efault_read);
unsafe_get_user(temp.v[3], p++, Efault_read);
powerpc: Use fallthrough pseudo-keyword Replace the existing /* fall through */ comments and its variants with the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary fall-through markings when it is the case. [1] https://www.kernel.org/doc/html/v5.7/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20200727224201.GA10133@embeddedor
2020-07-28 01:42:01 +03:00
fallthrough;
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
case 4:
powerpc/align: Convert emulate_spe() to user_access_begin This patch converts emulate_spe() to using user_access_begin logic. Since commit 662bbcb2747c ("mm, sched: Allow uaccess in atomic with pagefault_disable()"), might_fault() doesn't fire when called from sections where pagefaults are disabled, which must be the case when using _inatomic variants of __get_user and __put_user. So the might_fault() in user_access_begin() is not a problem. There was a verification of user_mode() together with the access_ok(), but there is a second verification of user_mode() just after, that leads to immediate return. The access_ok() is now part of the user_access_begin which is called after that other user_mode() verification, so no need to check user_mode() again. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/c95a648fdf75992c9d88f3c73cc23e7537fcf2ad.1615555354.git.christophe.leroy@csgroup.eu
2021-03-12 16:25:11 +03:00
unsafe_get_user(temp.v[4], p++, Efault_read);
unsafe_get_user(temp.v[5], p++, Efault_read);
powerpc: Use fallthrough pseudo-keyword Replace the existing /* fall through */ comments and its variants with the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary fall-through markings when it is the case. [1] https://www.kernel.org/doc/html/v5.7/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20200727224201.GA10133@embeddedor
2020-07-28 01:42:01 +03:00
fallthrough;
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
case 2:
powerpc/align: Convert emulate_spe() to user_access_begin This patch converts emulate_spe() to using user_access_begin logic. Since commit 662bbcb2747c ("mm, sched: Allow uaccess in atomic with pagefault_disable()"), might_fault() doesn't fire when called from sections where pagefaults are disabled, which must be the case when using _inatomic variants of __get_user and __put_user. So the might_fault() in user_access_begin() is not a problem. There was a verification of user_mode() together with the access_ok(), but there is a second verification of user_mode() just after, that leads to immediate return. The access_ok() is now part of the user_access_begin which is called after that other user_mode() verification, so no need to check user_mode() again. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/c95a648fdf75992c9d88f3c73cc23e7537fcf2ad.1615555354.git.christophe.leroy@csgroup.eu
2021-03-12 16:25:11 +03:00
unsafe_get_user(temp.v[6], p++, Efault_read);
unsafe_get_user(temp.v[7], p++, Efault_read);
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
}
powerpc/align: Convert emulate_spe() to user_access_begin This patch converts emulate_spe() to using user_access_begin logic. Since commit 662bbcb2747c ("mm, sched: Allow uaccess in atomic with pagefault_disable()"), might_fault() doesn't fire when called from sections where pagefaults are disabled, which must be the case when using _inatomic variants of __get_user and __put_user. So the might_fault() in user_access_begin() is not a problem. There was a verification of user_mode() together with the access_ok(), but there is a second verification of user_mode() just after, that leads to immediate return. The access_ok() is now part of the user_access_begin which is called after that other user_mode() verification, so no need to check user_mode() again. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/c95a648fdf75992c9d88f3c73cc23e7537fcf2ad.1615555354.git.christophe.leroy@csgroup.eu
2021-03-12 16:25:11 +03:00
user_read_access_end();
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
switch (instr) {
case EVLDD:
case EVLDW:
case EVLDH:
data.ll = temp.ll;
break;
case EVLHHESPLAT:
data.h[0] = temp.h[3];
data.h[2] = temp.h[3];
break;
case EVLHHOUSPLAT:
case EVLHHOSSPLAT:
data.h[1] = temp.h[3];
data.h[3] = temp.h[3];
break;
case EVLWHE:
data.h[0] = temp.h[2];
data.h[2] = temp.h[3];
break;
case EVLWHOU:
case EVLWHOS:
data.h[1] = temp.h[2];
data.h[3] = temp.h[3];
break;
case EVLWWSPLAT:
data.w[0] = temp.w[1];
data.w[1] = temp.w[1];
break;
case EVLWHSPLAT:
data.h[0] = temp.h[2];
data.h[1] = temp.h[2];
data.h[2] = temp.h[3];
data.h[3] = temp.h[3];
break;
default:
return -EINVAL;
}
}
if (flags & SW) {
switch (flags & 0xf0) {
case E8:
powerpc: Remove open coded byte swap macro in alignment handler Use swab64/32/16 instead of open coding it. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2013-09-23 06:04:46 +04:00
data.ll = swab64(data.ll);
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
break;
case E4:
powerpc: Remove open coded byte swap macro in alignment handler Use swab64/32/16 instead of open coding it. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2013-09-23 06:04:46 +04:00
data.w[0] = swab32(data.w[0]);
data.w[1] = swab32(data.w[1]);
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
break;
/* Its half word endian */
default:
powerpc: Remove open coded byte swap macro in alignment handler Use swab64/32/16 instead of open coding it. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2013-09-23 06:04:46 +04:00
data.h[0] = swab16(data.h[0]);
data.h[1] = swab16(data.h[1]);
data.h[2] = swab16(data.h[2]);
data.h[3] = swab16(data.h[3]);
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
break;
}
}
if (flags & SE) {
data.w[0] = (s16)data.h[1];
data.w[1] = (s16)data.h[3];
}
/* Store result to memory or update registers */
if (flags & ST) {
p = addr;
powerpc/align: Convert emulate_spe() to user_access_begin This patch converts emulate_spe() to using user_access_begin logic. Since commit 662bbcb2747c ("mm, sched: Allow uaccess in atomic with pagefault_disable()"), might_fault() doesn't fire when called from sections where pagefaults are disabled, which must be the case when using _inatomic variants of __get_user and __put_user. So the might_fault() in user_access_begin() is not a problem. There was a verification of user_mode() together with the access_ok(), but there is a second verification of user_mode() just after, that leads to immediate return. The access_ok() is now part of the user_access_begin which is called after that other user_mode() verification, so no need to check user_mode() again. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/c95a648fdf75992c9d88f3c73cc23e7537fcf2ad.1615555354.git.christophe.leroy@csgroup.eu
2021-03-12 16:25:11 +03:00
if (!user_write_access_begin(addr, nb))
return -EFAULT;
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
switch (nb) {
case 8:
powerpc/align: Convert emulate_spe() to user_access_begin This patch converts emulate_spe() to using user_access_begin logic. Since commit 662bbcb2747c ("mm, sched: Allow uaccess in atomic with pagefault_disable()"), might_fault() doesn't fire when called from sections where pagefaults are disabled, which must be the case when using _inatomic variants of __get_user and __put_user. So the might_fault() in user_access_begin() is not a problem. There was a verification of user_mode() together with the access_ok(), but there is a second verification of user_mode() just after, that leads to immediate return. The access_ok() is now part of the user_access_begin which is called after that other user_mode() verification, so no need to check user_mode() again. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/c95a648fdf75992c9d88f3c73cc23e7537fcf2ad.1615555354.git.christophe.leroy@csgroup.eu
2021-03-12 16:25:11 +03:00
unsafe_put_user(data.v[0], p++, Efault_write);
unsafe_put_user(data.v[1], p++, Efault_write);
unsafe_put_user(data.v[2], p++, Efault_write);
unsafe_put_user(data.v[3], p++, Efault_write);
powerpc: Use fallthrough pseudo-keyword Replace the existing /* fall through */ comments and its variants with the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary fall-through markings when it is the case. [1] https://www.kernel.org/doc/html/v5.7/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20200727224201.GA10133@embeddedor
2020-07-28 01:42:01 +03:00
fallthrough;
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
case 4:
powerpc/align: Convert emulate_spe() to user_access_begin This patch converts emulate_spe() to using user_access_begin logic. Since commit 662bbcb2747c ("mm, sched: Allow uaccess in atomic with pagefault_disable()"), might_fault() doesn't fire when called from sections where pagefaults are disabled, which must be the case when using _inatomic variants of __get_user and __put_user. So the might_fault() in user_access_begin() is not a problem. There was a verification of user_mode() together with the access_ok(), but there is a second verification of user_mode() just after, that leads to immediate return. The access_ok() is now part of the user_access_begin which is called after that other user_mode() verification, so no need to check user_mode() again. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/c95a648fdf75992c9d88f3c73cc23e7537fcf2ad.1615555354.git.christophe.leroy@csgroup.eu
2021-03-12 16:25:11 +03:00
unsafe_put_user(data.v[4], p++, Efault_write);
unsafe_put_user(data.v[5], p++, Efault_write);
powerpc: Use fallthrough pseudo-keyword Replace the existing /* fall through */ comments and its variants with the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary fall-through markings when it is the case. [1] https://www.kernel.org/doc/html/v5.7/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20200727224201.GA10133@embeddedor
2020-07-28 01:42:01 +03:00
fallthrough;
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
case 2:
powerpc/align: Convert emulate_spe() to user_access_begin This patch converts emulate_spe() to using user_access_begin logic. Since commit 662bbcb2747c ("mm, sched: Allow uaccess in atomic with pagefault_disable()"), might_fault() doesn't fire when called from sections where pagefaults are disabled, which must be the case when using _inatomic variants of __get_user and __put_user. So the might_fault() in user_access_begin() is not a problem. There was a verification of user_mode() together with the access_ok(), but there is a second verification of user_mode() just after, that leads to immediate return. The access_ok() is now part of the user_access_begin which is called after that other user_mode() verification, so no need to check user_mode() again. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/c95a648fdf75992c9d88f3c73cc23e7537fcf2ad.1615555354.git.christophe.leroy@csgroup.eu
2021-03-12 16:25:11 +03:00
unsafe_put_user(data.v[6], p++, Efault_write);
unsafe_put_user(data.v[7], p++, Efault_write);
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
}
powerpc/align: Convert emulate_spe() to user_access_begin This patch converts emulate_spe() to using user_access_begin logic. Since commit 662bbcb2747c ("mm, sched: Allow uaccess in atomic with pagefault_disable()"), might_fault() doesn't fire when called from sections where pagefaults are disabled, which must be the case when using _inatomic variants of __get_user and __put_user. So the might_fault() in user_access_begin() is not a problem. There was a verification of user_mode() together with the access_ok(), but there is a second verification of user_mode() just after, that leads to immediate return. The access_ok() is now part of the user_access_begin which is called after that other user_mode() verification, so no need to check user_mode() again. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/c95a648fdf75992c9d88f3c73cc23e7537fcf2ad.1615555354.git.christophe.leroy@csgroup.eu
2021-03-12 16:25:11 +03:00
user_write_access_end();
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
} else {
*evr = data.w[0];
regs->gpr[reg] = data.w[1];
}
return 1;
powerpc/align: Convert emulate_spe() to user_access_begin This patch converts emulate_spe() to using user_access_begin logic. Since commit 662bbcb2747c ("mm, sched: Allow uaccess in atomic with pagefault_disable()"), might_fault() doesn't fire when called from sections where pagefaults are disabled, which must be the case when using _inatomic variants of __get_user and __put_user. So the might_fault() in user_access_begin() is not a problem. There was a verification of user_mode() together with the access_ok(), but there is a second verification of user_mode() just after, that leads to immediate return. The access_ok() is now part of the user_access_begin which is called after that other user_mode() verification, so no need to check user_mode() again. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/c95a648fdf75992c9d88f3c73cc23e7537fcf2ad.1615555354.git.christophe.leroy@csgroup.eu
2021-03-12 16:25:11 +03:00
Efault_read:
user_read_access_end();
return -EFAULT;
Efault_write:
user_write_access_end();
return -EFAULT;
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
}
#endif /* CONFIG_SPE */
[PATCH] powerpc: merge align.c This patch merges align.c, the result isn't quite what was in ppc64 nor what was in ppc32 :) It should implement all the functionalities of both though. Kumar, since you played with that in the past, I suppose you have some test cases for verifying that it works properly before I dig out the 601 machine ? :) Since it's likely that I won't be able to test all scenario, code inspection is much welcome. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-18 06:09:41 +03:00
/*
* Called on alignment exception. Attempts to fixup
*
* Return 1 on success
* Return 0 if unable to handle the interrupt
* Return -EFAULT if data address is bad
powerpc: Use instruction emulation infrastructure to handle alignment faults This replaces almost all of the instruction emulation code in fix_alignment() with calls to analyse_instr(), emulate_loadstore() and emulate_dcbz(). The only emulation code left is the SPE emulation code; analyse_instr() etc. do not handle SPE instructions at present. One result of this is that we can now handle alignment faults on all the new VSX load and store instructions that were added in POWER9. VSX loads/stores will take alignment faults for unaligned accesses to cache-inhibited memory. Another effect is that we no longer rely on the DAR and DSISR values set by the processor. With this, we now need to include the instruction emulation code unconditionally. Signed-off-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-08-30 07:12:40 +03:00
* Other negative return values indicate that the instruction can't
* be emulated, and the process should be given a SIGBUS.
[PATCH] powerpc: merge align.c This patch merges align.c, the result isn't quite what was in ppc64 nor what was in ppc32 :) It should implement all the functionalities of both though. Kumar, since you played with that in the past, I suppose you have some test cases for verifying that it works properly before I dig out the 601 machine ? :) Since it's likely that I won't be able to test all scenario, code inspection is much welcome. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-18 06:09:41 +03:00
*/
int fix_alignment(struct pt_regs *regs)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 02:20:36 +04:00
{
powerpc: Use a datatype for instructions Currently unsigned ints are used to represent instructions on powerpc. This has worked well as instructions have always been 4 byte words. However, ISA v3.1 introduces some changes to instructions that mean this scheme will no longer work as well. This change is Prefixed Instructions. A prefixed instruction is made up of a word prefix followed by a word suffix to make an 8 byte double word instruction. No matter the endianness of the system the prefix always comes first. Prefixed instructions are only planned for powerpc64. Introduce a ppc_inst type to represent both prefixed and word instructions on powerpc64 while keeping it possible to exclusively have word instructions on powerpc32. Signed-off-by: Jordan Niethe <jniethe5@gmail.com> [mpe: Fix compile error in emulate_spe()] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20200506034050.24806-12-jniethe5@gmail.com
2020-05-06 06:40:31 +03:00
struct ppc_inst instr;
powerpc: Use instruction emulation infrastructure to handle alignment faults This replaces almost all of the instruction emulation code in fix_alignment() with calls to analyse_instr(), emulate_loadstore() and emulate_dcbz(). The only emulation code left is the SPE emulation code; analyse_instr() etc. do not handle SPE instructions at present. One result of this is that we can now handle alignment faults on all the new VSX load and store instructions that were added in POWER9. VSX loads/stores will take alignment faults for unaligned accesses to cache-inhibited memory. Another effect is that we no longer rely on the DAR and DSISR values set by the processor. With this, we now need to include the instruction emulation code unconditionally. Signed-off-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-08-30 07:12:40 +03:00
struct instruction_op op;
int r, type;
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 02:20:36 +04:00
powerpc/align: Don't use __get_user_instr() on kernel addresses In the old days, when we didn't have kernel userspace access protection and had set_fs(), it was wise to use __get_user() and friends to read kernel memory. Nowadays, get_user() is granting userspace access and is exclusively for userspace access. In alignment exception handler, use probe_kernel_read_inst() instead of __get_user_instr() for reading instructions in kernel. This will allow to remove the is_kernel_addr() check in __get/put_user() in a following patch. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/d9ecbce00178484e66ca7adec2ff210058037704.1615398265.git.christophe.leroy@csgroup.eu
2021-03-10 20:46:45 +03:00
if (is_kernel_addr(regs->nip))
powerpc: Rename probe_kernel_read_inst() When probe_kernel_read_inst() was created, it was to mimic probe_kernel_read() function. Since then, probe_kernel_read() has been renamed copy_from_kernel_nofault(). Rename probe_kernel_read_inst() into copy_inst_from_kernel_nofault(). Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/b783d1f7cdb8914992384a669a2af57051b6bdcf.1618405715.git.christophe.leroy@csgroup.eu
2021-04-14 16:08:42 +03:00
r = copy_inst_from_kernel_nofault(&instr, (void *)regs->nip);
powerpc/align: Don't use __get_user_instr() on kernel addresses In the old days, when we didn't have kernel userspace access protection and had set_fs(), it was wise to use __get_user() and friends to read kernel memory. Nowadays, get_user() is granting userspace access and is exclusively for userspace access. In alignment exception handler, use probe_kernel_read_inst() instead of __get_user_instr() for reading instructions in kernel. This will allow to remove the is_kernel_addr() check in __get/put_user() in a following patch. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Reviewed-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/d9ecbce00178484e66ca7adec2ff210058037704.1615398265.git.christophe.leroy@csgroup.eu
2021-03-10 20:46:45 +03:00
else
r = __get_user_instr(instr, (void __user *)regs->nip);
if (unlikely(r))
powerpc: Use instruction emulation infrastructure to handle alignment faults This replaces almost all of the instruction emulation code in fix_alignment() with calls to analyse_instr(), emulate_loadstore() and emulate_dcbz(). The only emulation code left is the SPE emulation code; analyse_instr() etc. do not handle SPE instructions at present. One result of this is that we can now handle alignment faults on all the new VSX load and store instructions that were added in POWER9. VSX loads/stores will take alignment faults for unaligned accesses to cache-inhibited memory. Another effect is that we no longer rely on the DAR and DSISR values set by the processor. With this, we now need to include the instruction emulation code unconditionally. Signed-off-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-08-30 07:12:40 +03:00
return -EFAULT;
if ((regs->msr & MSR_LE) != (MSR_KERNEL & MSR_LE)) {
/* We don't handle PPC little-endian any more... */
if (cpu_has_feature(CPU_FTR_PPC_LE))
return -EIO;
powerpc: Use a function for byte swapping instructions Use a function for byte swapping instructions in preparation of a more complicated instruction type. Signed-off-by: Jordan Niethe <jniethe5@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Reviewed-by: Balamuruhan S <bala24@linux.ibm.com> Link: https://lore.kernel.org/r/20200506034050.24806-10-jniethe5@gmail.com
2020-05-06 06:40:29 +03:00
instr = ppc_inst_swab(instr);
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 02:20:36 +04:00
}
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
#ifdef CONFIG_SPE
powerpc: Use a function for getting the instruction op code In preparation for using a data type for instructions that can not be directly used with the '>>' operator use a function for getting the op code of an instruction. Signed-off-by: Jordan Niethe <jniethe5@gmail.com> Reviewed-by: Alistair Popple <alistair@popple.id.au> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20200506034050.24806-9-jniethe5@gmail.com
2020-05-06 06:40:28 +03:00
if (ppc_inst_primary_opcode(instr) == 0x4) {
powerpc: Use an accessor for instructions In preparation for introducing a more complicated instruction type to accommodate prefixed instructions use an accessor for getting an instruction as a u32. Signed-off-by: Jordan Niethe <jniethe5@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20200506034050.24806-8-jniethe5@gmail.com
2020-05-06 06:40:27 +03:00
int reg = (ppc_inst_val(instr) >> 21) & 0x1f;
powerpc: Create PPC_WARN_ALIGNMENT to match PPC_WARN_EMULATED perf_event wants a separate event for alignment and emulation faults, so create another emulation event. This will make it easy to hook in perf_event at one spot. We pass in regs which will be required for these events. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-27 21:46:55 +03:00
PPC_WARN_ALIGNMENT(spe, regs);
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
return emulate_spe(regs, reg, instr);
powerpc: Keep track of emulated instructions If CONFIG_PPC_EMULATED_STATS is enabled, make available counters for the various classes of emulated instructions under /sys/kernel/debug/powerpc/emulated_instructions/ (assumed debugfs is mounted on /sys/kernel/debug). Optionally (controlled by /sys/kernel/debug/powerpc/emulated_instructions/do_warn), rate-limited warnings can be printed to the console when instructions are emulated. Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-05-18 06:10:05 +04:00
}
[POWERPC] Handle alignment faults on SPE load/store instructions This adds code to handle alignment traps generated by the following SPE (signal processing engine) load/store instructions, by emulating the instruction in the kernel (as is done for other instructions that generate alignment traps): evldd[x] Vector Load Double Word into Double Word [Indexed] evldw[x] Vector Load Double into Two Words [Indexed] evldh[x] Vector Load Double into Four Half Words [Indexed] evlhhesplat[x] Vector Load Half Word into Half Words Even and Splat [Indexed] evlhhousplat[x] Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed] evlhhossplat[x] Vector Load Half Word into Half Word Odd Signed and Splat [Indexed] evlwhe[x] Vector Load Word into Two Half Words Even [Indexed] evlwhou[x] Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed] evlwhos[x] Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed] evlwwsplat[x] Vector Load Word into Word and Splat [Indexed] evlwhsplat[x] Vector Load Word into Two Half Words and Splat [Indexed] evstdd[x] Vector Store Double of Double [Indexed] evstdw[x] Vector Store Double of Two Words [Indexed] evstdh[x] Vector Store Double of Four Half Words [Indexed] evstwhe[x] Vector Store Word of Two Half Words from Even [Indexed] evstwho[x] Vector Store Word of Two Half Words from Odd [Indexed] evstwwe[x] Vector Store Word of Word from Even [Indexed] evstwwo[x] Vector Store Word of Word from Odd [Indexed] Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
2007-08-25 01:42:53 +04:00
#endif
powerpc: Send SIGBUS on unaligned copy and paste Calling ISA 3.0 instructions copy, copy_first, paste and paste_last generates an alignment fault when copying or pasting unaligned data (128 byte). We catch this and send SIGBUS to the userspace process that caused it. We do not emulate these because paste may contain additional metadata when pasting to a co-processor and paste_last is the synchronisation point for preceding copy/paste sequences. Thanks to Michael Neuling <mikey@neuling.org> for his help. Signed-off-by: Chris Smart <chris@distroguy.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2016-06-17 02:33:45 +03:00
/*
* ISA 3.0 (such as P9) copy, copy_first, paste and paste_last alignment
* check.
*
* Send a SIGBUS to the process that caused the fault.
*
* We do not emulate these because paste may contain additional metadata
* when pasting to a co-processor. Furthermore, paste_last is the
* synchronisation point for preceding copy/paste sequences.
*/
powerpc: Use an accessor for instructions In preparation for introducing a more complicated instruction type to accommodate prefixed instructions use an accessor for getting an instruction as a u32. Signed-off-by: Jordan Niethe <jniethe5@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20200506034050.24806-8-jniethe5@gmail.com
2020-05-06 06:40:27 +03:00
if ((ppc_inst_val(instr) & 0xfc0006fe) == (PPC_INST_COPY & 0xfc0006fe))
powerpc: Send SIGBUS on unaligned copy and paste Calling ISA 3.0 instructions copy, copy_first, paste and paste_last generates an alignment fault when copying or pasting unaligned data (128 byte). We catch this and send SIGBUS to the userspace process that caused it. We do not emulate these because paste may contain additional metadata when pasting to a co-processor and paste_last is the synchronisation point for preceding copy/paste sequences. Thanks to Michael Neuling <mikey@neuling.org> for his help. Signed-off-by: Chris Smart <chris@distroguy.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2016-06-17 02:33:45 +03:00
return -EIO;
powerpc: Use instruction emulation infrastructure to handle alignment faults This replaces almost all of the instruction emulation code in fix_alignment() with calls to analyse_instr(), emulate_loadstore() and emulate_dcbz(). The only emulation code left is the SPE emulation code; analyse_instr() etc. do not handle SPE instructions at present. One result of this is that we can now handle alignment faults on all the new VSX load and store instructions that were added in POWER9. VSX loads/stores will take alignment faults for unaligned accesses to cache-inhibited memory. Another effect is that we no longer rely on the DAR and DSISR values set by the processor. With this, we now need to include the instruction emulation code unconditionally. Signed-off-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-08-30 07:12:40 +03:00
r = analyse_instr(&op, regs, instr);
if (r < 0)
return -EINVAL;
[PATCH] powerpc: merge align.c This patch merges align.c, the result isn't quite what was in ppc64 nor what was in ppc32 :) It should implement all the functionalities of both though. Kumar, since you played with that in the past, I suppose you have some test cases for verifying that it works properly before I dig out the 601 machine ? :) Since it's likely that I won't be able to test all scenario, code inspection is much welcome. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-18 06:09:41 +03:00
powerpc/sstep: Introduce GETTYPE macro Replace 'op->type & INSTR_TYPE_MASK' expression with GETTYPE(op->type) macro. Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.ibm.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-05-21 07:21:06 +03:00
type = GETTYPE(op.type);
powerpc: Use instruction emulation infrastructure to handle alignment faults This replaces almost all of the instruction emulation code in fix_alignment() with calls to analyse_instr(), emulate_loadstore() and emulate_dcbz(). The only emulation code left is the SPE emulation code; analyse_instr() etc. do not handle SPE instructions at present. One result of this is that we can now handle alignment faults on all the new VSX load and store instructions that were added in POWER9. VSX loads/stores will take alignment faults for unaligned accesses to cache-inhibited memory. Another effect is that we no longer rely on the DAR and DSISR values set by the processor. With this, we now need to include the instruction emulation code unconditionally. Signed-off-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-08-30 07:12:40 +03:00
if (!OP_IS_LOAD_STORE(type)) {
powerpc: Fix handling of alignment interrupt on dcbz instruction This fixes the emulation of the dcbz instruction in the alignment interrupt handler. The error was that we were comparing just the instruction type field of op.type rather than the whole thing, and therefore the comparison "type != CACHEOP + DCBZ" was always true. Fixes: 31bfdb036f12 ("powerpc: Use instruction emulation infrastructure to handle alignment faults") Signed-off-by: Paul Mackerras <paulus@ozlabs.org> Tested-by: Michal Sojka <sojkam1@fel.cvut.cz> Tested-by: Christian Zigotzky <chzigotzky@xenosoft.de> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-09-13 07:51:24 +03:00
if (op.type != CACHEOP + DCBZ)
powerpc: Use instruction emulation infrastructure to handle alignment faults This replaces almost all of the instruction emulation code in fix_alignment() with calls to analyse_instr(), emulate_loadstore() and emulate_dcbz(). The only emulation code left is the SPE emulation code; analyse_instr() etc. do not handle SPE instructions at present. One result of this is that we can now handle alignment faults on all the new VSX load and store instructions that were added in POWER9. VSX loads/stores will take alignment faults for unaligned accesses to cache-inhibited memory. Another effect is that we no longer rely on the DAR and DSISR values set by the processor. With this, we now need to include the instruction emulation code unconditionally. Signed-off-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-08-30 07:12:40 +03:00
return -EINVAL;
PPC_WARN_ALIGNMENT(dcbz, regs);
powerpc: warn on emulation of dcbz instruction in kernel mode dcbz instruction shouldn't be used on non-cached memory. Using it on non-cached memory can result in alignment exception and implies a heavy handling. Instead of silentely emulating the instruction and resulting in high performance degradation, warn whenever an alignment exception is taken in kernel mode due to dcbz, so that the user is made aware that dcbz instruction has been used unexpectedly by the kernel. Reported-by: Stan Johnson <userm57@yahoo.com> Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/2e3acfe63d289c6fba366e16973c9ab8369e8b75.1631803922.git.christophe.leroy@csgroup.eu
2021-09-16 17:52:09 +03:00
WARN_ON_ONCE(!user_mode(regs));
powerpc: Use instruction emulation infrastructure to handle alignment faults This replaces almost all of the instruction emulation code in fix_alignment() with calls to analyse_instr(), emulate_loadstore() and emulate_dcbz(). The only emulation code left is the SPE emulation code; analyse_instr() etc. do not handle SPE instructions at present. One result of this is that we can now handle alignment faults on all the new VSX load and store instructions that were added in POWER9. VSX loads/stores will take alignment faults for unaligned accesses to cache-inhibited memory. Another effect is that we no longer rely on the DAR and DSISR values set by the processor. With this, we now need to include the instruction emulation code unconditionally. Signed-off-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-08-30 07:12:40 +03:00
r = emulate_dcbz(op.ea, regs);
} else {
if (type == LARX || type == STCX)
return -EIO;
PPC_WARN_ALIGNMENT(unaligned, regs);
r = emulate_loadstore(regs, &op);
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 02:20:36 +04:00
}
[PATCH] powerpc: merge align.c This patch merges align.c, the result isn't quite what was in ppc64 nor what was in ppc32 :) It should implement all the functionalities of both though. Kumar, since you played with that in the past, I suppose you have some test cases for verifying that it works properly before I dig out the 601 machine ? :) Since it's likely that I won't be able to test all scenario, code inspection is much welcome. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-18 06:09:41 +03:00
powerpc: Use instruction emulation infrastructure to handle alignment faults This replaces almost all of the instruction emulation code in fix_alignment() with calls to analyse_instr(), emulate_loadstore() and emulate_dcbz(). The only emulation code left is the SPE emulation code; analyse_instr() etc. do not handle SPE instructions at present. One result of this is that we can now handle alignment faults on all the new VSX load and store instructions that were added in POWER9. VSX loads/stores will take alignment faults for unaligned accesses to cache-inhibited memory. Another effect is that we no longer rely on the DAR and DSISR values set by the processor. With this, we now need to include the instruction emulation code unconditionally. Signed-off-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-08-30 07:12:40 +03:00
if (!r)
return 1;
return r;
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 02:20:36 +04:00
}
Reference in New Issue Copy Permalink