iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
923c53caea
linux/arch/powerpc/kvm/book3s_rmhandlers.S

259 lines
6.4 KiB
ArmAsm
Raw Normal View History

Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
/*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License, version 2, as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* Copyright SUSE Linux Products GmbH 2009
*
* Authors: Alexander Graf <agraf@suse.de>
*/
#include <asm/ppc_asm.h>
#include <asm/kvm_asm.h>
#include <asm/reg.h>
#include <asm/page.h>
#include <asm/asm-offsets.h>
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
#ifdef CONFIG_PPC_BOOK3S_64
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
#include <asm/exception-64s.h>
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
#endif
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
/*****************************************************************************
* *
* Real Mode handlers that need to be in low physical memory *
* *
****************************************************************************/
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
#if defined(CONFIG_PPC_BOOK3S_64)
powerpc: In HV mode, use HSPRG0 for PACA When running in Hypervisor mode (arch 2.06 or later), we store the PACA in HSPRG0 instead of SPRG1. The architecture specifies that SPRGs may be lost during a "nap" power management operation (though they aren't currently on POWER7) and this enables use of SPRG1 by KVM guests. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2011-01-20 09:50:21 +03:00
#define LOAD_SHADOW_VCPU(reg) GET_PACA(reg)
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
#define SHADOW_VCPU_OFF PACA_KVM_SVCPU
#define MSR_NOIRQ MSR_KERNEL & ~(MSR_IR | MSR_DR)
#define FUNC(name) GLUE(.,name)
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 04:18:26 +04:00
kvmppc_skip_interrupt:
/*
* Here all GPRs are unchanged from when the interrupt happened
* except for r13, which is saved in SPRG_SCRATCH0.
*/
mfspr r13, SPRN_SRR0
addi r13, r13, 4
mtspr SPRN_SRR0, r13
GET_SCRATCH0(r13)
rfid
b .
kvmppc_skip_Hinterrupt:
/*
* Here all GPRs are unchanged from when the interrupt happened
* except for r13, which is saved in SPRG_SCRATCH0.
*/
mfspr r13, SPRN_HSRR0
addi r13, r13, 4
mtspr SPRN_HSRR0, r13
GET_SCRATCH0(r13)
hrfid
b .
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 04:18:26 +04:00
#elif defined(CONFIG_PPC_BOOK3S_32)
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
#define SHADOW_VCPU_OFF 0
#define MSR_NOIRQ MSR_KERNEL
#define FUNC(name) name
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
.macro INTERRUPT_TRAMPOLINE intno
.global kvmppc_trampoline_\intno
kvmppc_trampoline_\intno:
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 04:18:26 +04:00
mtspr SPRN_SPRG_SCRATCH0, r13 /* Save r13 */
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
/*
* First thing to do is to find out if we're coming
* from a KVM guest or a Linux process.
*
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
* To distinguish, we check a magic byte in the PACA/current
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
*/
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 04:18:26 +04:00
mfspr r13, SPRN_SPRG_THREAD
lwz r13, THREAD_KVM_SVCPU(r13)
/* PPC32 can have a NULL pointer - let's check for that */
mtspr SPRN_SPRG_SCRATCH1, r12 /* Save r12 */
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
mfcr r12
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 04:18:26 +04:00
cmpwi r13, 0
bne 1f
2: mtcr r12
mfspr r12, SPRN_SPRG_SCRATCH1
mfspr r13, SPRN_SPRG_SCRATCH0 /* r13 = original r13 */
b kvmppc_resume_\intno /* Get back original handler */
1: tophys(r13, r13)
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
stw r12, (SHADOW_VCPU_OFF + SVCPU_SCRATCH1)(r13)
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 04:18:26 +04:00
mfspr r12, SPRN_SPRG_SCRATCH1
stw r12, (SHADOW_VCPU_OFF + SVCPU_SCRATCH0)(r13)
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
lbz r12, (SHADOW_VCPU_OFF + SVCPU_IN_GUEST)(r13)
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:04 +03:00
cmpwi r12, KVM_GUEST_MODE_NONE
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
bne ..kvmppc_handler_hasmagic_\intno
/* No KVM guest? Then jump back to the Linux handler! */
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
lwz r12, (SHADOW_VCPU_OFF + SVCPU_SCRATCH1)(r13)
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 04:18:26 +04:00
b 2b
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
/* Now we know we're handling a KVM guest */
..kvmppc_handler_hasmagic_\intno:
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:04 +03:00
/* Should we just skip the faulting instruction? */
cmpwi r12, KVM_GUEST_MODE_SKIP
beq kvmppc_handler_skip_ins
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
/* Let's store which interrupt we're handling */
li r12, \intno
/* Jump into the SLB exit code that goes to the highmem handler */
b kvmppc_handler_trampoline_exit
.endm
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_SYSTEM_RESET
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_MACHINE_CHECK
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_DATA_STORAGE
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_INST_STORAGE
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_EXTERNAL
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_ALIGNMENT
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_PROGRAM
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_FP_UNAVAIL
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_DECREMENTER
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_SYSCALL
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_TRACE
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_PERFMON
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_ALTIVEC
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:04 +03:00
/*
* Bring us back to the faulting code, but skip the
* faulting instruction.
*
* This is a generic exit path from the interrupt
* trampolines above.
*
* Input Registers:
*
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
* R12 = free
* R13 = Shadow VCPU (PACA)
* SVCPU.SCRATCH0 = guest R12
* SVCPU.SCRATCH1 = guest CR
* SPRG_SCRATCH0 = guest R13
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:04 +03:00
*
*/
kvmppc_handler_skip_ins:
/* Patch the IP to the next instruction */
mfsrr0 r12
addi r12, r12, 4
mtsrr0 r12
/* Clean up all state */
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
lwz r12, (SHADOW_VCPU_OFF + SVCPU_SCRATCH1)(r13)
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:04 +03:00
mtcr r12
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
PPC_LL r12, (SHADOW_VCPU_OFF + SVCPU_SCRATCH0)(r13)
powerpc: Always use SPRN_SPRG_HSCRATCH0 when running in HV mode This uses feature sections to arrange that we always use HSPRG1 as the scratch register in the interrupt entry code rather than SPRG2 when we're running in hypervisor mode on POWER7. This will ensure that we don't trash the guest's SPRG2 when we are running KVM guests. To simplify the code, we define GET_SCRATCH0() and SET_SCRATCH0() macros like the GET_PACA/SET_PACA macros. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2011-04-05 07:59:58 +04:00
GET_SCRATCH0(r13)
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:04 +03:00
/* And get back into the code */
RFI
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 04:18:26 +04:00
#endif
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:04 +03:00
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
/*
* This trampoline brings us back to a real mode handler
*
* Input Registers:
*
KVM: PPC: Use PACA backed shadow vcpu We're being horribly racy right now. All the entry and exit code hijacks random fields from the PACA that could easily be used by different code in case we get interrupted, for example by a #MC or even page fault. After discussing this with Ben, we figured it's best to reserve some more space in the PACA and just shove off some vcpu state to there. That way we can drastically improve the readability of the code, make it less racy and less complex. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:03 +03:00
* R5 = SRR0
* R6 = SRR1
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
* LR = real-mode IP
*
*/
.global kvmppc_handler_lowmem_trampoline
kvmppc_handler_lowmem_trampoline:
KVM: PPC: Use PACA backed shadow vcpu We're being horribly racy right now. All the entry and exit code hijacks random fields from the PACA that could easily be used by different code in case we get interrupted, for example by a #MC or even page fault. After discussing this with Ben, we figured it's best to reserve some more space in the PACA and just shove off some vcpu state to there. That way we can drastically improve the readability of the code, make it less racy and less complex. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:03 +03:00
mtsrr0 r5
mtsrr1 r6
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 08:47:09 +03:00
blr
kvmppc_handler_lowmem_trampoline_end:
KVM: PPC: Call SLB patching code in interrupt safe manner Currently we're racy when doing the transition from IR=1 to IR=0, from the module memory entry code to the real mode SLB switching code. To work around that I took a look at the RTAS entry code which is faced with a similar problem and did the same thing: A small helper in linear mapped memory that does mtmsr with IR=0 and then RFIs info the actual handler. Thanks to that trick we can safely take page faults in the entry code and only need to be really wary of what to do as of the SLB switching part. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:06 +03:00
/*
* Call a function in real mode
*
* Input Registers:
*
* R3 = function
* R4 = MSR
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
* R5 = scratch register
KVM: PPC: Call SLB patching code in interrupt safe manner Currently we're racy when doing the transition from IR=1 to IR=0, from the module memory entry code to the real mode SLB switching code. To work around that I took a look at the RTAS entry code which is faced with a similar problem and did the same thing: A small helper in linear mapped memory that does mtmsr with IR=0 and then RFIs info the actual handler. Thanks to that trick we can safely take page faults in the entry code and only need to be really wary of what to do as of the SLB switching part. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:06 +03:00
*
*/
_GLOBAL(kvmppc_rmcall)
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
LOAD_REG_IMMEDIATE(r5, MSR_NOIRQ)
mtmsr r5 /* Disable relocation and interrupts, so mtsrr
KVM: PPC: Call SLB patching code in interrupt safe manner Currently we're racy when doing the transition from IR=1 to IR=0, from the module memory entry code to the real mode SLB switching code. To work around that I took a look at the RTAS entry code which is faced with a similar problem and did the same thing: A small helper in linear mapped memory that does mtmsr with IR=0 and then RFIs info the actual handler. Thanks to that trick we can safely take page faults in the entry code and only need to be really wary of what to do as of the SLB switching part. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:06 +03:00
doesn't get interrupted */
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
sync
KVM: PPC: Call SLB patching code in interrupt safe manner Currently we're racy when doing the transition from IR=1 to IR=0, from the module memory entry code to the real mode SLB switching code. To work around that I took a look at the RTAS entry code which is faced with a similar problem and did the same thing: A small helper in linear mapped memory that does mtmsr with IR=0 and then RFIs info the actual handler. Thanks to that trick we can safely take page faults in the entry code and only need to be really wary of what to do as of the SLB switching part. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 04:58:06 +03:00
mtsrr0 r3
mtsrr1 r4
RFI
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
#if defined(CONFIG_PPC_BOOK3S_32)
#define STACK_LR INT_FRAME_SIZE+4
KVM: PPC: Use MSR_DR for external load_up Book3S_32 requires MSR_DR to be disabled during load_up_xxx while on Book3S_64 it's supposed to be enabled. I misread the code and disabled it in both cases, potentially breaking the PS3 which has a really small RMA. This patch makes KVM work on the PS3 again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 17:04:20 +04:00
/* load_up_xxx have to run with MSR_DR=0 on Book3S_32 */
#define MSR_EXT_START \
PPC_STL r20, _NIP(r1); \
mfmsr r20; \
LOAD_REG_IMMEDIATE(r3, MSR_DR|MSR_EE); \
andc r3,r20,r3; /* Disable DR,EE */ \
mtmsr r3; \
sync
#define MSR_EXT_END \
mtmsr r20; /* Enable DR,EE */ \
sync; \
PPC_LL r20, _NIP(r1)
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
#elif defined(CONFIG_PPC_BOOK3S_64)
#define STACK_LR _LINK
KVM: PPC: Use MSR_DR for external load_up Book3S_32 requires MSR_DR to be disabled during load_up_xxx while on Book3S_64 it's supposed to be enabled. I misread the code and disabled it in both cases, potentially breaking the PS3 which has a really small RMA. This patch makes KVM work on the PS3 again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 17:04:20 +04:00
#define MSR_EXT_START
#define MSR_EXT_END
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
#endif
KVM: PPC: Add helper functions to call real mode loaders Linux contains quite some bits of code to load FPU, Altivec and VSX lazily for a task. It calls those bits in real mode, coming from an interrupt handler. For KVM we better reuse those, so let's wrap a bit of trampoline magic around them and then we can call them from normal module code. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-15 16:49:10 +03:00
/*
* Activate current's external feature (FPU/Altivec/VSX)
*/
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
#define define_load_up(what) \
\
_GLOBAL(kvmppc_load_up_ ## what); \
PPC_STLU r1, -INT_FRAME_SIZE(r1); \
mflr r3; \
PPC_STL r3, STACK_LR(r1); \
KVM: PPC: Use MSR_DR for external load_up Book3S_32 requires MSR_DR to be disabled during load_up_xxx while on Book3S_64 it's supposed to be enabled. I misread the code and disabled it in both cases, potentially breaking the PS3 which has a really small RMA. This patch makes KVM work on the PS3 again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 17:04:20 +04:00
MSR_EXT_START; \
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
\
bl FUNC(load_up_ ## what); \
\
KVM: PPC: Use MSR_DR for external load_up Book3S_32 requires MSR_DR to be disabled during load_up_xxx while on Book3S_64 it's supposed to be enabled. I misread the code and disabled it in both cases, potentially breaking the PS3 which has a really small RMA. This patch makes KVM work on the PS3 again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 17:04:20 +04:00
MSR_EXT_END; \
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:46 +04:00
PPC_LL r3, STACK_LR(r1); \
mtlr r3; \
addi r1, r1, INT_FRAME_SIZE; \
KVM: PPC: Add helper functions to call real mode loaders Linux contains quite some bits of code to load FPU, Altivec and VSX lazily for a task. It calls those bits in real mode, coming from an interrupt handler. For KVM we better reuse those, so let's wrap a bit of trampoline magic around them and then we can call them from normal module code. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-15 16:49:10 +03:00
blr
define_load_up(fpu)
#ifdef CONFIG_ALTIVEC
define_load_up(altivec)
#endif
#ifdef CONFIG_VSX
define_load_up(vsx)
#endif
KVM: PPC: Make SLB switching code the new segment framework We just introduced generic segment switching code that only needs to call small macros to do the actual switching, but keeps most of the entry / exit code generic. So let's move the SLB switching code over to use this new mechanism. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 02:11:48 +04:00
#include "book3s_segment.S"
Reference in New Issue Copy Permalink