2005-09-26 10:04:21 +04:00
/ *
* arch/ p p c64 / k e r n e l / h e a d . S
*
* PowerPC v e r s i o n
* Copyright ( C ) 1 9 9 5 - 1 9 9 6 G a r y T h o m a s ( g d t @linuxppc.org)
*
* Rewritten b y C o r t D o u g a n ( c o r t @cs.nmt.edu) for PReP
* Copyright ( C ) 1 9 9 6 C o r t D o u g a n < c o r t @cs.nmt.edu>
* Adapted f o r P o w e r M a c i n t o s h b y P a u l M a c k e r r a s .
* Low- l e v e l e x c e p t i o n h a n d l e r s a n d M M U s u p p o r t
* rewritten b y P a u l M a c k e r r a s .
* Copyright ( C ) 1 9 9 6 P a u l M a c k e r r a s .
*
* Adapted f o r 6 4 b i t P o w e r P C b y D a v e E n g e b r e t s e n , P e t e r B e r g n e r , a n d
* Mike C o r r i g a n { e n g e b r e t | b e r g n e r | m i k e j c } @us.ibm.com
*
* This f i l e c o n t a i n s t h e l o w - l e v e l s u p p o r t a n d s e t u p f o r t h e
* PowerPC- 6 4 p l a t f o r m , i n c l u d i n g t r a p a n d i n t e r r u p t d i s p a t c h .
*
* This p r o g r a m i s f r e e s o f t w a r e ; you can redistribute it and/or
* modify i t u n d e r t h e t e r m s o f t h e G N U G e n e r a l P u b l i c L i c e n s e
* as p u b l i s h e d b y t h e F r e e S o f t w a r e F o u n d a t i o n ; either version
* 2 of t h e L i c e n s e , o r ( a t y o u r o p t i o n ) a n y l a t e r v e r s i o n .
* /
# include < l i n u x / c o n f i g . h >
# include < l i n u x / t h r e a d s . h >
2005-10-10 08:01:07 +04:00
# include < a s m / r e g . h >
2005-09-26 10:04:21 +04:00
# include < a s m / p a g e . h >
# include < a s m / m m u . h >
# include < a s m / p p c _ a s m . h >
# include < a s m / a s m - o f f s e t s . h >
# include < a s m / b u g . h >
# include < a s m / c p u t a b l e . h >
# include < a s m / s e t u p . h >
# include < a s m / h v c a l l . h >
2005-11-02 07:02:47 +03:00
# include < a s m / i s e r i e s / l p a r _ m a p . h >
[PATCH] powerpc: Merge thread_info.h
Merge ppc32 and ppc64 versions of thread_info.h. They were pretty
similar already, the chief changes are:
- Instead of inline asm to implement current_thread_info(),
which needs to be different for ppc32 and ppc64, we use C with an
asm("r1") register variable. gcc turns it into the same asm as we
used to have for both platforms.
- We replace ppc32's 'local_flags' with the ppc64
'syscall_noerror' field. The noerror flag was in fact the only thing
in the local_flags field anyway, so the ppc64 approach is simpler, and
means we only need a load-immediate/store instead of load/mask/store
when clearing the flag.
- In readiness for 64k pages, when THREAD_SIZE will be less
than a page, ppc64 used kmalloc() rather than get_free_pages() to
allocate the kernel stack. With this patch we do the same for ppc32,
since there's no strong reason not to.
- For ppc64, we no longer export THREAD_SHIFT and THREAD_SIZE
via asm-offsets, thread_info.h can now be safely included in asm, as
on ppc32.
Built and booted on G4 Powerbook (ARCH=ppc and ARCH=powerpc) and
Power5 (ARCH=ppc64 and ARCH=powerpc).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-10-21 09:45:50 +04:00
# include < a s m / t h r e a d _ i n f o . h >
2005-09-26 10:04:21 +04:00
# ifdef C O N F I G _ P P C _ I S E R I E S
# define D O _ S O F T _ D I S A B L E
# endif
/ *
* We l a y o u t p h y s i c a l m e m o r y a s f o l l o w s :
* 0 x0 0 0 0 - 0 x00 f f : S e c o n d a r y p r o c e s s o r s p i n c o d e
* 0 x0 1 0 0 - 0 x2 f f f : p S e r i e s I n t e r r u p t p r o l o g s
* 0 x3 0 0 0 - 0 x5 f f f : i n t e r r u p t s u p p o r t , i S e r i e s a n d c o m m o n i n t e r r u p t p r o l o g s
* 0 x6 0 0 0 - 0 x6 f f f : I n i t i a l ( C P U 0 ) s e g m e n t t a b l e
* 0 x7 0 0 0 - 0 x7 f f f : F W N M I d a t a a r e a
* 0 x8 0 0 0 - : E a r l y i n i t a n d s u p p o r t c o d e
* /
/ *
* SPRG U s a g e
*
* Register D e f i n i t i o n
*
* SPRG0 r e s e r v e d f o r h y p e r v i s o r
* SPRG1 t e m p - u s e d t o s a v e g p r
* SPRG2 t e m p - u s e d t o s a v e g p r
* SPRG3 v i r t a d d r o f p a c a
* /
/ *
* Entering i n t o t h i s c o d e w e m a k e t h e f o l l o w i n g a s s u m p t i o n s :
* For p S e r i e s :
* 1 . The M M U i s o f f & o p e n f i r m w a r e i s r u n n i n g i n r e a l m o d e .
* 2 . The k e r n e l i s e n t e r e d a t _ _ s t a r t
*
* For i S e r i e s :
* 1 . The M M U i s o n ( a s i t a l w a y s i s f o r i S e r i e s )
* 2 . The k e r n e l i s e n t e r e d a t s y s t e m _ r e s e t _ i S e r i e s
* /
.text
.globl _stext
_stext :
# ifdef C O N F I G _ P P C _ M U L T I P L A T F O R M
_ GLOBAL( _ _ s t a r t )
/* NOP this out unconditionally */
BEGIN_ F T R _ S E C T I O N
2005-10-06 04:59:19 +04:00
b . _ _ s t a r t _ i n i t i a l i z a t i o n _ m u l t i p l a t f o r m
2005-09-26 10:04:21 +04:00
END_ F T R _ S E C T I O N ( 0 , 1 )
# endif / * C O N F I G _ P P C _ M U L T I P L A T F O R M * /
/* Catch branch to 0 in real mode */
trap
# ifdef C O N F I G _ P P C _ I S E R I E S
/ *
* At o f f s e t 0 x20 , t h e r e i s a p o i n t e r t o i S e r i e s L P A R d a t a .
* This i s r e q u i r e d b y t h e h y p e r v i s o r
* /
. = 0 x2 0
.llong - K E R N E L B A S E
/ *
* At o f f s e t 0 x28 a n d 0 x30 a r e o f f s e t s t o t h e m s c h u n k s _ m a p
* array ( u s e d b y t h e i S e r i e s L P A R d e b u g g e r t o d o t r a n s l a t i o n
* between p h y s i c a l a d d r e s s e s a n d a b s o l u t e a d d r e s s e s ) a n d
* to t h e p i d h a s h t a b l e ( a l s o u s e d b y t h e d e b u g g e r )
* /
.llong _ m a p - K E R N E L B A S E
.llong 0 /* pidhash-KERNELBASE SFRXXX */
/* Offset 0x38 - Pointer to start of embedded System.map */
.globl embedded_sysmap_start
embedded_sysmap_start :
.llong 0
/* Offset 0x40 - Pointer to end of embedded System.map */
.globl embedded_sysmap_end
embedded_sysmap_end :
.llong 0
# endif / * C O N F I G _ P P C _ I S E R I E S * /
/* Secondary processors spin on this value until it goes to 1. */
.globl __secondary_hold_spinloop
__secondary_hold_spinloop :
.llong 0x0
/* Secondary processors write this value with their cpu # */
/* after they enter the spin loop immediately below. */
.globl __secondary_hold_acknowledge
__secondary_hold_acknowledge :
.llong 0x0
. = 0 x6 0
/ *
* The f o l l o w i n g c o d e i s u s e d o n p S e r i e s t o h o l d s e c o n d a r y p r o c e s s o r s
* in a s p i n l o o p a f t e r t h e y h a v e b e e n f r e e d f r o m O p e n F i r m w a r e , b u t
* before t h e b u l k o f t h e k e r n e l h a s b e e n r e l o c a t e d . T h i s c o d e
* is r e l o c a t e d t o p h y s i c a l a d d r e s s 0 x60 b e f o r e p r o m _ i n i t i s r u n .
* All o f i t m u s t f i t b e l o w t h e f i r s t e x c e p t i o n v e c t o r a t 0 x10 0 .
* /
_ GLOBAL( _ _ s e c o n d a r y _ h o l d )
mfmsr r24
ori r24 ,r24 ,M S R _ R I
mtmsrd r24 / * R I o n * /
/* Grab our linux cpu number */
mr r24 ,r3
/* Tell the master cpu we're here */
/* Relocation is off & we are located at an address less */
/* than 0x100, so only need to grab low order offset. */
std r24 ,_ _ s e c o n d a r y _ h o l d _ a c k n o w l e d g e @l(0)
sync
/* All secondary cpus wait here until told to start. */
100 : ld r4 ,_ _ s e c o n d a r y _ h o l d _ s p i n l o o p @l(0)
cmpdi 0 ,r4 ,1
bne 1 0 0 b
# ifdef C O N F I G _ H M T
b . h m t _ i n i t
# else
# ifdef C O N F I G _ S M P
mr r3 ,r24
b . p S e r i e s _ s e c o n d a r y _ s m p _ i n i t
# else
BUG_ O P C O D E
# endif
# endif
/* This value is used to mark exception frames on the stack. */
.section " .toc " , " aw"
exception_marker :
.tc _ 7 2 6 5 6 7 7 3 _ 6 8 6 5 7 2 6 5 [ T C ] ,0 x72 6 5 6 7 7 3 6 8 6 5 7 2 6 5
.text
/ *
* The f o l l o w i n g m a c r o s d e f i n e t h e c o d e t h a t a p p e a r s a s
* the p r o l o g u e t o e a c h o f t h e e x c e p t i o n h a n d l e r s . T h e y
* are s p l i t i n t o t w o p a r t s t o a l l o w a s i n g l e k e r n e l b i n a r y
* to b e u s e d f o r p S e r i e s a n d i S e r i e s .
* LOL. O n e d a y . . . - p a u l u s
* /
/ *
* We m a k e a s m u c h o f t h e e x c e p t i o n c o d e c o m m o n b e t w e e n n a t i v e
* exception h a n d l e r s ( i n c l u d i n g p S e r i e s L P A R ) a n d i S e r i e s L P A R
* implementations a s p o s s i b l e .
* /
/ *
* This i s t h e s t a r t o f t h e i n t e r r u p t h a n d l e r s f o r p S e r i e s
* This c o d e r u n s w i t h r e l o c a t i o n o f f .
* /
# define E X _ R 9 0
# define E X _ R 1 0 8
# define E X _ R 1 1 1 6
# define E X _ R 1 2 2 4
# define E X _ R 1 3 3 2
# define E X _ S R R 0 4 0
# define E X _ D A R 4 8
# define E X _ D S I S R 5 6
# define E X _ C C R 6 0
2005-11-07 03:06:55 +03:00
# define E X _ R 3 6 4
# define E X _ L R 7 2
2005-09-26 10:04:21 +04:00
# define E X C E P T I O N _ P R O L O G _ P S E R I E S ( a r e a , l a b e l ) \
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3 ; /* get paca address into r13 */ \
2005-09-26 10:04:21 +04:00
std r9 ,a r e a + E X _ R 9 ( r13 ) ; /* save r9 - r12 */ \
std r10 ,a r e a + E X _ R 1 0 ( r13 ) ; \
std r11 ,a r e a + E X _ R 1 1 ( r13 ) ; \
std r12 ,a r e a + E X _ R 1 2 ( r13 ) ; \
2005-10-10 08:01:07 +04:00
mfspr r9 ,S P R N _ S P R G 1 ; \
2005-09-26 10:04:21 +04:00
std r9 ,a r e a + E X _ R 1 3 ( r13 ) ; \
mfcr r9 ; \
clrrdi r12 ,r13 ,3 2 ; /* get high part of &label */ \
mfmsr r10 ; \
2005-10-10 08:01:07 +04:00
mfspr r11 ,S P R N _ S R R 0 ; /* save SRR0 */ \
2005-09-26 10:04:21 +04:00
ori r12 ,r12 ,( l a b e l ) @l; /* virt addr of handler */ \
ori r10 ,r10 ,M S R _ I R | M S R _ D R | M S R _ R I ; \
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 0 ,r12 ; \
mfspr r12 ,S P R N _ S R R 1 ; /* and SRR1 */ \
mtspr S P R N _ S R R 1 ,r10 ; \
2005-09-26 10:04:21 +04:00
rfid; \
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
/ *
* This i s t h e s t a r t o f t h e i n t e r r u p t h a n d l e r s f o r i S e r i e s
* This c o d e r u n s w i t h r e l o c a t i o n o n .
* /
# define E X C E P T I O N _ P R O L O G _ I S E R I E S _ 1 ( a r e a ) \
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3 ; /* get paca address into r13 */ \
2005-09-26 10:04:21 +04:00
std r9 ,a r e a + E X _ R 9 ( r13 ) ; /* save r9 - r12 */ \
std r10 ,a r e a + E X _ R 1 0 ( r13 ) ; \
std r11 ,a r e a + E X _ R 1 1 ( r13 ) ; \
std r12 ,a r e a + E X _ R 1 2 ( r13 ) ; \
2005-10-10 08:01:07 +04:00
mfspr r9 ,S P R N _ S P R G 1 ; \
2005-09-26 10:04:21 +04:00
std r9 ,a r e a + E X _ R 1 3 ( r13 ) ; \
mfcr r9
# define E X C E P T I O N _ P R O L O G _ I S E R I E S _ 2 \
mfmsr r10 ; \
ld r11 ,P A C A L P P A C A + L P P A C A S R R 0 ( r13 ) ; \
ld r12 ,P A C A L P P A C A + L P P A C A S R R 1 ( r13 ) ; \
ori r10 ,r10 ,M S R _ R I ; \
mtmsrd r10 ,1
/ *
* The c o m m o n e x c e p t i o n p r o l o g i s u s e d f o r a l l e x c e p t a f e w e x c e p t i o n s
* such a s a s e g m e n t m i s s o n a k e r n e l a d d r e s s . W e h a v e t o b e p r e p a r e d
* to t a k e a n o t h e r e x c e p t i o n f r o m t h e p o i n t w h e r e w e f i r s t t o u c h t h e
* kernel s t a c k o n w a r d s .
*
* On e n t r y r13 p o i n t s t o t h e p a c a , r9 - r13 a r e s a v e d i n t h e p a c a ,
* r9 c o n t a i n s t h e s a v e d C R , r11 a n d r12 c o n t a i n t h e s a v e d S R R 0 a n d
* SRR1 , a n d r e l o c a t i o n i s o n .
* /
# define E X C E P T I O N _ P R O L O G _ C O M M O N ( n , a r e a ) \
andi. r10 ,r12 ,M S R _ P R ; /* See if coming from user */ \
mr r10 ,r1 ; /* Save r1 */ \
subi r1 ,r1 ,I N T _ F R A M E _ S I Z E ; /* alloc frame on kernel stack */ \
beq- 1 f ; \
ld r1 ,P A C A K S A V E ( r13 ) ; /* kernel stack to use */ \
1 : cmpdi c r1 ,r1 ,0 ; /* check if r1 is in userspace */ \
bge- c r1 ,b a d _ s t a c k ; /* abort if it is */ \
std r9 ,_ C C R ( r1 ) ; /* save CR in stackframe */ \
std r11 ,_ N I P ( r1 ) ; /* save SRR0 in stackframe */ \
std r12 ,_ M S R ( r1 ) ; /* save SRR1 in stackframe */ \
std r10 ,0 ( r1 ) ; /* make stack chain pointer */ \
std r0 ,G P R 0 ( r1 ) ; /* save r0 in stackframe */ \
std r10 ,G P R 1 ( r1 ) ; /* save r1 in stackframe */ \
std r2 ,G P R 2 ( r1 ) ; /* save r2 in stackframe */ \
SAVE_ 4 G P R S ( 3 , r1 ) ; /* save r3 - r6 in stackframe */ \
SAVE_ 2 G P R S ( 7 , r1 ) ; /* save r7, r8 in stackframe */ \
ld r9 ,a r e a + E X _ R 9 ( r13 ) ; /* move r9, r10 to stackframe */ \
ld r10 ,a r e a + E X _ R 1 0 ( r13 ) ; \
std r9 ,G P R 9 ( r1 ) ; \
std r10 ,G P R 1 0 ( r1 ) ; \
ld r9 ,a r e a + E X _ R 1 1 ( r13 ) ; /* move r11 - r13 to stackframe */ \
ld r10 ,a r e a + E X _ R 1 2 ( r13 ) ; \
ld r11 ,a r e a + E X _ R 1 3 ( r13 ) ; \
std r9 ,G P R 1 1 ( r1 ) ; \
std r10 ,G P R 1 2 ( r1 ) ; \
std r11 ,G P R 1 3 ( r1 ) ; \
ld r2 ,P A C A T O C ( r13 ) ; /* get kernel TOC into r2 */ \
mflr r9 ; /* save LR in stackframe */ \
std r9 ,_ L I N K ( r1 ) ; \
mfctr r10 ; /* save CTR in stackframe */ \
std r10 ,_ C T R ( r1 ) ; \
2005-10-10 08:01:07 +04:00
mfspr r11 ,S P R N _ X E R ; /* save XER in stackframe */ \
2005-09-26 10:04:21 +04:00
std r11 ,_ X E R ( r1 ) ; \
li r9 ,( n ) + 1 ; \
std r9 ,_ T R A P ( r1 ) ; /* set trap number */ \
li r10 ,0 ; \
ld r11 ,e x c e p t i o n _ m a r k e r @toc(r2); \
std r10 ,R E S U L T ( r1 ) ; /* clear regs->result */ \
std r11 ,S T A C K _ F R A M E _ O V E R H E A D - 1 6 ( r1 ) ; /* mark the frame */
/ *
* Exception v e c t o r s .
* /
# define S T D _ E X C E P T I O N _ P S E R I E S ( n , l a b e l ) \
. = n; \
.globl # _ p S e r i e s ; \
label## _ p S e r i e s : \
HMT_ M E D I U M ; \
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13 ; /* save r13 */ \
2005-09-26 10:04:21 +04:00
RUNLATCH_ O N ( r13 ) ; \
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X G E N , l a b e l ## _ c o m m o n )
# define S T D _ E X C E P T I O N _ I S E R I E S ( n , l a b e l , a r e a ) \
.globl # _ i S e r i e s ; \
label## _ i S e r i e s : \
HMT_ M E D I U M ; \
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13 ; /* save r13 */ \
2005-09-26 10:04:21 +04:00
RUNLATCH_ O N ( r13 ) ; \
EXCEPTION_ P R O L O G _ I S E R I E S _ 1 ( a r e a ) ; \
EXCEPTION_ P R O L O G _ I S E R I E S _ 2 ; \
b l a b e l ## _ c o m m o n
# define M A S K A B L E _ E X C E P T I O N _ I S E R I E S ( n , l a b e l ) \
.globl # _ i S e r i e s ; \
label## _ i S e r i e s : \
HMT_ M E D I U M ; \
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13 ; /* save r13 */ \
2005-09-26 10:04:21 +04:00
RUNLATCH_ O N ( r13 ) ; \
EXCEPTION_ P R O L O G _ I S E R I E S _ 1 ( P A C A _ E X G E N ) ; \
lbz r10 ,P A C A P R O C E N A B L E D ( r13 ) ; \
cmpwi 0 ,r10 ,0 ; \
beq- l a b e l ## _ i S e r i e s _ m a s k e d ; \
EXCEPTION_ P R O L O G _ I S E R I E S _ 2 ; \
b l a b e l ## _ c o m m o n ; \
# ifdef D O _ S O F T _ D I S A B L E
# define D I S A B L E _ I N T S \
lbz r10 ,P A C A P R O C E N A B L E D ( r13 ) ; \
li r11 ,0 ; \
std r10 ,S O F T E ( r1 ) ; \
mfmsr r10 ; \
stb r11 ,P A C A P R O C E N A B L E D ( r13 ) ; \
ori r10 ,r10 ,M S R _ E E ; \
mtmsrd r10 ,1
# define E N A B L E _ I N T S \
lbz r10 ,P A C A P R O C E N A B L E D ( r13 ) ; \
mfmsr r11 ; \
std r10 ,S O F T E ( r1 ) ; \
ori r11 ,r11 ,M S R _ E E ; \
mtmsrd r11 ,1
# else / * h a r d e n a b l e / d i s a b l e i n t e r r u p t s * /
# define D I S A B L E _ I N T S
# define E N A B L E _ I N T S \
ld r12 ,_ M S R ( r1 ) ; \
mfmsr r11 ; \
rlwimi r11 ,r12 ,0 ,M S R _ E E ; \
mtmsrd r11 ,1
# endif
# define S T D _ E X C E P T I O N _ C O M M O N ( t r a p , l a b e l , h d l r ) \
.align 7 ; \
.globl # _ c o m m o n ; \
label## _ c o m m o n : \
EXCEPTION_ P R O L O G _ C O M M O N ( t r a p , P A C A _ E X G E N ) ; \
DISABLE_ I N T S ; \
bl . s a v e _ n v g p r s ; \
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D ; \
bl h d l r ; \
b . r e t _ f r o m _ e x c e p t
# define S T D _ E X C E P T I O N _ C O M M O N _ L I T E ( t r a p , l a b e l , h d l r ) \
.align 7 ; \
.globl # _ c o m m o n ; \
label## _ c o m m o n : \
EXCEPTION_ P R O L O G _ C O M M O N ( t r a p , P A C A _ E X G E N ) ; \
DISABLE_ I N T S ; \
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D ; \
bl h d l r ; \
b . r e t _ f r o m _ e x c e p t _ l i t e
/ *
* Start o f p S e r i e s s y s t e m i n t e r r u p t r o u t i n e s
* /
. = 0 x1 0 0
.globl __start_interrupts
__start_interrupts :
STD_ E X C E P T I O N _ P S E R I E S ( 0 x10 0 , s y s t e m _ r e s e t )
. = 0 x2 0 0
_machine_check_pSeries :
HMT_ M E D I U M
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13 / * s a v e r13 * /
2005-09-26 10:04:21 +04:00
RUNLATCH_ O N ( r13 )
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X M C , m a c h i n e _ c h e c k _ c o m m o n )
. = 0 x3 0 0
.globl data_access_pSeries
data_access_pSeries :
HMT_ M E D I U M
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13
2005-09-26 10:04:21 +04:00
BEGIN_ F T R _ S E C T I O N
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 2 ,r12
mfspr r13 ,S P R N _ D A R
mfspr r12 ,S P R N _ D S I S R
2005-09-26 10:04:21 +04:00
srdi r13 ,r13 ,6 0
rlwimi r13 ,r12 ,1 6 ,0 x20
mfcr r12
cmpwi r13 ,0 x2 c
beq . d o _ s t a b _ b o l t e d _ p S e r i e s
mtcrf 0 x80 ,r12
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S P R G 2
2005-09-26 10:04:21 +04:00
END_ F T R _ S E C T I O N _ I F C L R ( C P U _ F T R _ S L B )
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X G E N , d a t a _ a c c e s s _ c o m m o n )
. = 0 x3 8 0
.globl data_access_slb_pSeries
data_access_slb_pSeries :
HMT_ M E D I U M
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13
2005-09-26 10:04:21 +04:00
RUNLATCH_ O N ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3 / * g e t p a c a a d d r e s s i n t o r13 * /
2005-11-07 03:06:55 +03:00
std r3 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
mfspr r3 ,S P R N _ D A R
2005-09-26 10:04:21 +04:00
std r9 ,P A C A _ E X S L B + E X _ R 9 ( r13 ) / * s a v e r9 - r12 * /
2005-11-07 03:06:55 +03:00
mfcr r9
# ifdef _ _ D I S A B L E D _ _
/* Keep that around for when we re-implement dynamic VSIDs */
cmpdi r3 ,0
bge s l b _ m i s s _ u s e r _ p s e r i e s
# endif / * _ _ D I S A B L E D _ _ * /
2005-09-26 10:04:21 +04:00
std r10 ,P A C A _ E X S L B + E X _ R 1 0 ( r13 )
std r11 ,P A C A _ E X S L B + E X _ R 1 1 ( r13 )
std r12 ,P A C A _ E X S L B + E X _ R 1 2 ( r13 )
2005-11-07 03:06:55 +03:00
mfspr r10 ,S P R N _ S P R G 1
std r10 ,P A C A _ E X S L B + E X _ R 1 3 ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S R R 1 / * a n d S R R 1 * /
2005-11-07 03:06:55 +03:00
b . s l b _ m i s s _ r e a l m o d e / * R e l . b r a n c h w o r k s i n r e a l m o d e * /
2005-09-26 10:04:21 +04:00
STD_ E X C E P T I O N _ P S E R I E S ( 0 x40 0 , i n s t r u c t i o n _ a c c e s s )
. = 0 x4 8 0
.globl instruction_access_slb_pSeries
instruction_access_slb_pSeries :
HMT_ M E D I U M
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13
2005-09-26 10:04:21 +04:00
RUNLATCH_ O N ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3 / * g e t p a c a a d d r e s s i n t o r13 * /
2005-11-07 03:06:55 +03:00
std r3 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
mfspr r3 ,S P R N _ S R R 0 / * S R R 0 i s f a u l t i n g a d d r e s s * /
2005-09-26 10:04:21 +04:00
std r9 ,P A C A _ E X S L B + E X _ R 9 ( r13 ) / * s a v e r9 - r12 * /
2005-11-07 03:06:55 +03:00
mfcr r9
# ifdef _ _ D I S A B L E D _ _
/* Keep that around for when we re-implement dynamic VSIDs */
cmpdi r3 ,0
bge s l b _ m i s s _ u s e r _ p s e r i e s
# endif / * _ _ D I S A B L E D _ _ * /
2005-09-26 10:04:21 +04:00
std r10 ,P A C A _ E X S L B + E X _ R 1 0 ( r13 )
std r11 ,P A C A _ E X S L B + E X _ R 1 1 ( r13 )
std r12 ,P A C A _ E X S L B + E X _ R 1 2 ( r13 )
2005-11-07 03:06:55 +03:00
mfspr r10 ,S P R N _ S P R G 1
std r10 ,P A C A _ E X S L B + E X _ R 1 3 ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S R R 1 / * a n d S R R 1 * /
2005-11-07 03:06:55 +03:00
b . s l b _ m i s s _ r e a l m o d e / * R e l . b r a n c h w o r k s i n r e a l m o d e * /
2005-09-26 10:04:21 +04:00
STD_ E X C E P T I O N _ P S E R I E S ( 0 x50 0 , h a r d w a r e _ i n t e r r u p t )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x60 0 , a l i g n m e n t )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x70 0 , p r o g r a m _ c h e c k )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x80 0 , f p _ u n a v a i l a b l e )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x90 0 , d e c r e m e n t e r )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x a00 , t r a p _ 0 a )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x b00 , t r a p _ 0 b )
. = 0 xc0 0
.globl system_call_pSeries
system_call_pSeries :
HMT_ M E D I U M
RUNLATCH_ O N ( r9 )
mr r9 ,r13
mfmsr r10
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3
mfspr r11 ,S P R N _ S R R 0
2005-09-26 10:04:21 +04:00
clrrdi r12 ,r13 ,3 2
oris r12 ,r12 ,s y s t e m _ c a l l _ c o m m o n @h
ori r12 ,r12 ,s y s t e m _ c a l l _ c o m m o n @l
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 0 ,r12
2005-09-26 10:04:21 +04:00
ori r10 ,r10 ,M S R _ I R | M S R _ D R | M S R _ R I
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S R R 1
mtspr S P R N _ S R R 1 ,r10
2005-09-26 10:04:21 +04:00
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
STD_ E X C E P T I O N _ P S E R I E S ( 0 x d00 , s i n g l e _ s t e p )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x e 0 0 , t r a p _ 0 e )
/ * We n e e d t o d e a l w i t h t h e A l t i v e c u n a v a i l a b l e e x c e p t i o n
* here w h i c h i s a t 0 x f20 , t h u s i n t h e m i d d l e o f t h e
* prolog c o d e o f t h e P e r f o r m a n c e M o n i t o r o n e . A l i t t l e
* trickery i s t h u s n e c e s s a r y
* /
. = 0 xf0 0
b p e r f o r m a n c e _ m o n i t o r _ p S e r i e s
STD_ E X C E P T I O N _ P S E R I E S ( 0 x f20 , a l t i v e c _ u n a v a i l a b l e )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x13 0 0 , i n s t r u c t i o n _ b r e a k p o i n t )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x17 0 0 , a l t i v e c _ a s s i s t )
. = 0 x3 0 0 0
/*** pSeries interrupt support ***/
/* moved from 0xf00 */
STD_ E X C E P T I O N _ P S E R I E S ( . , p e r f o r m a n c e _ m o n i t o r )
.align 7
_ GLOBAL( d o _ s t a b _ b o l t e d _ p S e r i e s )
mtcrf 0 x80 ,r12
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S P R G 2
2005-09-26 10:04:21 +04:00
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X S L B , . d o _ s t a b _ b o l t e d )
2005-11-07 03:06:55 +03:00
/ *
* We h a v e s o m e r o o m h e r e w e u s e t h a t t o p u t
* the p e r i e s s l b m i s s u s e r t r a m p o l i n e c o d e s o i t ' s r e a s o n a b l y
* away f r o m s l b _ m i s s _ u s e r _ c o m m o n t o a v o i d p r o b l e m s w i t h r f i d
*
* This i s u s e d f o r w h e n t h e S L B m i s s h a n d l e r h a s t o g o v i r t u a l ,
* which d o e s n ' t h a p p e n f o r n o w a n y m o r e b u t w i l l o n c e w e r e - i m p l e m e n t
* dynamic V S I D s f o r s h a r e d p a g e t a b l e s
* /
# ifdef _ _ D I S A B L E D _ _
slb_miss_user_pseries :
std r10 ,P A C A _ E X G E N + E X _ R 1 0 ( r13 )
std r11 ,P A C A _ E X G E N + E X _ R 1 1 ( r13 )
std r12 ,P A C A _ E X G E N + E X _ R 1 2 ( r13 )
mfspr r10 ,S P R G 1
ld r11 ,P A C A _ E X S L B + E X _ R 9 ( r13 )
ld r12 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
std r10 ,P A C A _ E X G E N + E X _ R 1 3 ( r13 )
std r11 ,P A C A _ E X G E N + E X _ R 9 ( r13 )
std r12 ,P A C A _ E X G E N + E X _ R 3 ( r13 )
clrrdi r12 ,r13 ,3 2
mfmsr r10
mfspr r11 ,S R R 0 / * s a v e S R R 0 * /
ori r12 ,r12 ,s l b _ m i s s _ u s e r _ c o m m o n @l /* virt addr of handler */
ori r10 ,r10 ,M S R _ I R | M S R _ D R | M S R _ R I
mtspr S R R 0 ,r12
mfspr r12 ,S R R 1 / * a n d S R R 1 * /
mtspr S R R 1 ,r10
rfid
b . / * p r e v e n t s p e c . e x e c u t i o n * /
# endif / * _ _ D I S A B L E D _ _ * /
2005-09-26 10:04:21 +04:00
/ *
* Vectors f o r t h e F W N M I o p t i o n . S h a r e c o m m o n c o d e .
* /
2005-10-10 08:01:07 +04:00
.globl system_reset_fwnmi
2005-09-26 10:04:21 +04:00
system_reset_fwnmi :
2005-10-10 08:01:07 +04:00
HMT_ M E D I U M
mtspr S P R N _ S P R G 1 ,r13 / * s a v e r13 * /
RUNLATCH_ O N ( r13 )
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X G E N , s y s t e m _ r e s e t _ c o m m o n )
2005-09-26 10:04:21 +04:00
2005-10-10 08:01:07 +04:00
.globl machine_check_fwnmi
2005-09-26 10:04:21 +04:00
machine_check_fwnmi :
2005-10-10 08:01:07 +04:00
HMT_ M E D I U M
mtspr S P R N _ S P R G 1 ,r13 / * s a v e r13 * /
RUNLATCH_ O N ( r13 )
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X M C , m a c h i n e _ c h e c k _ c o m m o n )
2005-09-26 10:04:21 +04:00
# ifdef C O N F I G _ P P C _ I S E R I E S
/*** ISeries-LPAR interrupt handlers ***/
STD_ E X C E P T I O N _ I S E R I E S ( 0 x20 0 , m a c h i n e _ c h e c k , P A C A _ E X M C )
.globl data_access_iSeries
data_access_iSeries :
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13
2005-09-26 10:04:21 +04:00
BEGIN_ F T R _ S E C T I O N
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 2 ,r12
mfspr r13 ,S P R N _ D A R
mfspr r12 ,S P R N _ D S I S R
2005-09-26 10:04:21 +04:00
srdi r13 ,r13 ,6 0
rlwimi r13 ,r12 ,1 6 ,0 x20
mfcr r12
cmpwi r13 ,0 x2 c
beq . d o _ s t a b _ b o l t e d _ i S e r i e s
mtcrf 0 x80 ,r12
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S P R G 2
2005-09-26 10:04:21 +04:00
END_ F T R _ S E C T I O N _ I F C L R ( C P U _ F T R _ S L B )
EXCEPTION_ P R O L O G _ I S E R I E S _ 1 ( P A C A _ E X G E N )
EXCEPTION_ P R O L O G _ I S E R I E S _ 2
b d a t a _ a c c e s s _ c o m m o n
.do_stab_bolted_iSeries :
mtcrf 0 x80 ,r12
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S P R G 2
2005-09-26 10:04:21 +04:00
EXCEPTION_ P R O L O G _ I S E R I E S _ 1 ( P A C A _ E X S L B )
EXCEPTION_ P R O L O G _ I S E R I E S _ 2
b . d o _ s t a b _ b o l t e d
.globl data_access_slb_iSeries
data_access_slb_iSeries :
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13 / * s a v e r13 * /
2005-11-07 03:06:55 +03:00
mfspr r13 ,S P R N _ S P R G 3 / * g e t p a c a a d d r e s s i n t o r13 * /
2005-09-26 10:04:21 +04:00
std r3 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r3 ,S P R N _ D A R
2005-11-07 03:06:55 +03:00
std r9 ,P A C A _ E X S L B + E X _ R 9 ( r13 )
mfcr r9
# ifdef _ _ D I S A B L E D _ _
cmpdi r3 ,0
bge s l b _ m i s s _ u s e r _ i s e r i e s
# endif
std r10 ,P A C A _ E X S L B + E X _ R 1 0 ( r13 )
std r11 ,P A C A _ E X S L B + E X _ R 1 1 ( r13 )
std r12 ,P A C A _ E X S L B + E X _ R 1 2 ( r13 )
mfspr r10 ,S P R N _ S P R G 1
std r10 ,P A C A _ E X S L B + E X _ R 1 3 ( r13 )
ld r12 ,P A C A L P P A C A + L P P A C A S R R 1 ( r13 ) ;
b . s l b _ m i s s _ r e a l m o d e
2005-09-26 10:04:21 +04:00
STD_ E X C E P T I O N _ I S E R I E S ( 0 x40 0 , i n s t r u c t i o n _ a c c e s s , P A C A _ E X G E N )
.globl instruction_access_slb_iSeries
instruction_access_slb_iSeries :
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13 / * s a v e r13 * /
2005-11-07 03:06:55 +03:00
mfspr r13 ,S P R N _ S P R G 3 / * g e t p a c a a d d r e s s i n t o r13 * /
2005-09-26 10:04:21 +04:00
std r3 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
2005-11-07 03:06:55 +03:00
ld r3 ,P A C A L P P A C A + L P P A C A S R R 0 ( r13 ) / * g e t S R R 0 v a l u e * /
std r9 ,P A C A _ E X S L B + E X _ R 9 ( r13 )
mfcr r9
# ifdef _ _ D I S A B L E D _ _
cmpdi r3 ,0
bge . s l b _ m i s s _ u s e r _ i s e r i e s
# endif
std r10 ,P A C A _ E X S L B + E X _ R 1 0 ( r13 )
std r11 ,P A C A _ E X S L B + E X _ R 1 1 ( r13 )
std r12 ,P A C A _ E X S L B + E X _ R 1 2 ( r13 )
mfspr r10 ,S P R N _ S P R G 1
std r10 ,P A C A _ E X S L B + E X _ R 1 3 ( r13 )
ld r12 ,P A C A L P P A C A + L P P A C A S R R 1 ( r13 ) ;
b . s l b _ m i s s _ r e a l m o d e
# ifdef _ _ D I S A B L E D _ _
slb_miss_user_iseries :
std r10 ,P A C A _ E X G E N + E X _ R 1 0 ( r13 )
std r11 ,P A C A _ E X G E N + E X _ R 1 1 ( r13 )
std r12 ,P A C A _ E X G E N + E X _ R 1 2 ( r13 )
mfspr r10 ,S P R G 1
ld r11 ,P A C A _ E X S L B + E X _ R 9 ( r13 )
ld r12 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
std r10 ,P A C A _ E X G E N + E X _ R 1 3 ( r13 )
std r11 ,P A C A _ E X G E N + E X _ R 9 ( r13 )
std r12 ,P A C A _ E X G E N + E X _ R 3 ( r13 )
EXCEPTION_ P R O L O G _ I S E R I E S _ 2
b s l b _ m i s s _ u s e r _ c o m m o n
# endif
2005-09-26 10:04:21 +04:00
MASKABLE_ E X C E P T I O N _ I S E R I E S ( 0 x50 0 , h a r d w a r e _ i n t e r r u p t )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x60 0 , a l i g n m e n t , P A C A _ E X G E N )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x70 0 , p r o g r a m _ c h e c k , P A C A _ E X G E N )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x80 0 , f p _ u n a v a i l a b l e , P A C A _ E X G E N )
MASKABLE_ E X C E P T I O N _ I S E R I E S ( 0 x90 0 , d e c r e m e n t e r )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x a00 , t r a p _ 0 a , P A C A _ E X G E N )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x b00 , t r a p _ 0 b , P A C A _ E X G E N )
.globl system_call_iSeries
system_call_iSeries :
mr r9 ,r13
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3
2005-09-26 10:04:21 +04:00
EXCEPTION_ P R O L O G _ I S E R I E S _ 2
b s y s t e m _ c a l l _ c o m m o n
STD_ E X C E P T I O N _ I S E R I E S ( 0 x d00 , s i n g l e _ s t e p , P A C A _ E X G E N )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x e 0 0 , t r a p _ 0 e , P A C A _ E X G E N )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x f00 , p e r f o r m a n c e _ m o n i t o r , P A C A _ E X G E N )
.globl system_reset_iSeries
system_reset_iSeries :
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3 / * G e t p a c a a d d r e s s * /
2005-09-26 10:04:21 +04:00
mfmsr r24
ori r24 ,r24 ,M S R _ R I
mtmsrd r24 / * R I o n * /
lhz r24 ,P A C A P A C A I N D E X ( r13 ) / * G e t p r o c e s s o r # * /
cmpwi 0 ,r24 ,0 / * A r e w e p r o c e s s o r 0 ? * /
beq . _ _ s t a r t _ i n i t i a l i z a t i o n _ i S e r i e s / * S t a r t u p t h e f i r s t p r o c e s s o r * /
mfspr r4 ,S P R N _ C T R L F
li r5 ,C T R L _ R U N L A T C H / * T u r n o f f t h e r u n l i g h t * /
andc r4 ,r4 ,r5
mtspr S P R N _ C T R L T ,r4
1 :
HMT_ L O W
# ifdef C O N F I G _ S M P
lbz r23 ,P A C A P R O C S T A R T ( r13 ) / * T e s t i f t h i s p r o c e s s o r
* should s t a r t * /
sync
LOADADDR( r3 ,c u r r e n t _ s e t )
sldi r28 ,r24 ,3 / * g e t c u r r e n t _ s e t [ c p u #] * /
ldx r3 ,r3 ,r28
addi r1 ,r3 ,T H R E A D _ S I Z E
subi r1 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
cmpwi 0 ,r23 ,0
beq i S e r i e s _ s e c o n d a r y _ s m p _ l o o p / * L o o p u n t i l t o l d t o g o * /
bne . _ _ s e c o n d a r y _ s t a r t / * L o o p u n t i l t o l d t o g o * /
iSeries_secondary_smp_loop :
/* Let the Hypervisor know we are alive */
/* 8002 is a call to HvCallCfg::getLps, a harmless Hypervisor function */
lis r3 ,0 x80 0 2
rldicr r3 ,r3 ,3 2 ,1 5 / * r0 = ( r3 < < 3 2 ) & 0 x f f f f00 0 0 0 0 0 0 0 0 0 0 * /
# else / * C O N F I G _ S M P * /
/ * Yield t h e p r o c e s s o r . T h i s i s r e q u i r e d f o r n o n - S M P k e r n e l s
which a r e r u n n i n g o n m u l t i - t h r e a d e d m a c h i n e s . * /
lis r3 ,0 x80 0 0
rldicr r3 ,r3 ,3 2 ,1 5 / * r3 = ( r3 < < 3 2 ) & 0 x f f f f00 0 0 0 0 0 0 0 0 0 0 * /
addi r3 ,r3 ,1 8 / * r3 = 0 x80 0 0 0 0 0 0 0 0 0 0 0 0 1 2 w h i c h i s " y i e l d " * /
li r4 ,0 / * " y i e l d t i m e d " * /
li r5 ,- 1 / * " y i e l d f o r e v e r " * /
# endif / * C O N F I G _ S M P * /
li r0 ,- 1 / * r0 = - 1 i n d i c a t e s a H y p e r v i s o r c a l l * /
sc / * I n v o k e t h e h y p e r v i s o r v i a a s y s t e m c a l l * /
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3 / * P u t r13 b a c k ? ? ? ? * /
2005-09-26 10:04:21 +04:00
b 1 b / * I f S M P n o t c o n f i g u r e d , s e c o n d a r i e s
* loop f o r e v e r * /
.globl decrementer_iSeries_masked
decrementer_iSeries_masked :
li r11 ,1
stb r11 ,P A C A L P P A C A + L P P A C A D E C R I N T ( r13 )
lwz r12 ,P A C A D E F A U L T D E C R ( r13 )
mtspr S P R N _ D E C ,r12
/* fall through */
.globl hardware_interrupt_iSeries_masked
hardware_interrupt_iSeries_masked :
mtcrf 0 x80 ,r9 / * R e s t o r e r e g s * /
ld r11 ,P A C A L P P A C A + L P P A C A S R R 0 ( r13 )
ld r12 ,P A C A L P P A C A + L P P A C A S R R 1 ( r13 )
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 0 ,r11
mtspr S P R N _ S R R 1 ,r12
2005-09-26 10:04:21 +04:00
ld r9 ,P A C A _ E X G E N + E X _ R 9 ( r13 )
ld r10 ,P A C A _ E X G E N + E X _ R 1 0 ( r13 )
ld r11 ,P A C A _ E X G E N + E X _ R 1 1 ( r13 )
ld r12 ,P A C A _ E X G E N + E X _ R 1 2 ( r13 )
ld r13 ,P A C A _ E X G E N + E X _ R 1 3 ( r13 )
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
# endif / * C O N F I G _ P P C _ I S E R I E S * /
/*** Common interrupt handlers ***/
STD_ E X C E P T I O N _ C O M M O N ( 0 x10 0 , s y s t e m _ r e s e t , . s y s t e m _ r e s e t _ e x c e p t i o n )
/ *
* Machine c h e c k i s d i f f e r e n t b e c a u s e w e u s e a d i f f e r e n t
* save a r e a : P A C A _ E X M C i n s t e a d o f P A C A _ E X G E N .
* /
.align 7
.globl machine_check_common
machine_check_common :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x20 0 , P A C A _ E X M C )
DISABLE_ I N T S
bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . m a c h i n e _ c h e c k _ e x c e p t i o n
b . r e t _ f r o m _ e x c e p t
STD_ E X C E P T I O N _ C O M M O N _ L I T E ( 0 x90 0 , d e c r e m e n t e r , . t i m e r _ i n t e r r u p t )
STD_ E X C E P T I O N _ C O M M O N ( 0 x a00 , t r a p _ 0 a , . u n k n o w n _ e x c e p t i o n )
STD_ E X C E P T I O N _ C O M M O N ( 0 x b00 , t r a p _ 0 b , . u n k n o w n _ e x c e p t i o n )
STD_ E X C E P T I O N _ C O M M O N ( 0 x d00 , s i n g l e _ s t e p , . s i n g l e _ s t e p _ e x c e p t i o n )
STD_ E X C E P T I O N _ C O M M O N ( 0 x e 0 0 , t r a p _ 0 e , . u n k n o w n _ e x c e p t i o n )
STD_ E X C E P T I O N _ C O M M O N ( 0 x f00 , p e r f o r m a n c e _ m o n i t o r , . p e r f o r m a n c e _ m o n i t o r _ e x c e p t i o n )
STD_ E X C E P T I O N _ C O M M O N ( 0 x13 0 0 , i n s t r u c t i o n _ b r e a k p o i n t , . i n s t r u c t i o n _ b r e a k p o i n t _ e x c e p t i o n )
# ifdef C O N F I G _ A L T I V E C
STD_ E X C E P T I O N _ C O M M O N ( 0 x17 0 0 , a l t i v e c _ a s s i s t , . a l t i v e c _ a s s i s t _ e x c e p t i o n )
# else
STD_ E X C E P T I O N _ C O M M O N ( 0 x17 0 0 , a l t i v e c _ a s s i s t , . u n k n o w n _ e x c e p t i o n )
# endif
/ *
* Here w e h a v e d e t e c t e d t h a t t h e k e r n e l s t a c k p o i n t e r i s b a d .
* R9 c o n t a i n s t h e s a v e d C R , r13 p o i n t s t o t h e p a c a ,
* r1 0 c o n t a i n s t h e ( b a d ) k e r n e l s t a c k p o i n t e r ,
* r1 1 a n d r12 c o n t a i n t h e s a v e d S R R 0 a n d S R R 1 .
* We s w i t c h t o u s i n g a n e m e r g e n c y s t a c k , s a v e t h e r e g i s t e r s t h e r e ,
* and c a l l k e r n e l _ b a d _ s t a c k ( ) , w h i c h p a n i c s .
* /
bad_stack :
ld r1 ,P A C A E M E R G S P ( r13 )
subi r1 ,r1 ,6 4 + I N T _ F R A M E _ S I Z E
std r9 ,_ C C R ( r1 )
std r10 ,G P R 1 ( r1 )
std r11 ,_ N I P ( r1 )
std r12 ,_ M S R ( r1 )
2005-10-10 08:01:07 +04:00
mfspr r11 ,S P R N _ D A R
mfspr r12 ,S P R N _ D S I S R
2005-09-26 10:04:21 +04:00
std r11 ,_ D A R ( r1 )
std r12 ,_ D S I S R ( r1 )
mflr r10
mfctr r11
mfxer r12
std r10 ,_ L I N K ( r1 )
std r11 ,_ C T R ( r1 )
std r12 ,_ X E R ( r1 )
SAVE_ G P R ( 0 ,r1 )
SAVE_ G P R ( 2 ,r1 )
SAVE_ 4 G P R S ( 3 ,r1 )
SAVE_ 2 G P R S ( 7 ,r1 )
SAVE_ 1 0 G P R S ( 1 2 ,r1 )
SAVE_ 1 0 G P R S ( 2 2 ,r1 )
addi r11 ,r1 ,I N T _ F R A M E _ S I Z E
std r11 ,0 ( r1 )
li r12 ,0
std r12 ,0 ( r11 )
ld r2 ,P A C A T O C ( r13 )
1 : addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . k e r n e l _ b a d _ s t a c k
b 1 b
/ *
* Return f r o m a n e x c e p t i o n w i t h m i n i m a l c h e c k s .
* The c a l l e r i s a s s u m e d t o h a v e d o n e E X C E P T I O N _ P R O L O G _ C O M M O N .
* If i n t e r r u p t s h a v e b e e n e n a b l e d , o r a n y t h i n g h a s b e e n
* done t h a t m i g h t h a v e c h a n g e d t h e s c h e d u l i n g s t a t u s o f
* any t a s k o r s e n t a n y t a s k a s i g n a l , y o u s h o u l d u s e
* ret_ f r o m _ e x c e p t o r r e t _ f r o m _ e x c e p t _ l i t e i n s t e a d o f t h i s .
* /
2005-10-10 16:50:37 +04:00
.globl fast_exception_return
2005-09-26 10:04:21 +04:00
fast_exception_return :
ld r12 ,_ M S R ( r1 )
ld r11 ,_ N I P ( r1 )
andi. r3 ,r12 ,M S R _ R I / * c h e c k i f R I i s s e t * /
beq- u n r e c o v _ f e r
ld r3 ,_ C C R ( r1 )
ld r4 ,_ L I N K ( r1 )
ld r5 ,_ C T R ( r1 )
ld r6 ,_ X E R ( r1 )
mtcr r3
mtlr r4
mtctr r5
mtxer r6
REST_ G P R ( 0 , r1 )
REST_ 8 G P R S ( 2 , r1 )
mfmsr r10
clrrdi r10 ,r10 ,2 / * c l e a r R I ( L E i s 0 a l r e a d y ) * /
mtmsrd r10 ,1
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 1 ,r12
mtspr S P R N _ S R R 0 ,r11
2005-09-26 10:04:21 +04:00
REST_ 4 G P R S ( 1 0 , r1 )
ld r1 ,G P R 1 ( r1 )
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
unrecov_fer :
bl . s a v e _ n v g p r s
1 : addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . u n r e c o v e r a b l e _ e x c e p t i o n
b 1 b
/ *
* Here r13 p o i n t s t o t h e p a c a , r9 c o n t a i n s t h e s a v e d C R ,
* SRR0 a n d S R R 1 a r e s a v e d i n r11 a n d r12 ,
* r9 - r13 a r e s a v e d i n p a c a - > e x g e n .
* /
.align 7
.globl data_access_common
data_access_common :
RUNLATCH_ O N ( r10 ) / * I t w o n t f i t i n t h e 0 x30 0 h a n d l e r * /
2005-10-10 08:01:07 +04:00
mfspr r10 ,S P R N _ D A R
2005-09-26 10:04:21 +04:00
std r10 ,P A C A _ E X G E N + E X _ D A R ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r10 ,S P R N _ D S I S R
2005-09-26 10:04:21 +04:00
stw r10 ,P A C A _ E X G E N + E X _ D S I S R ( r13 )
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x30 0 , P A C A _ E X G E N )
ld r3 ,P A C A _ E X G E N + E X _ D A R ( r13 )
lwz r4 ,P A C A _ E X G E N + E X _ D S I S R ( r13 )
li r5 ,0 x30 0
b . d o _ h a s h _ p a g e / * T r y t o h a n d l e a s h p t e f a u l t * /
.align 7
.globl instruction_access_common
instruction_access_common :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x40 0 , P A C A _ E X G E N )
ld r3 ,_ N I P ( r1 )
andis. r4 ,r12 ,0 x58 2 0
li r5 ,0 x40 0
b . d o _ h a s h _ p a g e / * T r y t o h a n d l e a s h p t e f a u l t * /
2005-11-07 03:06:55 +03:00
/ *
* Here i s t h e c o m m o n S L B m i s s u s e r t h a t i s u s e d w h e n g o i n g t o v i r t u a l
* mode f o r S L B m i s s e s , t h a t i s c u r r e n t l y n o t u s e d
* /
# ifdef _ _ D I S A B L E D _ _
.align 7
.globl slb_miss_user_common
slb_miss_user_common :
mflr r10
std r3 ,P A C A _ E X G E N + E X _ D A R ( r13 )
stw r9 ,P A C A _ E X G E N + E X _ C C R ( r13 )
std r10 ,P A C A _ E X G E N + E X _ L R ( r13 )
std r11 ,P A C A _ E X G E N + E X _ S R R 0 ( r13 )
bl . s l b _ a l l o c a t e _ u s e r
ld r10 ,P A C A _ E X G E N + E X _ L R ( r13 )
ld r3 ,P A C A _ E X G E N + E X _ R 3 ( r13 )
lwz r9 ,P A C A _ E X G E N + E X _ C C R ( r13 )
ld r11 ,P A C A _ E X G E N + E X _ S R R 0 ( r13 )
mtlr r10
beq- s l b _ m i s s _ f a u l t
andi. r10 ,r12 ,M S R _ R I / * c h e c k f o r u n r e c o v e r a b l e e x c e p t i o n * /
beq- u n r e c o v _ u s e r _ s l b
mfmsr r10
.machine push
.machine " power4 "
mtcrf 0 x80 ,r9
.machine pop
clrrdi r10 ,r10 ,2 / * c l e a r R I b e f o r e s e t t i n g S R R 0 / 1 * /
mtmsrd r10 ,1
mtspr S R R 0 ,r11
mtspr S R R 1 ,r12
ld r9 ,P A C A _ E X G E N + E X _ R 9 ( r13 )
ld r10 ,P A C A _ E X G E N + E X _ R 1 0 ( r13 )
ld r11 ,P A C A _ E X G E N + E X _ R 1 1 ( r13 )
ld r12 ,P A C A _ E X G E N + E X _ R 1 2 ( r13 )
ld r13 ,P A C A _ E X G E N + E X _ R 1 3 ( r13 )
rfid
b .
slb_miss_fault :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x38 0 , P A C A _ E X G E N )
ld r4 ,P A C A _ E X G E N + E X _ D A R ( r13 )
li r5 ,0
std r4 ,_ D A R ( r1 )
std r5 ,_ D S I S R ( r1 )
b . h a n d l e _ p a g e _ f a u l t
unrecov_user_slb :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x42 0 0 , P A C A _ E X G E N )
DISABLE_ I N T S
bl . s a v e _ n v g p r s
1 : addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . u n r e c o v e r a b l e _ e x c e p t i o n
b 1 b
# endif / * _ _ D I S A B L E D _ _ * /
/ *
* r1 3 p o i n t s t o t h e P A C A , r9 c o n t a i n s t h e s a v e d C R ,
* r1 2 c o n t a i n t h e s a v e d S R R 1 , S R R 0 i s s t i l l r e a d y f o r r e t u r n
* r3 h a s t h e f a u l t i n g a d d r e s s
* r9 - r13 a r e s a v e d i n p a c a - > e x s l b .
* r3 i s s a v e d i n p a c a - > s l b _ r3
* We a s s u m e w e a r e n ' t g o i n g t o t a k e a n y e x c e p t i o n s d u r i n g t h i s p r o c e d u r e .
* /
_ GLOBAL( s l b _ m i s s _ r e a l m o d e )
mflr r10
stw r9 ,P A C A _ E X S L B + E X _ C C R ( r13 ) / * s a v e C R i n e x c . f r a m e * /
std r10 ,P A C A _ E X S L B + E X _ L R ( r13 ) / * s a v e L R * /
bl . s l b _ a l l o c a t e _ r e a l m o d e
/* All done -- return from exception. */
ld r10 ,P A C A _ E X S L B + E X _ L R ( r13 )
ld r3 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
lwz r9 ,P A C A _ E X S L B + E X _ C C R ( r13 ) / * g e t s a v e d C R * /
# ifdef C O N F I G _ P P C _ I S E R I E S
ld r11 ,P A C A L P P A C A + L P P A C A S R R 0 ( r13 ) / * g e t S R R 0 v a l u e * /
# endif / * C O N F I G _ P P C _ I S E R I E S * /
mtlr r10
andi. r10 ,r12 ,M S R _ R I / * c h e c k f o r u n r e c o v e r a b l e e x c e p t i o n * /
beq- u n r e c o v _ s l b
.machine push
.machine " power4 "
mtcrf 0 x80 ,r9
mtcrf 0 x01 ,r9 / * s l b _ a l l o c a t e u s e s c r0 a n d c r7 * /
.machine pop
# ifdef C O N F I G _ P P C _ I S E R I E S
mtspr S P R N _ S R R 0 ,r11
mtspr S P R N _ S R R 1 ,r12
# endif / * C O N F I G _ P P C _ I S E R I E S * /
ld r9 ,P A C A _ E X S L B + E X _ R 9 ( r13 )
ld r10 ,P A C A _ E X S L B + E X _ R 1 0 ( r13 )
ld r11 ,P A C A _ E X S L B + E X _ R 1 1 ( r13 )
ld r12 ,P A C A _ E X S L B + E X _ R 1 2 ( r13 )
ld r13 ,P A C A _ E X S L B + E X _ R 1 3 ( r13 )
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
unrecov_slb :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x41 0 0 , P A C A _ E X S L B )
DISABLE_ I N T S
bl . s a v e _ n v g p r s
1 : addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . u n r e c o v e r a b l e _ e x c e p t i o n
b 1 b
2005-09-26 10:04:21 +04:00
.align 7
.globl hardware_interrupt_common
.globl hardware_interrupt_entry
hardware_interrupt_common :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x50 0 , P A C A _ E X G E N )
hardware_interrupt_entry :
DISABLE_ I N T S
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . d o _ I R Q
b . r e t _ f r o m _ e x c e p t _ l i t e
.align 7
.globl alignment_common
alignment_common :
2005-10-10 08:01:07 +04:00
mfspr r10 ,S P R N _ D A R
2005-09-26 10:04:21 +04:00
std r10 ,P A C A _ E X G E N + E X _ D A R ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r10 ,S P R N _ D S I S R
2005-09-26 10:04:21 +04:00
stw r10 ,P A C A _ E X G E N + E X _ D S I S R ( r13 )
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x60 0 , P A C A _ E X G E N )
ld r3 ,P A C A _ E X G E N + E X _ D A R ( r13 )
lwz r4 ,P A C A _ E X G E N + E X _ D S I S R ( r13 )
std r3 ,_ D A R ( r1 )
std r4 ,_ D S I S R ( r1 )
bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
ENABLE_ I N T S
bl . a l i g n m e n t _ e x c e p t i o n
b . r e t _ f r o m _ e x c e p t
.align 7
.globl program_check_common
program_check_common :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x70 0 , P A C A _ E X G E N )
bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
ENABLE_ I N T S
bl . p r o g r a m _ c h e c k _ e x c e p t i o n
b . r e t _ f r o m _ e x c e p t
.align 7
.globl fp_unavailable_common
fp_unavailable_common :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x80 0 , P A C A _ E X G E N )
bne . l o a d _ u p _ f p u / * i f f r o m u s e r , j u s t l o a d i t u p * /
bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
ENABLE_ I N T S
bl . k e r n e l _ f p _ u n a v a i l a b l e _ e x c e p t i o n
BUG_ O P C O D E
.align 7
.globl altivec_unavailable_common
altivec_unavailable_common :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x f20 , P A C A _ E X G E N )
# ifdef C O N F I G _ A L T I V E C
BEGIN_ F T R _ S E C T I O N
bne . l o a d _ u p _ a l t i v e c / * i f f r o m u s e r , j u s t l o a d i t u p * /
END_ F T R _ S E C T I O N _ I F S E T ( C P U _ F T R _ A L T I V E C )
# endif
bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
ENABLE_ I N T S
bl . a l t i v e c _ u n a v a i l a b l e _ e x c e p t i o n
b . r e t _ f r o m _ e x c e p t
# ifdef C O N F I G _ A L T I V E C
/ *
* load_ u p _ a l t i v e c ( u n u s e d , u n u s e d , t s k )
* Disable V M X f o r t h e t a s k w h i c h h a d i t p r e v i o u s l y ,
* and s a v e i t s v e c t o r r e g i s t e r s i n i t s t h r e a d _ s t r u c t .
* Enables t h e V M X f o r u s e i n t h e k e r n e l o n r e t u r n .
* On S M P w e k n o w t h e V M X i s f r e e , s i n c e w e g i v e i t u p e v e r y
* switch ( i e , n o l a z y s a v e o f t h e v e c t o r r e g i s t e r s ) .
* On e n t r y : r13 = = ' c u r r e n t ' & & l a s t _ t a s k _ u s e d _ a l t i v e c ! = ' c u r r e n t '
* /
_ STATIC( l o a d _ u p _ a l t i v e c )
mfmsr r5 / * g r a b t h e c u r r e n t M S R * /
oris r5 ,r5 ,M S R _ V E C @h
mtmsrd r5 / * e n a b l e u s e o f V M X n o w * /
isync
/ *
* For S M P , w e d o n ' t d o l a z y V M X s w i t c h i n g b e c a u s e i t j u s t g e t s t o o
* horrendously c o m p l e x , e s p e c i a l l y w h e n a t a s k s w i t c h e s f r o m o n e C P U
* to a n o t h e r . I n s t e a d w e c a l l g i v e u p _ a l t v e c i n s w i t c h _ t o .
* VRSAVE i s n ' t d e a l t w i t h h e r e , t h a t i s d o n e i n t h e n o r m a l c o n t e x t
* switch c o d e . N o t e t h a t w e c o u l d r e l y o n v r s a v e v a l u e t o e v e n t u a l l y
* avoid s a v i n g a l l o f t h e V R E G s h e r e . . .
* /
# ifndef C O N F I G _ S M P
ld r3 ,l a s t _ t a s k _ u s e d _ a l t i v e c @got(r2)
ld r4 ,0 ( r3 )
cmpdi 0 ,r4 ,0
beq 1 f
/* Save VMX state to last_task_used_altivec's THREAD struct */
addi r4 ,r4 ,T H R E A D
SAVE_ 3 2 V R S ( 0 ,r5 ,r4 )
mfvscr v r0
li r10 ,T H R E A D _ V S C R
stvx v r0 ,r10 ,r4
/* Disable VMX for last_task_used_altivec */
ld r5 ,P T _ R E G S ( r4 )
ld r4 ,_ M S R - S T A C K _ F R A M E _ O V E R H E A D ( r5 )
lis r6 ,M S R _ V E C @h
andc r4 ,r4 ,r6
std r4 ,_ M S R - S T A C K _ F R A M E _ O V E R H E A D ( r5 )
1 :
# endif / * C O N F I G _ S M P * /
/ * Hack : if w e g e t a n a l t i v e c u n a v a i l a b l e t r a p w i t h V R S A V E
* set t o a l l z e r o s , w e a s s u m e t h i s i s a b r o k e n a p p l i c a t i o n
* that f a i l s t o s e t i t p r o p e r l y , a n d t h u s w e s w i t c h i t t o
* all 1 ' s
* /
mfspr r4 ,S P R N _ V R S A V E
cmpdi 0 ,r4 ,0
bne+ 1 f
li r4 ,- 1
mtspr S P R N _ V R S A V E ,r4
1 :
/* enable use of VMX after return */
ld r4 ,P A C A C U R R E N T ( r13 )
addi r5 ,r4 ,T H R E A D / * G e t T H R E A D * /
oris r12 ,r12 ,M S R _ V E C @h
std r12 ,_ M S R ( r1 )
li r4 ,1
li r10 ,T H R E A D _ V S C R
stw r4 ,T H R E A D _ U S E D _ V R ( r5 )
lvx v r0 ,r10 ,r5
mtvscr v r0
REST_ 3 2 V R S ( 0 ,r4 ,r5 )
# ifndef C O N F I G _ S M P
/* Update last_task_used_math to 'current' */
subi r4 ,r5 ,T H R E A D / * B a c k t o ' c u r r e n t ' * /
std r4 ,0 ( r3 )
# endif / * C O N F I G _ S M P * /
/* restore registers and return */
b f a s t _ e x c e p t i o n _ r e t u r n
# endif / * C O N F I G _ A L T I V E C * /
/ *
* Hash t a b l e s t u f f
* /
.align 7
_ GLOBAL( d o _ h a s h _ p a g e )
std r3 ,_ D A R ( r1 )
std r4 ,_ D S I S R ( r1 )
andis. r0 ,r4 ,0 x a45 0 / * w e i r d e r r o r ? * /
bne- . h a n d l e _ p a g e _ f a u l t / * i f n o t , t r y t o i n s e r t a H P T E * /
BEGIN_ F T R _ S E C T I O N
andis. r0 ,r4 ,0 x00 2 0 / * I s i t a s e g m e n t t a b l e f a u l t ? * /
bne- . d o _ s t e _ a l l o c / * I f s o h a n d l e i t * /
END_ F T R _ S E C T I O N _ I F C L R ( C P U _ F T R _ S L B )
/ *
* We n e e d t o s e t t h e _ P A G E _ U S E R b i t i f M S R _ P R i s s e t o r i f w e a r e
* accessing a u s e r s p a c e s e g m e n t ( e v e n f r o m t h e k e r n e l ) . W e a s s u m e
* kernel a d d r e s s e s a l w a y s h a v e t h e h i g h b i t s e t .
* /
rlwinm r4 ,r4 ,3 2 - 2 5 + 9 ,3 1 - 9 ,3 1 - 9 / * D S I S R _ S T O R E - > _ P A G E _ R W * /
rotldi r0 ,r3 ,1 5 / * M o v e h i g h b i t i n t o M S R _ P R p o s n * /
orc r0 ,r12 ,r0 / * M S R _ P R | ~ h i g h _ b i t * /
rlwimi r4 ,r0 ,3 2 - 1 3 ,3 0 ,3 0 / * b e c o m e s _ P A G E _ U S E R a c c e s s b i t * /
ori r4 ,r4 ,1 / * a d d _ P A G E _ P R E S E N T * /
rlwimi r4 ,r5 ,2 2 + 2 ,3 1 - 2 ,3 1 - 2 / * S e t _ P A G E _ E X E C i f t r a p i s 0 x40 0 * /
/ *
* On i S e r i e s , w e s o f t - d i s a b l e i n t e r r u p t s h e r e , t h e n
* hard- e n a b l e i n t e r r u p t s s o t h a t t h e h a s h _ p a g e c o d e c a n s p i n o n
* the h a s h _ t a b l e _ l o c k w i t h o u t p r o b l e m s o n a s h a r e d p r o c e s s o r .
* /
DISABLE_ I N T S
/ *
* r3 c o n t a i n s t h e f a u l t i n g a d d r e s s
* r4 c o n t a i n s t h e r e q u i r e d a c c e s s p e r m i s s i o n s
* r5 c o n t a i n s t h e t r a p n u m b e r
*
* at r e t u r n r3 = 0 f o r s u c c e s s
* /
bl . h a s h _ p a g e / * b u i l d H P T E i f p o s s i b l e * /
cmpdi r3 ,0 / * s e e i f h a s h _ p a g e s u c c e e d e d * /
# ifdef D O _ S O F T _ D I S A B L E
/ *
* If w e h a d i n t e r r u p t s s o f t - e n a b l e d a t t h e p o i n t w h e r e t h e
* DSI/ I S I o c c u r r e d , a n d a n i n t e r r u p t c a m e i n d u r i n g h a s h _ p a g e ,
* handle i t n o w .
* We j u m p t o r e t _ f r o m _ e x c e p t _ l i t e r a t h e r t h a n f a s t _ e x c e p t i o n _ r e t u r n
* because r e t _ f r o m _ e x c e p t _ l i t e w i l l c h e c k f o r a n d h a n d l e p e n d i n g
* interrupts i f n e c e s s a r y .
* /
beq . r e t _ f r o m _ e x c e p t _ l i t e
/* For a hash failure, we don't bother re-enabling interrupts */
ble- 1 2 f
/ *
* hash_ p a g e c o u l d n ' t h a n d l e i t , s e t s o f t i n t e r r u p t e n a b l e b a c k
* to w h a t i t w a s b e f o r e t h e t r a p . N o t e t h a t . l o c a l _ i r q _ r e s t o r e
* handles a n y i n t e r r u p t s p e n d i n g a t t h i s p o i n t .
* /
ld r3 ,S O F T E ( r1 )
bl . l o c a l _ i r q _ r e s t o r e
b 1 1 f
# else
beq f a s t _ e x c e p t i o n _ r e t u r n / * R e t u r n f r o m e x c e p t i o n o n s u c c e s s * /
ble- 1 2 f / * F a i l u r e r e t u r n f r o m h a s h _ p a g e * /
/* fall through */
# endif
/* Here we have a page fault that hash_page can't handle. */
_ GLOBAL( h a n d l e _ p a g e _ f a u l t )
ENABLE_ I N T S
11 : ld r4 ,_ D A R ( r1 )
ld r5 ,_ D S I S R ( r1 )
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . d o _ p a g e _ f a u l t
cmpdi r3 ,0
beq+ . r e t _ f r o m _ e x c e p t _ l i t e
bl . s a v e _ n v g p r s
mr r5 ,r3
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
lwz r4 ,_ D A R ( r1 )
bl . b a d _ p a g e _ f a u l t
b . r e t _ f r o m _ e x c e p t
/ * We h a v e a p a g e f a u l t t h a t h a s h _ p a g e c o u l d h a n d l e b u t H V r e f u s e d
* the P T E i n s e r t i o n
* /
12 : bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
lwz r4 ,_ D A R ( r1 )
bl . l o w _ h a s h _ f a u l t
b . r e t _ f r o m _ e x c e p t
/* here we have a segment miss */
_ GLOBAL( d o _ s t e _ a l l o c )
bl . s t e _ a l l o c a t e / * t r y t o i n s e r t s t a b e n t r y * /
cmpdi r3 ,0
beq+ f a s t _ e x c e p t i o n _ r e t u r n
b . h a n d l e _ p a g e _ f a u l t
/ *
* r1 3 p o i n t s t o t h e P A C A , r9 c o n t a i n s t h e s a v e d C R ,
* r1 1 a n d r12 c o n t a i n t h e s a v e d S R R 0 a n d S R R 1 .
* r9 - r13 a r e s a v e d i n p a c a - > e x s l b .
* We a s s u m e w e a r e n ' t g o i n g t o t a k e a n y e x c e p t i o n s d u r i n g t h i s p r o c e d u r e .
* We a s s u m e ( D A R > > 6 0 ) = = 0 x c .
* /
.align 7
_ GLOBAL( d o _ s t a b _ b o l t e d )
stw r9 ,P A C A _ E X S L B + E X _ C C R ( r13 ) / * s a v e C R i n e x c . f r a m e * /
std r11 ,P A C A _ E X S L B + E X _ S R R 0 ( r13 ) / * s a v e S R R 0 i n e x c . f r a m e * /
/* Hash to the primary group */
ld r10 ,P A C A S T A B V I R T ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r11 ,S P R N _ D A R
2005-09-26 10:04:21 +04:00
srdi r11 ,r11 ,2 8
rldimi r10 ,r11 ,7 ,5 2 / * r10 = f i r s t s t e o f t h e g r o u p * /
/* Calculate VSID */
/* This is a kernel address, so protovsid = ESID */
ASM_ V S I D _ S C R A M B L E ( r11 , r9 )
rldic r9 ,r11 ,1 2 ,1 6 / * r9 = v s i d < < 1 2 * /
/* Search the primary group for a free entry */
1 : ld r11 ,0 ( r10 ) / * T e s t v a l i d b i t o f t h e c u r r e n t s t e * /
andi. r11 ,r11 ,0 x80
beq 2 f
addi r10 ,r10 ,1 6
andi. r11 ,r10 ,0 x70
bne 1 b
/* Stick for only searching the primary group for now. */
/* At least for now, we use a very simple random castout scheme */
/* Use the TB as a random number ; OR in 1 to avoid entry 0 */
mftb r11
rldic r11 ,r11 ,4 ,5 7 / * r11 = ( r11 < < 4 ) & 0 x70 * /
ori r11 ,r11 ,0 x10
/* r10 currently points to an ste one past the group of interest */
/* make it point to the randomly selected entry */
subi r10 ,r10 ,1 2 8
or r10 ,r10 ,r11 / * r10 i s t h e e n t r y t o i n v a l i d a t e * /
isync / * m a r k t h e e n t r y i n v a l i d * /
ld r11 ,0 ( r10 )
rldicl r11 ,r11 ,5 6 ,1 / * c l e a r t h e v a l i d b i t * /
rotldi r11 ,r11 ,8
std r11 ,0 ( r10 )
sync
clrrdi r11 ,r11 ,2 8 / * G e t t h e e s i d p a r t o f t h e s t e * /
slbie r11
2 : std r9 ,8 ( r10 ) / * S t o r e t h e v s i d p a r t o f t h e s t e * /
eieio
2005-10-10 08:01:07 +04:00
mfspr r11 ,S P R N _ D A R / * G e t t h e n e w e s i d * /
2005-09-26 10:04:21 +04:00
clrrdi r11 ,r11 ,2 8 / * P e r m i t s a f u l l 3 2 b o f E S I D * /
ori r11 ,r11 ,0 x90 / * T u r n o n v a l i d a n d k p * /
std r11 ,0 ( r10 ) / * P u t n e w e n t r y b a c k i n t o t h e s t a b * /
sync
/* All done -- return from exception. */
lwz r9 ,P A C A _ E X S L B + E X _ C C R ( r13 ) / * g e t s a v e d C R * /
ld r11 ,P A C A _ E X S L B + E X _ S R R 0 ( r13 ) / * g e t s a v e d S R R 0 * /
andi. r10 ,r12 ,M S R _ R I
beq- u n r e c o v _ s l b
mtcrf 0 x80 ,r9 / * r e s t o r e C R * /
mfmsr r10
clrrdi r10 ,r10 ,2
mtmsrd r10 ,1
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 0 ,r11
mtspr S P R N _ S R R 1 ,r12
2005-09-26 10:04:21 +04:00
ld r9 ,P A C A _ E X S L B + E X _ R 9 ( r13 )
ld r10 ,P A C A _ E X S L B + E X _ R 1 0 ( r13 )
ld r11 ,P A C A _ E X S L B + E X _ R 1 1 ( r13 )
ld r12 ,P A C A _ E X S L B + E X _ R 1 2 ( r13 )
ld r13 ,P A C A _ E X S L B + E X _ R 1 3 ( r13 )
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
/ *
* Space f o r C P U 0 ' s s e g m e n t t a b l e .
*
* On i S e r i e s , t h e h y p e r v i s o r m u s t f i l l i n a t l e a s t o n e e n t r y b e f o r e
* we g e t c o n t r o l ( w i t h r e l o c a t e o n ) . T h e a d d r e s s i s g i v e t o t h e h v
2005-09-29 05:50:22 +04:00
* as a p a g e n u m b e r ( s e e x L p a r M a p i n l p a r d a t a . c ) , s o t h i s m u s t b e a t a
2005-09-26 10:04:21 +04:00
* fixed a d d r e s s ( t h e l i n k e r c a n ' t c o m p u t e ( u 6 4 ) & i n i t i a l _ s t a b > >
* PAGE_ S H I F T ) .
* /
. = STAB0 _ P H Y S _ A D D R / * 0 x60 0 0 * /
.globl initial_stab
initial_stab :
.space 4096
/ *
* Data a r e a r e s e r v e d f o r F W N M I o p t i o n .
* This a d d r e s s ( 0 x70 0 0 ) i s f i x e d b y t h e R P A .
* /
. = 0 x7 0 0 0
.globl fwnmi_data_area
fwnmi_data_area :
/ * iSeries d o e s n o t u s e t h e F W N M I s t u f f , s o i t i s s a f e t o p u t
* this h e r e , e v e n i f w e l a t e r a l l o w k e r n e l s t h a t w i l l b o o t o n
* both p S e r i e s a n d i S e r i e s * /
# ifdef C O N F I G _ P P C _ I S E R I E S
. = LPARMAP_ P H Y S
# include " l p a r m a p . s "
/ *
* This " . t e x t " i s h e r e f o r o l d c o m p i l e r s t h a t g e n e r a t e a t r a i l i n g
* .note section when c o m p i l i n g . c f i l e s t o . s
* /
.text
# endif / * C O N F I G _ P P C _ I S E R I E S * /
. = 0 x8 0 0 0
/ *
* On p S e r i e s , s e c o n d a r y p r o c e s s o r s s p i n i n t h e f o l l o w i n g c o d e .
* At e n t r y , r3 = t h i s p r o c e s s o r ' s n u m b e r ( p h y s i c a l c p u i d )
* /
_ GLOBAL( p S e r i e s _ s e c o n d a r y _ s m p _ i n i t )
mr r24 ,r3
/* turn on 64-bit mode */
bl . e n a b l e _ 6 4 b _ m o d e
isync
/* Copy some CPU settings from CPU 0 */
bl . _ _ r e s t o r e _ c p u _ s e t u p
/ * Set u p a p a c a v a l u e f o r t h i s p r o c e s s o r . S i n c e w e h a v e t h e
* physical c p u i d i n r24 , w e n e e d t o s e a r c h t h e p a c a s t o f i n d
* which l o g i c a l i d m a p s t o o u r p h y s i c a l o n e .
* /
LOADADDR( r13 , p a c a ) / * G e t b a s e v a d d r o f p a c a a r r a y * /
li r5 ,0 / * l o g i c a l c p u i d * /
1 : lhz r6 ,P A C A H W C P U I D ( r13 ) / * L o a d H W p r o c i d f r o m p a c a * /
cmpw r6 ,r24 / * C o m p a r e t o o u r i d * /
beq 2 f
addi r13 ,r13 ,P A C A _ S I Z E / * L o o p t o n e x t P A C A o n m i s s * /
addi r5 ,r5 ,1
cmpwi r5 ,N R _ C P U S
blt 1 b
mr r3 ,r24 / * n o t f o u n d , c o p y p h y s t o r3 * /
b . k e x e c _ w a i t / * n e x t k e r n e l m i g h t d o b e t t e r * /
2005-10-10 08:01:07 +04:00
2 : mtspr S P R N _ S P R G 3 ,r13 / * S a v e v a d d r o f p a c a i n S P R G 3 * /
2005-09-26 10:04:21 +04:00
/* From now on, r24 is expected to be logical cpuid */
mr r24 ,r5
3 : HMT_ L O W
lbz r23 ,P A C A P R O C S T A R T ( r13 ) / * T e s t i f t h i s p r o c e s s o r s h o u l d * /
/* start. */
sync
/* Create a temp kernel stack for use before relocation is on. */
ld r1 ,P A C A E M E R G S P ( r13 )
subi r1 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
cmpwi 0 ,r23 ,0
# ifdef C O N F I G _ S M P
bne . _ _ s e c o n d a r y _ s t a r t
# endif
b 3 b / * L o o p u n t i l t o l d t o g o * /
# ifdef C O N F I G _ P P C _ I S E R I E S
_ STATIC( _ _ s t a r t _ i n i t i a l i z a t i o n _ i S e r i e s )
/* Clear out the BSS */
LOADADDR( r11 ,_ _ b s s _ s t o p )
LOADADDR( r8 ,_ _ b s s _ s t a r t )
sub r11 ,r11 ,r8 / * b s s s i z e * /
addi r11 ,r11 ,7 / * r o u n d u p t o a n e v e n d o u b l e w o r d * /
rldicl. r11 ,r11 ,6 1 ,3 / * s h i f t r i g h t b y 3 * /
beq 4 f
addi r8 ,r8 ,- 8
li r0 ,0
mtctr r11 / * z e r o t h i s m a n y d o u b l e w o r d s * /
3 : stdu r0 ,8 ( r8 )
bdnz 3 b
4 :
LOADADDR( r1 ,i n i t _ t h r e a d _ u n i o n )
addi r1 ,r1 ,T H R E A D _ S I Z E
li r0 ,0
stdu r0 ,- S T A C K _ F R A M E _ O V E R H E A D ( r1 )
LOADADDR( r3 ,c p u _ s p e c s )
LOADADDR( r4 ,c u r _ c p u _ s p e c )
li r5 ,0
bl . i d e n t i f y _ c p u
LOADADDR( r2 ,_ _ t o c _ s t a r t )
addi r2 ,r2 ,0 x40 0 0
addi r2 ,r2 ,0 x40 0 0
bl . i S e r i e s _ e a r l y _ s e t u p
2005-09-29 05:50:22 +04:00
bl . e a r l y _ s e t u p
2005-09-26 10:04:21 +04:00
/* relocation is on at this point */
b . s t a r t _ h e r e _ c o m m o n
# endif / * C O N F I G _ P P C _ I S E R I E S * /
# ifdef C O N F I G _ P P C _ M U L T I P L A T F O R M
_ STATIC( _ _ m m u _ o f f )
mfmsr r3
andi. r0 ,r3 ,M S R _ I R | M S R _ D R
beqlr
andc r3 ,r3 ,r0
mtspr S P R N _ S R R 0 ,r4
mtspr S P R N _ S R R 1 ,r3
sync
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
/ *
* Here i s o u r m a i n k e r n e l e n t r y p o i n t . W e s u p p o r t c u r r e n t l y 2 k i n d o f e n t r i e s
* depending o n t h e v a l u e o f r5 .
*
* r5 ! = N U L L - > O F e n t r y , w e g o t o p r o m _ i n i t , " l e g a c y " p a r a m e t e r c o n t e n t
* in r3 . . . r7
*
* r5 = = N U L L - > k e x e c s t y l e e n t r y . r3 i s a p h y s i c a l p o i n t e r t o t h e
* DT b l o c k , r4 i s a p h y s i c a l p o i n t e r t o t h e k e r n e l i t s e l f
*
* /
_ GLOBAL( _ _ s t a r t _ i n i t i a l i z a t i o n _ m u l t i p l a t f o r m )
/ *
* Are w e b o o t e d f r o m a P R O M O f - t y p e c l i e n t - i n t e r f a c e ?
* /
cmpldi c r0 ,r5 ,0
bne . _ _ b o o t _ f r o m _ p r o m / * y e s - > p r o m * /
/* Save parameters */
mr r31 ,r3
mr r30 ,r4
/* Make sure we are running in 64 bits mode */
bl . e n a b l e _ 6 4 b _ m o d e
/* Setup some critical 970 SPRs before switching MMU off */
bl . _ _ 9 7 0 _ c p u _ p r e i n i t
/* cpu # */
li r24 ,0
/* Switch off MMU if not already */
LOADADDR( r4 , . _ _ a f t e r _ p r o m _ s t a r t - K E R N E L B A S E )
add r4 ,r4 ,r30
bl . _ _ m m u _ o f f
b . _ _ a f t e r _ p r o m _ s t a r t
_ STATIC( _ _ b o o t _ f r o m _ p r o m )
/* Save parameters */
mr r31 ,r3
mr r30 ,r4
mr r29 ,r5
mr r28 ,r6
mr r27 ,r7
/* Make sure we are running in 64 bits mode */
bl . e n a b l e _ 6 4 b _ m o d e
/* put a relocation offset into r3 */
bl . r e l o c _ o f f s e t
LOADADDR( r2 ,_ _ t o c _ s t a r t )
addi r2 ,r2 ,0 x40 0 0
addi r2 ,r2 ,0 x40 0 0
/* Relocate the TOC from a virt addr to a real addr */
2005-10-10 16:41:25 +04:00
add r2 ,r2 ,r3
2005-09-26 10:04:21 +04:00
/* Restore parameters */
mr r3 ,r31
mr r4 ,r30
mr r5 ,r29
mr r6 ,r28
mr r7 ,r27
/* Do all of the interaction with OF client interface */
bl . p r o m _ i n i t
/* We never return */
trap
/ *
* At t h i s p o i n t , r3 c o n t a i n s t h e p h y s i c a l a d d r e s s w e a r e r u n n i n g a t ,
* returned b y p r o m _ i n i t ( )
* /
_ STATIC( _ _ a f t e r _ p r o m _ s t a r t )
/ *
* We n e e d t o r u n w i t h _ _ s t a r t a t p h y s i c a l a d d r e s s 0 .
* This w i l l l e a v e s o m e c o d e i n t h e f i r s t 2 5 6 B o f
* real m e m o r y , w h i c h a r e r e s e r v e d f o r s o f t w a r e u s e .
* The r e m a i n d e r o f t h e f i r s t p a g e i s l o a d e d w i t h t h e f i x e d
* interrupt v e c t o r s . T h e n e x t t w o p a g e s a r e f i l l e d w i t h
* unknown e x c e p t i o n p l a c e h o l d e r s .
*
* Note : This p r o c e s s o v e r w r i t e s t h e O F e x c e p t i o n v e c t o r s .
* r2 6 = = r e l o c a t i o n o f f s e t
* r2 7 = = K E R N E L B A S E
* /
bl . r e l o c _ o f f s e t
mr r26 ,r3
SET_ R E G _ T O _ C O N S T ( r27 ,K E R N E L B A S E )
li r3 ,0 / * t a r g e t a d d r * /
/ / XXX F I X M E : U s e p h y s r e t u r n e d b y O F ( r30 )
2005-10-10 16:41:25 +04:00
add r4 ,r27 ,r26 / * s o u r c e a d d r * /
2005-09-26 10:04:21 +04:00
/* current address of _start */
/* i.e. where we are running */
/* the source addr */
LOADADDR( r5 ,c o p y _ t o _ h e r e ) / * # b y t e s o f m e m o r y t o c o p y * /
sub r5 ,r5 ,r27
li r6 ,0 x10 0 / * S t a r t o f f s e t , t h e f i r s t 0 x10 0 * /
/* bytes were copied earlier. */
bl . c o p y _ a n d _ f l u s h / * c o p y t h e f i r s t n b y t e s * /
/* this includes the code being */
/* executed here. */
LOADADDR( r0 , 4 f ) / * J u m p t o t h e c o p y o f t h i s c o d e * /
mtctr r0 / * t h a t w e j u s t m a d e / r e l o c a t e d * /
bctr
4 : LOADADDR( r5 ,k l i m i t )
2005-10-10 16:41:25 +04:00
add r5 ,r5 ,r26
2005-09-26 10:04:21 +04:00
ld r5 ,0 ( r5 ) / * g e t t h e v a l u e o f k l i m i t * /
sub r5 ,r5 ,r27
bl . c o p y _ a n d _ f l u s h / * c o p y t h e r e s t * /
b . s t a r t _ h e r e _ m u l t i p l a t f o r m
# endif / * C O N F I G _ P P C _ M U L T I P L A T F O R M * /
/ *
* Copy r o u t i n e u s e d t o c o p y t h e k e r n e l t o s t a r t a t p h y s i c a l a d d r e s s 0
* and f l u s h a n d i n v a l i d a t e t h e c a c h e s a s n e e d e d .
* r3 = d e s t a d d r , r4 = s o u r c e a d d r , r5 = c o p y l i m i t , r6 = s t a r t o f f s e t
* on e x i t , r3 , r4 , r5 a r e u n c h a n g e d , r6 i s u p d a t e d t o b e > = r5 .
*
* Note : this r o u t i n e * o n l y * c l o b b e r s r0 , r6 a n d l r
* /
_ GLOBAL( c o p y _ a n d _ f l u s h )
addi r5 ,r5 ,- 8
addi r6 ,r6 ,- 8
4 : li r0 ,1 6 / * U s e t h e l e a s t c o m m o n * /
/* denominator cache line */
/* size. This results in */
/* extra cache line flushes */
/* but operation is correct. */
/* Can't get cache line size */
/* from NACA as it is being */
/* moved too. */
mtctr r0 / * p u t # w o r d s / l i n e i n c t r * /
3 : addi r6 ,r6 ,8 / * c o p y a c a c h e l i n e * /
ldx r0 ,r6 ,r4
stdx r0 ,r6 ,r3
bdnz 3 b
dcbst r6 ,r3 / * w r i t e i t t o m e m o r y * /
sync
icbi r6 ,r3 / * f l u s h t h e i c a c h e l i n e * /
cmpld 0 ,r6 ,r5
blt 4 b
sync
addi r5 ,r5 ,8
addi r6 ,r6 ,8
blr
.align 8
copy_to_here :
# ifdef C O N F I G _ S M P
# ifdef C O N F I G _ P P C _ P M A C
/ *
* On P o w e r M a c , s e c o n d a r y p r o c e s s o r s s t a r t s f r o m t h e r e s e t v e c t o r , w h i c h
* is t e m p o r a r i l y t u r n e d i n t o a c a l l t o o n e o f t h e f u n c t i o n s b e l o w .
* /
.section " .text " ;
.align 2 ;
2005-10-22 10:02:39 +04:00
.globl __secondary_start_pmac_0
__secondary_start_pmac_0 :
/* NB the entries for cpus 0, 1, 2 must each occupy 8 bytes. */
li r24 ,0
b 1 f
li r24 ,1
b 1 f
li r24 ,2
b 1 f
li r24 ,3
1 :
2005-09-26 10:04:21 +04:00
_ GLOBAL( p m a c _ s e c o n d a r y _ s t a r t )
/* turn on 64-bit mode */
bl . e n a b l e _ 6 4 b _ m o d e
isync
/* Copy some CPU settings from CPU 0 */
bl . _ _ r e s t o r e _ c p u _ s e t u p
/* pSeries do that early though I don't think we really need it */
mfmsr r3
ori r3 ,r3 ,M S R _ R I
mtmsrd r3 / * R I o n * /
/* Set up a paca value for this processor. */
LOADADDR( r4 , p a c a ) / * G e t b a s e v a d d r o f p a c a a r r a y * /
mulli r13 ,r24 ,P A C A _ S I Z E / * C a l c u l a t e v a d d r o f r i g h t p a c a * /
add r13 ,r13 ,r4 / * f o r t h i s p r o c e s s o r . * /
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 3 ,r13 / * S a v e v a d d r o f p a c a i n S P R G 3 * /
2005-09-26 10:04:21 +04:00
/* Create a temp kernel stack for use before relocation is on. */
ld r1 ,P A C A E M E R G S P ( r13 )
subi r1 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
b . _ _ s e c o n d a r y _ s t a r t
# endif / * C O N F I G _ P P C _ P M A C * /
/ *
* This f u n c t i o n i s c a l l e d a f t e r t h e m a s t e r C P U h a s r e l e a s e d t h e
* secondary p r o c e s s o r s . T h e e x e c u t i o n e n v i r o n m e n t i s r e l o c a t i o n o f f .
* The p a c a f o r t h i s p r o c e s s o r h a s t h e f o l l o w i n g f i e l d s i n i t i a l i z e d a t
* this p o i n t :
* 1 . Processor n u m b e r
* 2 . Segment t a b l e p o i n t e r ( v i r t u a l a d d r e s s )
* On e n t r y t h e f o l l o w i n g a r e s e t :
* r1 = s t a c k p o i n t e r . v a d d r f o r i S e r i e s , r a d d r ( t e m p s t a c k ) f o r p S e r i e s
* r2 4 = c p u # ( i n L i n u x t e r m s )
* r1 3 = p a c a v i r t u a l a d d r e s s
* SPRG3 = p a c a v i r t u a l a d d r e s s
* /
_ GLOBAL( _ _ s e c o n d a r y _ s t a r t )
2005-11-10 05:37:51 +03:00
/* Set thread priority to MEDIUM */
HMT_ M E D I U M
2005-09-26 10:04:21 +04:00
2005-11-10 05:37:51 +03:00
/* Load TOC */
2005-09-26 10:04:21 +04:00
ld r2 ,P A C A T O C ( r13 )
2005-11-10 05:37:51 +03:00
/* Do early setup for that CPU (stab, slb, hash table pointer) */
bl . e a r l y _ s e t u p _ s e c o n d a r y
2005-09-26 10:04:21 +04:00
/* Initialize the kernel stack. Just a repeat for iSeries. */
LOADADDR( r3 ,c u r r e n t _ s e t )
sldi r28 ,r24 ,3 / * g e t c u r r e n t _ s e t [ c p u #] * /
ldx r1 ,r3 ,r28
addi r1 ,r1 ,T H R E A D _ S I Z E - S T A C K _ F R A M E _ O V E R H E A D
std r1 ,P A C A K S A V E ( r13 )
2005-11-10 05:37:51 +03:00
/* Clear backchain so we get nice backtraces */
2005-09-26 10:04:21 +04:00
li r7 ,0
mtlr r7
/* enable MMU and jump to start_secondary */
LOADADDR( r3 ,. s t a r t _ s e c o n d a r y _ p r o l o g )
SET_ R E G _ T O _ C O N S T ( r4 , M S R _ K E R N E L )
# ifdef D O _ S O F T _ D I S A B L E
ori r4 ,r4 ,M S R _ E E
# endif
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 0 ,r3
mtspr S P R N _ S R R 1 ,r4
2005-09-26 10:04:21 +04:00
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
/ *
* Running w i t h r e l o c a t i o n o n a t t h i s p o i n t . A l l w e w a n t t o d o i s
* zero t h e s t a c k b a c k - c h a i n p o i n t e r b e f o r e g o i n g i n t o C c o d e .
* /
_ GLOBAL( s t a r t _ s e c o n d a r y _ p r o l o g )
li r3 ,0
std r3 ,0 ( r1 ) / * Z e r o t h e s t a c k f r a m e p o i n t e r * /
bl . s t a r t _ s e c o n d a r y
2005-11-10 05:37:51 +03:00
b .
2005-09-26 10:04:21 +04:00
# endif
/ *
* This s u b r o u t i n e c l o b b e r s r11 a n d r12
* /
_ GLOBAL( e n a b l e _ 6 4 b _ m o d e )
mfmsr r11 / * g r a b t h e c u r r e n t M S R * /
li r12 ,1
rldicr r12 ,r12 ,M S R _ S F _ L G ,( 6 3 - M S R _ S F _ L G )
or r11 ,r11 ,r12
li r12 ,1
rldicr r12 ,r12 ,M S R _ I S F _ L G ,( 6 3 - M S R _ I S F _ L G )
or r11 ,r11 ,r12
mtmsrd r11
isync
blr
# ifdef C O N F I G _ P P C _ M U L T I P L A T F O R M
/ *
* This i s w h e r e t h e m a i n k e r n e l c o d e s t a r t s .
* /
_ STATIC( s t a r t _ h e r e _ m u l t i p l a t f o r m )
/* get a new offset, now that the kernel has moved. */
bl . r e l o c _ o f f s e t
mr r26 ,r3
/ * Clear o u t t h e B S S . I t m a y h a v e b e e n d o n e i n p r o m _ i n i t ,
* already b u t t h a t ' s i r r e l e v a n t s i n c e p r o m _ i n i t w i l l s o o n
* be d e t a c h e d f r o m t h e k e r n e l c o m p l e t e l y . B e s i d e s , w e n e e d
* to c l e a r i t n o w f o r k e x e c - s t y l e e n t r y .
* /
LOADADDR( r11 ,_ _ b s s _ s t o p )
LOADADDR( r8 ,_ _ b s s _ s t a r t )
sub r11 ,r11 ,r8 / * b s s s i z e * /
addi r11 ,r11 ,7 / * r o u n d u p t o a n e v e n d o u b l e w o r d * /
rldicl. r11 ,r11 ,6 1 ,3 / * s h i f t r i g h t b y 3 * /
beq 4 f
addi r8 ,r8 ,- 8
li r0 ,0
mtctr r11 / * z e r o t h i s m a n y d o u b l e w o r d s * /
3 : stdu r0 ,8 ( r8 )
bdnz 3 b
4 :
mfmsr r6
ori r6 ,r6 ,M S R _ R I
mtmsrd r6 / * R I o n * /
# ifdef C O N F I G _ H M T
/* Start up the second thread on cpu 0 */
2005-10-10 08:01:07 +04:00
mfspr r3 ,S P R N _ P V R
2005-09-26 10:04:21 +04:00
srwi r3 ,r3 ,1 6
cmpwi r3 ,0 x34 / * P u l s a r * /
beq 9 0 f
cmpwi r3 ,0 x36 / * I c e s t a r * /
beq 9 0 f
cmpwi r3 ,0 x37 / * S S t a r * /
beq 9 0 f
b 9 1 f / * H M T n o t s u p p o r t e d * /
90 : li r3 ,0
bl . h m t _ s t a r t _ s e c o n d a r y
91 :
# endif
/* The following gets the stack and TOC set up with the regs */
/* pointing to the real addr of the kernel stack. This is */
/* all done to support the C function call below which sets */
/* up the htab. This is done because we have relocated the */
/* kernel but are still running in real mode. */
LOADADDR( r3 ,i n i t _ t h r e a d _ u n i o n )
2005-10-10 16:41:25 +04:00
add r3 ,r3 ,r26
2005-09-26 10:04:21 +04:00
/* set up a stack pointer (physical address) */
addi r1 ,r3 ,T H R E A D _ S I Z E
li r0 ,0
stdu r0 ,- S T A C K _ F R A M E _ O V E R H E A D ( r1 )
/* set up the TOC (physical address) */
LOADADDR( r2 ,_ _ t o c _ s t a r t )
addi r2 ,r2 ,0 x40 0 0
addi r2 ,r2 ,0 x40 0 0
2005-10-10 16:41:25 +04:00
add r2 ,r2 ,r26
2005-09-26 10:04:21 +04:00
LOADADDR( r3 ,c p u _ s p e c s )
2005-10-10 16:41:25 +04:00
add r3 ,r3 ,r26
2005-09-26 10:04:21 +04:00
LOADADDR( r4 ,c u r _ c p u _ s p e c )
2005-10-10 16:41:25 +04:00
add r4 ,r4 ,r26
2005-09-26 10:04:21 +04:00
mr r5 ,r26
bl . i d e n t i f y _ c p u
/ * Save s o m e l o w l e v e l c o n f i g H I D s o f C P U 0 t o b e c o p i e d t o
* other C P U s l a t e r o n , o r u s e d f o r s u s p e n d / r e s u m e
* /
bl . _ _ s a v e _ c p u _ s e t u p
sync
/ * Setup a v a l i d p h y s i c a l P A C A p o i n t e r i n S P R G 3 f o r e a r l y _ s e t u p
* note t h a t b o o t _ c p u i d c a n a l w a y s b e 0 n o w a d a y s s i n c e t h e r e i s
* nowhere i t c a n b e i n i t i a l i z e d d i f f e r e n t l y b e f o r e w e r e a c h t h i s
* code
* /
LOADADDR( r27 , b o o t _ c p u i d )
2005-10-10 16:41:25 +04:00
add r27 ,r27 ,r26
2005-09-26 10:04:21 +04:00
lwz r27 ,0 ( r27 )
LOADADDR( r24 , p a c a ) / * G e t b a s e v a d d r o f p a c a a r r a y * /
mulli r13 ,r27 ,P A C A _ S I Z E / * C a l c u l a t e v a d d r o f r i g h t p a c a * /
add r13 ,r13 ,r24 / * f o r t h i s p r o c e s s o r . * /
2005-10-10 16:41:25 +04:00
add r13 ,r13 ,r26 / * c o n v e r t t o p h y s i c a l a d d r * /
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 3 ,r13 / * P P P B B B : T e m p . . . - P e t e r * /
2005-09-26 10:04:21 +04:00
/ * Do v e r y e a r l y k e r n e l i n i t i a l i z a t i o n s , i n c l u d i n g i n i t i a l h a s h t a b l e ,
* stab a n d s l b s e t u p b e f o r e w e t u r n o n r e l o c a t i o n . * /
/* Restore parameters passed from prom_init/kexec */
mr r3 ,r31
bl . e a r l y _ s e t u p
LOADADDR( r3 ,. s t a r t _ h e r e _ c o m m o n )
SET_ R E G _ T O _ C O N S T ( r4 , M S R _ K E R N E L )
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 0 ,r3
mtspr S P R N _ S R R 1 ,r4
2005-09-26 10:04:21 +04:00
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
# endif / * C O N F I G _ P P C _ M U L T I P L A T F O R M * /
/* This is where all platforms converge execution */
_ STATIC( s t a r t _ h e r e _ c o m m o n )
/* relocation is on at this point */
/* The following code sets up the SP and TOC now that we are */
/* running with translation enabled. */
LOADADDR( r3 ,i n i t _ t h r e a d _ u n i o n )
/* set up the stack */
addi r1 ,r3 ,T H R E A D _ S I Z E
li r0 ,0
stdu r0 ,- S T A C K _ F R A M E _ O V E R H E A D ( r1 )
/ * Apply t h e C P U s - s p e c i f i c f i x u p s ( n o p o u t s e c t i o n s n o t r e l e v a n t
* to t h i s C P U
* /
li r3 ,0
bl . d o _ c p u _ f t r _ f i x u p s
LOADADDR( r26 , b o o t _ c p u i d )
lwz r26 ,0 ( r26 )
LOADADDR( r24 , p a c a ) / * G e t b a s e v a d d r o f p a c a a r r a y * /
mulli r13 ,r26 ,P A C A _ S I Z E / * C a l c u l a t e v a d d r o f r i g h t p a c a * /
add r13 ,r13 ,r24 / * f o r t h i s p r o c e s s o r . * /
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 3 ,r13
2005-09-26 10:04:21 +04:00
/* ptr to current */
LOADADDR( r4 ,i n i t _ t a s k )
std r4 ,P A C A C U R R E N T ( r13 )
/* Load the TOC */
ld r2 ,P A C A T O C ( r13 )
std r1 ,P A C A K S A V E ( r13 )
bl . s e t u p _ s y s t e m
/* Load up the kernel context */
5 :
# ifdef D O _ S O F T _ D I S A B L E
li r5 ,0
stb r5 ,P A C A P R O C E N A B L E D ( r13 ) / * S o f t D i s a b l e d * /
mfmsr r5
ori r5 ,r5 ,M S R _ E E / * H a r d E n a b l e d * /
mtmsrd r5
# endif
bl . s t a r t _ k e r n e l
_ GLOBAL( h m t _ i n i t )
# ifdef C O N F I G _ H M T
LOADADDR( r5 , h m t _ t h r e a d _ d a t a )
2005-10-10 08:01:07 +04:00
mfspr r7 ,S P R N _ P V R
2005-09-26 10:04:21 +04:00
srwi r7 ,r7 ,1 6
cmpwi r7 ,0 x34 / * P u l s a r * /
beq 9 0 f
cmpwi r7 ,0 x36 / * I c e s t a r * /
beq 9 1 f
cmpwi r7 ,0 x37 / * S S t a r * /
beq 9 1 f
b 1 0 1 f
2005-10-10 08:01:07 +04:00
90 : mfspr r6 ,S P R N _ P I R
2005-09-26 10:04:21 +04:00
andi. r6 ,r6 ,0 x1 f
b 9 2 f
2005-10-10 08:01:07 +04:00
91 : mfspr r6 ,S P R N _ P I R
2005-09-26 10:04:21 +04:00
andi. r6 ,r6 ,0 x3 f f
92 : sldi r4 ,r24 ,3
stwx r6 ,r5 ,r4
bl . h m t _ s t a r t _ s e c o n d a r y
b 1 0 1 f
__hmt_secondary_hold :
LOADADDR( r5 , h m t _ t h r e a d _ d a t a )
clrldi r5 ,r5 ,4
li r7 ,0
2005-10-10 08:01:07 +04:00
mfspr r6 ,S P R N _ P I R
mfspr r8 ,S P R N _ P V R
2005-09-26 10:04:21 +04:00
srwi r8 ,r8 ,1 6
cmpwi r8 ,0 x34
bne 9 3 f
andi. r6 ,r6 ,0 x1 f
b 1 0 3 f
93 : andi. r6 ,r6 ,0 x3 f
103 : lwzx r8 ,r5 ,r7
cmpw r8 ,r6
beq 1 0 4 f
addi r7 ,r7 ,8
b 1 0 3 b
104 : addi r7 ,r7 ,4
lwzx r9 ,r5 ,r7
mr r24 ,r9
101 :
# endif
mr r3 ,r24
b . p S e r i e s _ s e c o n d a r y _ s m p _ i n i t
# ifdef C O N F I G _ H M T
_ GLOBAL( h m t _ s t a r t _ s e c o n d a r y )
LOADADDR( r4 ,_ _ h m t _ s e c o n d a r y _ h o l d )
clrldi r4 ,r4 ,4
2005-10-10 08:01:07 +04:00
mtspr S P R N _ N I A D O R M , r4
mfspr r4 , S P R N _ M S R D O R M
2005-09-26 10:04:21 +04:00
li r5 , - 6 5
and r4 , r4 , r5
2005-10-10 08:01:07 +04:00
mtspr S P R N _ M S R D O R M , r4
2005-09-26 10:04:21 +04:00
lis r4 ,0 x f f e f
ori r4 ,r4 ,0 x74 0 3
2005-10-10 08:01:07 +04:00
mtspr S P R N _ T S C , r4
2005-09-26 10:04:21 +04:00
li r4 ,0 x1 f4
2005-10-10 08:01:07 +04:00
mtspr S P R N _ T S T , r4
mfspr r4 , S P R N _ H I D 0
2005-09-26 10:04:21 +04:00
ori r4 , r4 , 0 x1
2005-10-10 08:01:07 +04:00
mtspr S P R N _ H I D 0 , r4
2005-09-26 10:04:21 +04:00
mfspr r4 , S P R N _ C T R L F
oris r4 , r4 , 0 x40
mtspr S P R N _ C T R L T , r4
blr
# endif
/ *
* We p u t a f e w t h i n g s h e r e t h a t h a v e t o b e p a g e - a l i g n e d .
* This s t u f f g o e s a t t h e b e g i n n i n g o f t h e b s s , w h i c h i s p a g e - a l i g n e d .
* /
.section " .bss "
.align PAGE_SHIFT
.globl empty_zero_page
empty_zero_page :
.space PAGE_SIZE
.globl swapper_pg_dir
swapper_pg_dir :
.space PAGE_SIZE
/ *
* This s p a c e g e t s a c o p y o f o p t i o n a l i n f o p a s s e d t o u s b y t h e b o o t s t r a p
* Used t o p a s s p a r a m e t e r s i n t o t h e k e r n e l l i k e r o o t = / d e v / s d a1 , e t c .
* /
.globl cmd_line
cmd_line :
.space COMMAND_LINE_SIZE
