2005-09-26 10:04:21 +04:00
/ *
* PowerPC v e r s i o n
* Copyright ( C ) 1 9 9 5 - 1 9 9 6 G a r y T h o m a s ( g d t @linuxppc.org)
*
* Rewritten b y C o r t D o u g a n ( c o r t @cs.nmt.edu) for PReP
* Copyright ( C ) 1 9 9 6 C o r t D o u g a n < c o r t @cs.nmt.edu>
* Adapted f o r P o w e r M a c i n t o s h b y P a u l M a c k e r r a s .
* Low- l e v e l e x c e p t i o n h a n d l e r s a n d M M U s u p p o r t
* rewritten b y P a u l M a c k e r r a s .
* Copyright ( C ) 1 9 9 6 P a u l M a c k e r r a s .
*
* Adapted f o r 6 4 b i t P o w e r P C b y D a v e E n g e b r e t s e n , P e t e r B e r g n e r , a n d
* Mike C o r r i g a n { e n g e b r e t | b e r g n e r | m i k e j c } @us.ibm.com
*
* This f i l e c o n t a i n s t h e l o w - l e v e l s u p p o r t a n d s e t u p f o r t h e
* PowerPC- 6 4 p l a t f o r m , i n c l u d i n g t r a p a n d i n t e r r u p t d i s p a t c h .
*
* This p r o g r a m i s f r e e s o f t w a r e ; you can redistribute it and/or
* modify i t u n d e r t h e t e r m s o f t h e G N U G e n e r a l P u b l i c L i c e n s e
* as p u b l i s h e d b y t h e F r e e S o f t w a r e F o u n d a t i o n ; either version
* 2 of t h e L i c e n s e , o r ( a t y o u r o p t i o n ) a n y l a t e r v e r s i o n .
* /
# include < l i n u x / c o n f i g . h >
# include < l i n u x / t h r e a d s . h >
2005-10-10 08:01:07 +04:00
# include < a s m / r e g . h >
2005-09-26 10:04:21 +04:00
# include < a s m / p a g e . h >
# include < a s m / m m u . h >
# include < a s m / p p c _ a s m . h >
# include < a s m / a s m - o f f s e t s . h >
# include < a s m / b u g . h >
# include < a s m / c p u t a b l e . h >
# include < a s m / s e t u p . h >
# include < a s m / h v c a l l . h >
2005-11-02 07:02:47 +03:00
# include < a s m / i s e r i e s / l p a r _ m a p . h >
[PATCH] powerpc: Merge thread_info.h
Merge ppc32 and ppc64 versions of thread_info.h. They were pretty
similar already, the chief changes are:
- Instead of inline asm to implement current_thread_info(),
which needs to be different for ppc32 and ppc64, we use C with an
asm("r1") register variable. gcc turns it into the same asm as we
used to have for both platforms.
- We replace ppc32's 'local_flags' with the ppc64
'syscall_noerror' field. The noerror flag was in fact the only thing
in the local_flags field anyway, so the ppc64 approach is simpler, and
means we only need a load-immediate/store instead of load/mask/store
when clearing the flag.
- In readiness for 64k pages, when THREAD_SIZE will be less
than a page, ppc64 used kmalloc() rather than get_free_pages() to
allocate the kernel stack. With this patch we do the same for ppc32,
since there's no strong reason not to.
- For ppc64, we no longer export THREAD_SHIFT and THREAD_SIZE
via asm-offsets, thread_info.h can now be safely included in asm, as
on ppc32.
Built and booted on G4 Powerbook (ARCH=ppc and ARCH=powerpc) and
Power5 (ARCH=ppc64 and ARCH=powerpc).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-10-21 09:45:50 +04:00
# include < a s m / t h r e a d _ i n f o . h >
2005-09-26 10:04:21 +04:00
# ifdef C O N F I G _ P P C _ I S E R I E S
# define D O _ S O F T _ D I S A B L E
# endif
/ *
* We l a y o u t p h y s i c a l m e m o r y a s f o l l o w s :
* 0 x0 0 0 0 - 0 x00 f f : S e c o n d a r y p r o c e s s o r s p i n c o d e
* 0 x0 1 0 0 - 0 x2 f f f : p S e r i e s I n t e r r u p t p r o l o g s
* 0 x3 0 0 0 - 0 x5 f f f : i n t e r r u p t s u p p o r t , i S e r i e s a n d c o m m o n i n t e r r u p t p r o l o g s
* 0 x6 0 0 0 - 0 x6 f f f : I n i t i a l ( C P U 0 ) s e g m e n t t a b l e
* 0 x7 0 0 0 - 0 x7 f f f : F W N M I d a t a a r e a
* 0 x8 0 0 0 - : E a r l y i n i t a n d s u p p o r t c o d e
* /
/ *
* SPRG U s a g e
*
* Register D e f i n i t i o n
*
* SPRG0 r e s e r v e d f o r h y p e r v i s o r
* SPRG1 t e m p - u s e d t o s a v e g p r
* SPRG2 t e m p - u s e d t o s a v e g p r
* SPRG3 v i r t a d d r o f p a c a
* /
/ *
* Entering i n t o t h i s c o d e w e m a k e t h e f o l l o w i n g a s s u m p t i o n s :
* For p S e r i e s :
* 1 . The M M U i s o f f & o p e n f i r m w a r e i s r u n n i n g i n r e a l m o d e .
* 2 . The k e r n e l i s e n t e r e d a t _ _ s t a r t
*
* For i S e r i e s :
* 1 . The M M U i s o n ( a s i t a l w a y s i s f o r i S e r i e s )
* 2 . The k e r n e l i s e n t e r e d a t s y s t e m _ r e s e t _ i S e r i e s
* /
.text
.globl _stext
_stext :
# ifdef C O N F I G _ P P C _ M U L T I P L A T F O R M
_ GLOBAL( _ _ s t a r t )
/* NOP this out unconditionally */
BEGIN_ F T R _ S E C T I O N
2005-10-06 04:59:19 +04:00
b . _ _ s t a r t _ i n i t i a l i z a t i o n _ m u l t i p l a t f o r m
2005-09-26 10:04:21 +04:00
END_ F T R _ S E C T I O N ( 0 , 1 )
# endif / * C O N F I G _ P P C _ M U L T I P L A T F O R M * /
/* Catch branch to 0 in real mode */
trap
# ifdef C O N F I G _ P P C _ I S E R I E S
/ *
* At o f f s e t 0 x20 , t h e r e i s a p o i n t e r t o i S e r i e s L P A R d a t a .
* This i s r e q u i r e d b y t h e h y p e r v i s o r
* /
. = 0 x2 0
.llong - K E R N E L B A S E
/ *
* At o f f s e t 0 x28 a n d 0 x30 a r e o f f s e t s t o t h e m s c h u n k s _ m a p
* array ( u s e d b y t h e i S e r i e s L P A R d e b u g g e r t o d o t r a n s l a t i o n
* between p h y s i c a l a d d r e s s e s a n d a b s o l u t e a d d r e s s e s ) a n d
* to t h e p i d h a s h t a b l e ( a l s o u s e d b y t h e d e b u g g e r )
* /
.llong _ m a p - K E R N E L B A S E
.llong 0 /* pidhash-KERNELBASE SFRXXX */
/* Offset 0x38 - Pointer to start of embedded System.map */
.globl embedded_sysmap_start
embedded_sysmap_start :
.llong 0
/* Offset 0x40 - Pointer to end of embedded System.map */
.globl embedded_sysmap_end
embedded_sysmap_end :
.llong 0
# endif / * C O N F I G _ P P C _ I S E R I E S * /
/* Secondary processors spin on this value until it goes to 1. */
.globl __secondary_hold_spinloop
__secondary_hold_spinloop :
.llong 0x0
/* Secondary processors write this value with their cpu # */
/* after they enter the spin loop immediately below. */
.globl __secondary_hold_acknowledge
__secondary_hold_acknowledge :
.llong 0x0
. = 0 x6 0
/ *
* The f o l l o w i n g c o d e i s u s e d o n p S e r i e s t o h o l d s e c o n d a r y p r o c e s s o r s
* in a s p i n l o o p a f t e r t h e y h a v e b e e n f r e e d f r o m O p e n F i r m w a r e , b u t
* before t h e b u l k o f t h e k e r n e l h a s b e e n r e l o c a t e d . T h i s c o d e
* is r e l o c a t e d t o p h y s i c a l a d d r e s s 0 x60 b e f o r e p r o m _ i n i t i s r u n .
* All o f i t m u s t f i t b e l o w t h e f i r s t e x c e p t i o n v e c t o r a t 0 x10 0 .
* /
_ GLOBAL( _ _ s e c o n d a r y _ h o l d )
mfmsr r24
ori r24 ,r24 ,M S R _ R I
mtmsrd r24 / * R I o n * /
2006-02-13 10:11:13 +03:00
/* Grab our physical cpu number */
2005-09-26 10:04:21 +04:00
mr r24 ,r3
/* Tell the master cpu we're here */
/* Relocation is off & we are located at an address less */
/* than 0x100, so only need to grab low order offset. */
std r24 ,_ _ s e c o n d a r y _ h o l d _ a c k n o w l e d g e @l(0)
sync
/* All secondary cpus wait here until told to start. */
100 : ld r4 ,_ _ s e c o n d a r y _ h o l d _ s p i n l o o p @l(0)
cmpdi 0 ,r4 ,1
bne 1 0 0 b
2006-02-13 10:11:13 +03:00
# if d e f i n e d ( C O N F I G _ S M P ) | | d e f i n e d ( C O N F I G _ K E X E C )
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r4 , . p S e r i e s _ s e c o n d a r y _ s m p _ i n i t )
2005-12-06 00:49:00 +03:00
mtctr r4
2005-09-26 10:04:21 +04:00
mr r3 ,r24
2005-12-06 00:49:00 +03:00
bctr
2005-09-26 10:04:21 +04:00
# else
BUG_ O P C O D E
# endif
/* This value is used to mark exception frames on the stack. */
.section " .toc " , " aw"
exception_marker :
.tc _ 7 2 6 5 6 7 7 3 _ 6 8 6 5 7 2 6 5 [ T C ] ,0 x72 6 5 6 7 7 3 6 8 6 5 7 2 6 5
.text
/ *
* The f o l l o w i n g m a c r o s d e f i n e t h e c o d e t h a t a p p e a r s a s
* the p r o l o g u e t o e a c h o f t h e e x c e p t i o n h a n d l e r s . T h e y
* are s p l i t i n t o t w o p a r t s t o a l l o w a s i n g l e k e r n e l b i n a r y
* to b e u s e d f o r p S e r i e s a n d i S e r i e s .
* LOL. O n e d a y . . . - p a u l u s
* /
/ *
* We m a k e a s m u c h o f t h e e x c e p t i o n c o d e c o m m o n b e t w e e n n a t i v e
* exception h a n d l e r s ( i n c l u d i n g p S e r i e s L P A R ) a n d i S e r i e s L P A R
* implementations a s p o s s i b l e .
* /
/ *
* This i s t h e s t a r t o f t h e i n t e r r u p t h a n d l e r s f o r p S e r i e s
* This c o d e r u n s w i t h r e l o c a t i o n o f f .
* /
# define E X _ R 9 0
# define E X _ R 1 0 8
# define E X _ R 1 1 1 6
# define E X _ R 1 2 2 4
# define E X _ R 1 3 3 2
# define E X _ S R R 0 4 0
# define E X _ D A R 4 8
# define E X _ D S I S R 5 6
# define E X _ C C R 6 0
2005-11-07 03:06:55 +03:00
# define E X _ R 3 6 4
# define E X _ L R 7 2
2005-09-26 10:04:21 +04:00
2005-12-06 00:49:00 +03:00
/ *
2006-01-13 06:56:25 +03:00
* We' r e s h o r t o n s p a c e a n d t i m e i n t h e e x c e p t i o n p r o l o g , s o w e c a n ' t
* use t h e n o r m a l S E T _ R E G _ I M M E D I A T E m a c r o . N o r m a l l y w e j u s t n e e d t h e
* low h a l f w o r d o f t h e a d d r e s s , b u t f o r K d u m p w e n e e d t h e w h o l e l o w
* word.
2005-12-06 00:49:00 +03:00
* /
# ifdef C O N F I G _ C R A S H _ D U M P
# define L O A D _ H A N D L E R ( r e g , l a b e l ) \
oris r e g ,r e g ,( l a b e l ) @h; /* virt addr of handler ... */ \
ori r e g ,r e g ,( l a b e l ) @l; /* .. and the rest */
# else
# define L O A D _ H A N D L E R ( r e g , l a b e l ) \
ori r e g ,r e g ,( l a b e l ) @l; /* virt addr of handler ... */
# endif
2005-09-26 10:04:21 +04:00
# define E X C E P T I O N _ P R O L O G _ P S E R I E S ( a r e a , l a b e l ) \
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3 ; /* get paca address into r13 */ \
2005-09-26 10:04:21 +04:00
std r9 ,a r e a + E X _ R 9 ( r13 ) ; /* save r9 - r12 */ \
std r10 ,a r e a + E X _ R 1 0 ( r13 ) ; \
std r11 ,a r e a + E X _ R 1 1 ( r13 ) ; \
std r12 ,a r e a + E X _ R 1 2 ( r13 ) ; \
2005-10-10 08:01:07 +04:00
mfspr r9 ,S P R N _ S P R G 1 ; \
2005-09-26 10:04:21 +04:00
std r9 ,a r e a + E X _ R 1 3 ( r13 ) ; \
mfcr r9 ; \
clrrdi r12 ,r13 ,3 2 ; /* get high part of &label */ \
mfmsr r10 ; \
2005-10-10 08:01:07 +04:00
mfspr r11 ,S P R N _ S R R 0 ; /* save SRR0 */ \
2005-12-06 00:49:00 +03:00
LOAD_ H A N D L E R ( r12 ,l a b e l ) \
2005-09-26 10:04:21 +04:00
ori r10 ,r10 ,M S R _ I R | M S R _ D R | M S R _ R I ; \
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 0 ,r12 ; \
mfspr r12 ,S P R N _ S R R 1 ; /* and SRR1 */ \
mtspr S P R N _ S R R 1 ,r10 ; \
2005-09-26 10:04:21 +04:00
rfid; \
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
/ *
* This i s t h e s t a r t o f t h e i n t e r r u p t h a n d l e r s f o r i S e r i e s
* This c o d e r u n s w i t h r e l o c a t i o n o n .
* /
# define E X C E P T I O N _ P R O L O G _ I S E R I E S _ 1 ( a r e a ) \
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3 ; /* get paca address into r13 */ \
2005-09-26 10:04:21 +04:00
std r9 ,a r e a + E X _ R 9 ( r13 ) ; /* save r9 - r12 */ \
std r10 ,a r e a + E X _ R 1 0 ( r13 ) ; \
std r11 ,a r e a + E X _ R 1 1 ( r13 ) ; \
std r12 ,a r e a + E X _ R 1 2 ( r13 ) ; \
2005-10-10 08:01:07 +04:00
mfspr r9 ,S P R N _ S P R G 1 ; \
2005-09-26 10:04:21 +04:00
std r9 ,a r e a + E X _ R 1 3 ( r13 ) ; \
mfcr r9
# define E X C E P T I O N _ P R O L O G _ I S E R I E S _ 2 \
mfmsr r10 ; \
2006-01-13 02:26:42 +03:00
ld r12 ,P A C A L P P A C A P T R ( r13 ) ; \
ld r11 ,L P P A C A S R R 0 ( r12 ) ; \
ld r12 ,L P P A C A S R R 1 ( r12 ) ; \
2005-09-26 10:04:21 +04:00
ori r10 ,r10 ,M S R _ R I ; \
mtmsrd r10 ,1
/ *
* The c o m m o n e x c e p t i o n p r o l o g i s u s e d f o r a l l e x c e p t a f e w e x c e p t i o n s
* such a s a s e g m e n t m i s s o n a k e r n e l a d d r e s s . W e h a v e t o b e p r e p a r e d
* to t a k e a n o t h e r e x c e p t i o n f r o m t h e p o i n t w h e r e w e f i r s t t o u c h t h e
* kernel s t a c k o n w a r d s .
*
* On e n t r y r13 p o i n t s t o t h e p a c a , r9 - r13 a r e s a v e d i n t h e p a c a ,
* r9 c o n t a i n s t h e s a v e d C R , r11 a n d r12 c o n t a i n t h e s a v e d S R R 0 a n d
* SRR1 , a n d r e l o c a t i o n i s o n .
* /
# define E X C E P T I O N _ P R O L O G _ C O M M O N ( n , a r e a ) \
andi. r10 ,r12 ,M S R _ P R ; /* See if coming from user */ \
mr r10 ,r1 ; /* Save r1 */ \
subi r1 ,r1 ,I N T _ F R A M E _ S I Z E ; /* alloc frame on kernel stack */ \
beq- 1 f ; \
ld r1 ,P A C A K S A V E ( r13 ) ; /* kernel stack to use */ \
1 : cmpdi c r1 ,r1 ,0 ; /* check if r1 is in userspace */ \
bge- c r1 ,b a d _ s t a c k ; /* abort if it is */ \
std r9 ,_ C C R ( r1 ) ; /* save CR in stackframe */ \
std r11 ,_ N I P ( r1 ) ; /* save SRR0 in stackframe */ \
std r12 ,_ M S R ( r1 ) ; /* save SRR1 in stackframe */ \
std r10 ,0 ( r1 ) ; /* make stack chain pointer */ \
std r0 ,G P R 0 ( r1 ) ; /* save r0 in stackframe */ \
std r10 ,G P R 1 ( r1 ) ; /* save r1 in stackframe */ \
powerpc: Implement accurate task and CPU time accounting
This implements accurate task and cpu time accounting for 64-bit
powerpc kernels. Instead of accounting a whole jiffy of time to a
task on a timer interrupt because that task happened to be running at
the time, we now account time in units of timebase ticks according to
the actual time spent by the task in user mode and kernel mode. We
also count the time spent processing hardware and software interrupts
accurately. This is conditional on CONFIG_VIRT_CPU_ACCOUNTING. If
that is not set, we do tick-based approximate accounting as before.
To get this accurate information, we read either the PURR (processor
utilization of resources register) on POWER5 machines, or the timebase
on other machines on
* each entry to the kernel from usermode
* each exit to usermode
* transitions between process context, hard irq context and soft irq
context in kernel mode
* context switches.
On POWER5 systems with shared-processor logical partitioning we also
read both the PURR and the timebase at each timer interrupt and
context switch in order to determine how much time has been taken by
the hypervisor to run other partitions ("steal" time). Unfortunately,
since we need values of the PURR on both threads at the same time to
accurately calculate the steal time, and since we can only calculate
steal time on a per-core basis, the apportioning of the steal time
between idle time (time which we ceded to the hypervisor in the idle
loop) and actual stolen time is somewhat approximate at the moment.
This is all based quite heavily on what s390 does, and it uses the
generic interfaces that were added by the s390 developers,
i.e. account_system_time(), account_user_time(), etc.
This patch doesn't add any new interfaces between the kernel and
userspace, and doesn't change the units in which time is reported to
userspace by things such as /proc/stat, /proc/<pid>/stat, getrusage(),
times(), etc. Internally the various task and cpu times are stored in
timebase units, but they are converted to USER_HZ units (1/100th of a
second) when reported to userspace. Some precision is therefore lost
but there should not be any accumulating error, since the internal
accumulation is at full precision.
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-02-24 02:06:59 +03:00
ACCOUNT_ C P U _ U S E R _ E N T R Y ( r9 , r10 ) ; \
2005-09-26 10:04:21 +04:00
std r2 ,G P R 2 ( r1 ) ; /* save r2 in stackframe */ \
SAVE_ 4 G P R S ( 3 , r1 ) ; /* save r3 - r6 in stackframe */ \
SAVE_ 2 G P R S ( 7 , r1 ) ; /* save r7, r8 in stackframe */ \
ld r9 ,a r e a + E X _ R 9 ( r13 ) ; /* move r9, r10 to stackframe */ \
ld r10 ,a r e a + E X _ R 1 0 ( r13 ) ; \
std r9 ,G P R 9 ( r1 ) ; \
std r10 ,G P R 1 0 ( r1 ) ; \
ld r9 ,a r e a + E X _ R 1 1 ( r13 ) ; /* move r11 - r13 to stackframe */ \
ld r10 ,a r e a + E X _ R 1 2 ( r13 ) ; \
ld r11 ,a r e a + E X _ R 1 3 ( r13 ) ; \
std r9 ,G P R 1 1 ( r1 ) ; \
std r10 ,G P R 1 2 ( r1 ) ; \
std r11 ,G P R 1 3 ( r1 ) ; \
ld r2 ,P A C A T O C ( r13 ) ; /* get kernel TOC into r2 */ \
mflr r9 ; /* save LR in stackframe */ \
std r9 ,_ L I N K ( r1 ) ; \
mfctr r10 ; /* save CTR in stackframe */ \
std r10 ,_ C T R ( r1 ) ; \
2005-10-10 08:01:07 +04:00
mfspr r11 ,S P R N _ X E R ; /* save XER in stackframe */ \
2005-09-26 10:04:21 +04:00
std r11 ,_ X E R ( r1 ) ; \
li r9 ,( n ) + 1 ; \
std r9 ,_ T R A P ( r1 ) ; /* set trap number */ \
li r10 ,0 ; \
ld r11 ,e x c e p t i o n _ m a r k e r @toc(r2); \
std r10 ,R E S U L T ( r1 ) ; /* clear regs->result */ \
std r11 ,S T A C K _ F R A M E _ O V E R H E A D - 1 6 ( r1 ) ; /* mark the frame */
/ *
* Exception v e c t o r s .
* /
# define S T D _ E X C E P T I O N _ P S E R I E S ( n , l a b e l ) \
. = n; \
.globl # _ p S e r i e s ; \
label## _ p S e r i e s : \
HMT_ M E D I U M ; \
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13 ; /* save r13 */ \
2005-09-26 10:04:21 +04:00
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X G E N , l a b e l ## _ c o m m o n )
2006-06-19 22:33:16 +04:00
# define H S T D _ E X C E P T I O N _ P S E R I E S ( n , l a b e l ) \
. = n; \
.globl # _ p S e r i e s ; \
label## _ p S e r i e s : \
HMT_ M E D I U M ; \
mtspr S P R N _ S P R G 1 ,r20 ; /* save r20 */ \
mfspr r20 ,S P R N _ H S R R 0 ; /* copy HSRR0 to SRR0 */ \
mtspr S P R N _ S R R 0 ,r20 ; \
mfspr r20 ,S P R N _ H S R R 1 ; /* copy HSRR0 to SRR0 */ \
mtspr S P R N _ S R R 1 ,r20 ; \
mfspr r20 ,S P R N _ S P R G 1 ; /* restore r20 */ \
mtspr S P R N _ S P R G 1 ,r13 ; /* save r13 */ \
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X G E N , l a b e l ## _ c o m m o n )
2005-09-26 10:04:21 +04:00
# define S T D _ E X C E P T I O N _ I S E R I E S ( n , l a b e l , a r e a ) \
.globl # _ i S e r i e s ; \
label## _ i S e r i e s : \
HMT_ M E D I U M ; \
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13 ; /* save r13 */ \
2005-09-26 10:04:21 +04:00
EXCEPTION_ P R O L O G _ I S E R I E S _ 1 ( a r e a ) ; \
EXCEPTION_ P R O L O G _ I S E R I E S _ 2 ; \
b l a b e l ## _ c o m m o n
# define M A S K A B L E _ E X C E P T I O N _ I S E R I E S ( n , l a b e l ) \
.globl # _ i S e r i e s ; \
label## _ i S e r i e s : \
HMT_ M E D I U M ; \
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13 ; /* save r13 */ \
2005-09-26 10:04:21 +04:00
EXCEPTION_ P R O L O G _ I S E R I E S _ 1 ( P A C A _ E X G E N ) ; \
lbz r10 ,P A C A P R O C E N A B L E D ( r13 ) ; \
cmpwi 0 ,r10 ,0 ; \
beq- l a b e l ## _ i S e r i e s _ m a s k e d ; \
EXCEPTION_ P R O L O G _ I S E R I E S _ 2 ; \
b l a b e l ## _ c o m m o n ; \
# ifdef D O _ S O F T _ D I S A B L E
# define D I S A B L E _ I N T S \
lbz r10 ,P A C A P R O C E N A B L E D ( r13 ) ; \
li r11 ,0 ; \
std r10 ,S O F T E ( r1 ) ; \
mfmsr r10 ; \
stb r11 ,P A C A P R O C E N A B L E D ( r13 ) ; \
ori r10 ,r10 ,M S R _ E E ; \
mtmsrd r10 ,1
# define E N A B L E _ I N T S \
lbz r10 ,P A C A P R O C E N A B L E D ( r13 ) ; \
mfmsr r11 ; \
std r10 ,S O F T E ( r1 ) ; \
ori r11 ,r11 ,M S R _ E E ; \
mtmsrd r11 ,1
# else / * h a r d e n a b l e / d i s a b l e i n t e r r u p t s * /
# define D I S A B L E _ I N T S
# define E N A B L E _ I N T S \
ld r12 ,_ M S R ( r1 ) ; \
mfmsr r11 ; \
rlwimi r11 ,r12 ,0 ,M S R _ E E ; \
mtmsrd r11 ,1
# endif
# define S T D _ E X C E P T I O N _ C O M M O N ( t r a p , l a b e l , h d l r ) \
.align 7 ; \
.globl # _ c o m m o n ; \
label## _ c o m m o n : \
EXCEPTION_ P R O L O G _ C O M M O N ( t r a p , P A C A _ E X G E N ) ; \
DISABLE_ I N T S ; \
bl . s a v e _ n v g p r s ; \
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D ; \
bl h d l r ; \
b . r e t _ f r o m _ e x c e p t
2006-04-18 15:49:11 +04:00
/ *
* Like S T D _ E X C E P T I O N _ C O M M O N , b u t f o r e x c e p t i o n s t h a t c a n o c c u r
* in t h e i d l e t a s k a n d t h e r e f o r e n e e d t h e s p e c i a l i d l e h a n d l i n g .
* /
# define S T D _ E X C E P T I O N _ C O M M O N _ I D L E ( t r a p , l a b e l , h d l r ) \
.align 7 ; \
.globl # _ c o m m o n ; \
label## _ c o m m o n : \
EXCEPTION_ P R O L O G _ C O M M O N ( t r a p , P A C A _ E X G E N ) ; \
FINISH_ N A P ; \
DISABLE_ I N T S ; \
bl . s a v e _ n v g p r s ; \
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D ; \
bl h d l r ; \
b . r e t _ f r o m _ e x c e p t
2005-09-26 10:04:21 +04:00
# define S T D _ E X C E P T I O N _ C O M M O N _ L I T E ( t r a p , l a b e l , h d l r ) \
.align 7 ; \
.globl # _ c o m m o n ; \
label## _ c o m m o n : \
EXCEPTION_ P R O L O G _ C O M M O N ( t r a p , P A C A _ E X G E N ) ; \
2006-04-18 15:49:11 +04:00
FINISH_ N A P ; \
2005-09-26 10:04:21 +04:00
DISABLE_ I N T S ; \
2006-02-13 06:48:35 +03:00
bl . p p c64 _ r u n l a t c h _ o n ; \
2005-09-26 10:04:21 +04:00
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D ; \
bl h d l r ; \
b . r e t _ f r o m _ e x c e p t _ l i t e
2006-04-18 15:49:11 +04:00
/ *
* When t h e i d l e c o d e i n p o w e r4 _ i d l e p u t s t h e C P U i n t o N A P m o d e ,
* it h a s t o d o s o i n a l o o p , a n d r e l i e s o n t h e e x t e r n a l i n t e r r u p t
* and d e c r e m e n t e r i n t e r r u p t e n t r y c o d e t o g e t i t o u t o f t h e l o o p .
* It s e t s t h e _ T L F _ N A P P I N G b i t i n c u r r e n t _ t h r e a d _ i n f o ( ) - > l o c a l _ f l a g s
* to s i g n a l t h a t i t i s i n t h e l o o p a n d n e e d s h e l p t o g e t o u t .
* /
# ifdef C O N F I G _ P P C _ 9 7 0 _ N A P
# define F I N I S H _ N A P \
BEGIN_ F T R _ S E C T I O N \
clrrdi r11 ,r1 ,T H R E A D _ S H I F T ; \
ld r9 ,T I _ L O C A L _ F L A G S ( r11 ) ; \
andi. r10 ,r9 ,_ T L F _ N A P P I N G ; \
bnel p o w e r4 _ f i x u p _ n a p ; \
END_ F T R _ S E C T I O N _ I F S E T ( C P U _ F T R _ C A N _ N A P )
# else
# define F I N I S H _ N A P
# endif
2005-09-26 10:04:21 +04:00
/ *
* Start o f p S e r i e s s y s t e m i n t e r r u p t r o u t i n e s
* /
. = 0 x1 0 0
.globl __start_interrupts
__start_interrupts :
STD_ E X C E P T I O N _ P S E R I E S ( 0 x10 0 , s y s t e m _ r e s e t )
. = 0 x2 0 0
_machine_check_pSeries :
HMT_ M E D I U M
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13 / * s a v e r13 * /
2005-09-26 10:04:21 +04:00
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X M C , m a c h i n e _ c h e c k _ c o m m o n )
. = 0 x3 0 0
.globl data_access_pSeries
data_access_pSeries :
HMT_ M E D I U M
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13
2005-09-26 10:04:21 +04:00
BEGIN_ F T R _ S E C T I O N
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 2 ,r12
mfspr r13 ,S P R N _ D A R
mfspr r12 ,S P R N _ D S I S R
2005-09-26 10:04:21 +04:00
srdi r13 ,r13 ,6 0
rlwimi r13 ,r12 ,1 6 ,0 x20
mfcr r12
cmpwi r13 ,0 x2 c
beq . d o _ s t a b _ b o l t e d _ p S e r i e s
mtcrf 0 x80 ,r12
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S P R G 2
2005-09-26 10:04:21 +04:00
END_ F T R _ S E C T I O N _ I F C L R ( C P U _ F T R _ S L B )
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X G E N , d a t a _ a c c e s s _ c o m m o n )
. = 0 x3 8 0
.globl data_access_slb_pSeries
data_access_slb_pSeries :
HMT_ M E D I U M
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13
mfspr r13 ,S P R N _ S P R G 3 / * g e t p a c a a d d r e s s i n t o r13 * /
2005-11-07 03:06:55 +03:00
std r3 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
mfspr r3 ,S P R N _ D A R
2005-09-26 10:04:21 +04:00
std r9 ,P A C A _ E X S L B + E X _ R 9 ( r13 ) / * s a v e r9 - r12 * /
2005-11-07 03:06:55 +03:00
mfcr r9
# ifdef _ _ D I S A B L E D _ _
/* Keep that around for when we re-implement dynamic VSIDs */
cmpdi r3 ,0
bge s l b _ m i s s _ u s e r _ p s e r i e s
# endif / * _ _ D I S A B L E D _ _ * /
2005-09-26 10:04:21 +04:00
std r10 ,P A C A _ E X S L B + E X _ R 1 0 ( r13 )
std r11 ,P A C A _ E X S L B + E X _ R 1 1 ( r13 )
std r12 ,P A C A _ E X S L B + E X _ R 1 2 ( r13 )
2005-11-07 03:06:55 +03:00
mfspr r10 ,S P R N _ S P R G 1
std r10 ,P A C A _ E X S L B + E X _ R 1 3 ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S R R 1 / * a n d S R R 1 * /
2005-11-07 03:06:55 +03:00
b . s l b _ m i s s _ r e a l m o d e / * R e l . b r a n c h w o r k s i n r e a l m o d e * /
2005-09-26 10:04:21 +04:00
STD_ E X C E P T I O N _ P S E R I E S ( 0 x40 0 , i n s t r u c t i o n _ a c c e s s )
. = 0 x4 8 0
.globl instruction_access_slb_pSeries
instruction_access_slb_pSeries :
HMT_ M E D I U M
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13
mfspr r13 ,S P R N _ S P R G 3 / * g e t p a c a a d d r e s s i n t o r13 * /
2005-11-07 03:06:55 +03:00
std r3 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
mfspr r3 ,S P R N _ S R R 0 / * S R R 0 i s f a u l t i n g a d d r e s s * /
2005-09-26 10:04:21 +04:00
std r9 ,P A C A _ E X S L B + E X _ R 9 ( r13 ) / * s a v e r9 - r12 * /
2005-11-07 03:06:55 +03:00
mfcr r9
# ifdef _ _ D I S A B L E D _ _
/* Keep that around for when we re-implement dynamic VSIDs */
cmpdi r3 ,0
bge s l b _ m i s s _ u s e r _ p s e r i e s
# endif / * _ _ D I S A B L E D _ _ * /
2005-09-26 10:04:21 +04:00
std r10 ,P A C A _ E X S L B + E X _ R 1 0 ( r13 )
std r11 ,P A C A _ E X S L B + E X _ R 1 1 ( r13 )
std r12 ,P A C A _ E X S L B + E X _ R 1 2 ( r13 )
2005-11-07 03:06:55 +03:00
mfspr r10 ,S P R N _ S P R G 1
std r10 ,P A C A _ E X S L B + E X _ R 1 3 ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S R R 1 / * a n d S R R 1 * /
2005-11-07 03:06:55 +03:00
b . s l b _ m i s s _ r e a l m o d e / * R e l . b r a n c h w o r k s i n r e a l m o d e * /
2005-09-26 10:04:21 +04:00
STD_ E X C E P T I O N _ P S E R I E S ( 0 x50 0 , h a r d w a r e _ i n t e r r u p t )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x60 0 , a l i g n m e n t )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x70 0 , p r o g r a m _ c h e c k )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x80 0 , f p _ u n a v a i l a b l e )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x90 0 , d e c r e m e n t e r )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x a00 , t r a p _ 0 a )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x b00 , t r a p _ 0 b )
. = 0 xc0 0
.globl system_call_pSeries
system_call_pSeries :
HMT_ M E D I U M
mr r9 ,r13
mfmsr r10
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3
mfspr r11 ,S P R N _ S R R 0
2005-09-26 10:04:21 +04:00
clrrdi r12 ,r13 ,3 2
oris r12 ,r12 ,s y s t e m _ c a l l _ c o m m o n @h
ori r12 ,r12 ,s y s t e m _ c a l l _ c o m m o n @l
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 0 ,r12
2005-09-26 10:04:21 +04:00
ori r10 ,r10 ,M S R _ I R | M S R _ D R | M S R _ R I
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S R R 1
mtspr S P R N _ S R R 1 ,r10
2005-09-26 10:04:21 +04:00
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
STD_ E X C E P T I O N _ P S E R I E S ( 0 x d00 , s i n g l e _ s t e p )
STD_ E X C E P T I O N _ P S E R I E S ( 0 x e 0 0 , t r a p _ 0 e )
/ * We n e e d t o d e a l w i t h t h e A l t i v e c u n a v a i l a b l e e x c e p t i o n
* here w h i c h i s a t 0 x f20 , t h u s i n t h e m i d d l e o f t h e
* prolog c o d e o f t h e P e r f o r m a n c e M o n i t o r o n e . A l i t t l e
* trickery i s t h u s n e c e s s a r y
* /
. = 0 xf0 0
b p e r f o r m a n c e _ m o n i t o r _ p S e r i e s
STD_ E X C E P T I O N _ P S E R I E S ( 0 x f20 , a l t i v e c _ u n a v a i l a b l e )
2006-06-19 22:33:16 +04:00
# ifdef C O N F I G _ C B E _ R A S
HSTD_ E X C E P T I O N _ P S E R I E S ( 0 x12 0 0 , c b e _ s y s t e m _ e r r o r )
# endif / * C O N F I G _ C B E _ R A S * /
2005-09-26 10:04:21 +04:00
STD_ E X C E P T I O N _ P S E R I E S ( 0 x13 0 0 , i n s t r u c t i o n _ b r e a k p o i n t )
2006-06-19 22:33:16 +04:00
# ifdef C O N F I G _ C B E _ R A S
HSTD_ E X C E P T I O N _ P S E R I E S ( 0 x16 0 0 , c b e _ m a i n t e n a n c e )
# endif / * C O N F I G _ C B E _ R A S * /
2005-09-26 10:04:21 +04:00
STD_ E X C E P T I O N _ P S E R I E S ( 0 x17 0 0 , a l t i v e c _ a s s i s t )
2006-06-19 22:33:16 +04:00
# ifdef C O N F I G _ C B E _ R A S
HSTD_ E X C E P T I O N _ P S E R I E S ( 0 x18 0 0 , c b e _ t h e r m a l )
# endif / * C O N F I G _ C B E _ R A S * /
2005-09-26 10:04:21 +04:00
. = 0 x3 0 0 0
/*** pSeries interrupt support ***/
/* moved from 0xf00 */
STD_ E X C E P T I O N _ P S E R I E S ( . , p e r f o r m a n c e _ m o n i t o r )
.align 7
_ GLOBAL( d o _ s t a b _ b o l t e d _ p S e r i e s )
mtcrf 0 x80 ,r12
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S P R G 2
2005-09-26 10:04:21 +04:00
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X S L B , . d o _ s t a b _ b o l t e d )
2005-11-07 03:06:55 +03:00
/ *
* We h a v e s o m e r o o m h e r e w e u s e t h a t t o p u t
* the p e r i e s s l b m i s s u s e r t r a m p o l i n e c o d e s o i t ' s r e a s o n a b l y
* away f r o m s l b _ m i s s _ u s e r _ c o m m o n t o a v o i d p r o b l e m s w i t h r f i d
*
* This i s u s e d f o r w h e n t h e S L B m i s s h a n d l e r h a s t o g o v i r t u a l ,
* which d o e s n ' t h a p p e n f o r n o w a n y m o r e b u t w i l l o n c e w e r e - i m p l e m e n t
* dynamic V S I D s f o r s h a r e d p a g e t a b l e s
* /
# ifdef _ _ D I S A B L E D _ _
slb_miss_user_pseries :
std r10 ,P A C A _ E X G E N + E X _ R 1 0 ( r13 )
std r11 ,P A C A _ E X G E N + E X _ R 1 1 ( r13 )
std r12 ,P A C A _ E X G E N + E X _ R 1 2 ( r13 )
mfspr r10 ,S P R G 1
ld r11 ,P A C A _ E X S L B + E X _ R 9 ( r13 )
ld r12 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
std r10 ,P A C A _ E X G E N + E X _ R 1 3 ( r13 )
std r11 ,P A C A _ E X G E N + E X _ R 9 ( r13 )
std r12 ,P A C A _ E X G E N + E X _ R 3 ( r13 )
clrrdi r12 ,r13 ,3 2
mfmsr r10
mfspr r11 ,S R R 0 / * s a v e S R R 0 * /
ori r12 ,r12 ,s l b _ m i s s _ u s e r _ c o m m o n @l /* virt addr of handler */
ori r10 ,r10 ,M S R _ I R | M S R _ D R | M S R _ R I
mtspr S R R 0 ,r12
mfspr r12 ,S R R 1 / * a n d S R R 1 * /
mtspr S R R 1 ,r10
rfid
b . / * p r e v e n t s p e c . e x e c u t i o n * /
# endif / * _ _ D I S A B L E D _ _ * /
2005-09-26 10:04:21 +04:00
/ *
* Vectors f o r t h e F W N M I o p t i o n . S h a r e c o m m o n c o d e .
* /
2005-10-10 08:01:07 +04:00
.globl system_reset_fwnmi
2005-12-04 10:39:33 +03:00
.align 7
2005-09-26 10:04:21 +04:00
system_reset_fwnmi :
2005-10-10 08:01:07 +04:00
HMT_ M E D I U M
mtspr S P R N _ S P R G 1 ,r13 / * s a v e r13 * /
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X G E N , s y s t e m _ r e s e t _ c o m m o n )
2005-09-26 10:04:21 +04:00
2005-10-10 08:01:07 +04:00
.globl machine_check_fwnmi
2005-12-04 10:39:33 +03:00
.align 7
2005-09-26 10:04:21 +04:00
machine_check_fwnmi :
2005-10-10 08:01:07 +04:00
HMT_ M E D I U M
mtspr S P R N _ S P R G 1 ,r13 / * s a v e r13 * /
EXCEPTION_ P R O L O G _ P S E R I E S ( P A C A _ E X M C , m a c h i n e _ c h e c k _ c o m m o n )
2005-09-26 10:04:21 +04:00
# ifdef C O N F I G _ P P C _ I S E R I E S
/*** ISeries-LPAR interrupt handlers ***/
STD_ E X C E P T I O N _ I S E R I E S ( 0 x20 0 , m a c h i n e _ c h e c k , P A C A _ E X M C )
.globl data_access_iSeries
data_access_iSeries :
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13
2005-09-26 10:04:21 +04:00
BEGIN_ F T R _ S E C T I O N
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 2 ,r12
mfspr r13 ,S P R N _ D A R
mfspr r12 ,S P R N _ D S I S R
2005-09-26 10:04:21 +04:00
srdi r13 ,r13 ,6 0
rlwimi r13 ,r12 ,1 6 ,0 x20
mfcr r12
cmpwi r13 ,0 x2 c
beq . d o _ s t a b _ b o l t e d _ i S e r i e s
mtcrf 0 x80 ,r12
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S P R G 2
2005-09-26 10:04:21 +04:00
END_ F T R _ S E C T I O N _ I F C L R ( C P U _ F T R _ S L B )
EXCEPTION_ P R O L O G _ I S E R I E S _ 1 ( P A C A _ E X G E N )
EXCEPTION_ P R O L O G _ I S E R I E S _ 2
b d a t a _ a c c e s s _ c o m m o n
.do_stab_bolted_iSeries :
mtcrf 0 x80 ,r12
2005-10-10 08:01:07 +04:00
mfspr r12 ,S P R N _ S P R G 2
2005-09-26 10:04:21 +04:00
EXCEPTION_ P R O L O G _ I S E R I E S _ 1 ( P A C A _ E X S L B )
EXCEPTION_ P R O L O G _ I S E R I E S _ 2
b . d o _ s t a b _ b o l t e d
.globl data_access_slb_iSeries
data_access_slb_iSeries :
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13 / * s a v e r13 * /
2005-11-07 03:06:55 +03:00
mfspr r13 ,S P R N _ S P R G 3 / * g e t p a c a a d d r e s s i n t o r13 * /
2005-09-26 10:04:21 +04:00
std r3 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r3 ,S P R N _ D A R
2005-11-07 03:06:55 +03:00
std r9 ,P A C A _ E X S L B + E X _ R 9 ( r13 )
mfcr r9
# ifdef _ _ D I S A B L E D _ _
cmpdi r3 ,0
bge s l b _ m i s s _ u s e r _ i s e r i e s
# endif
std r10 ,P A C A _ E X S L B + E X _ R 1 0 ( r13 )
std r11 ,P A C A _ E X S L B + E X _ R 1 1 ( r13 )
std r12 ,P A C A _ E X S L B + E X _ R 1 2 ( r13 )
mfspr r10 ,S P R N _ S P R G 1
std r10 ,P A C A _ E X S L B + E X _ R 1 3 ( r13 )
2006-01-13 02:26:42 +03:00
ld r12 ,P A C A L P P A C A P T R ( r13 )
ld r12 ,L P P A C A S R R 1 ( r12 )
2005-11-07 03:06:55 +03:00
b . s l b _ m i s s _ r e a l m o d e
2005-09-26 10:04:21 +04:00
STD_ E X C E P T I O N _ I S E R I E S ( 0 x40 0 , i n s t r u c t i o n _ a c c e s s , P A C A _ E X G E N )
.globl instruction_access_slb_iSeries
instruction_access_slb_iSeries :
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 1 ,r13 / * s a v e r13 * /
2005-11-07 03:06:55 +03:00
mfspr r13 ,S P R N _ S P R G 3 / * g e t p a c a a d d r e s s i n t o r13 * /
2005-09-26 10:04:21 +04:00
std r3 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
2006-01-13 02:26:42 +03:00
ld r3 ,P A C A L P P A C A P T R ( r13 )
ld r3 ,L P P A C A S R R 0 ( r3 ) / * g e t S R R 0 v a l u e * /
2005-11-07 03:06:55 +03:00
std r9 ,P A C A _ E X S L B + E X _ R 9 ( r13 )
mfcr r9
# ifdef _ _ D I S A B L E D _ _
cmpdi r3 ,0
bge . s l b _ m i s s _ u s e r _ i s e r i e s
# endif
std r10 ,P A C A _ E X S L B + E X _ R 1 0 ( r13 )
std r11 ,P A C A _ E X S L B + E X _ R 1 1 ( r13 )
std r12 ,P A C A _ E X S L B + E X _ R 1 2 ( r13 )
mfspr r10 ,S P R N _ S P R G 1
std r10 ,P A C A _ E X S L B + E X _ R 1 3 ( r13 )
2006-01-13 02:26:42 +03:00
ld r12 ,P A C A L P P A C A P T R ( r13 )
ld r12 ,L P P A C A S R R 1 ( r12 )
2005-11-07 03:06:55 +03:00
b . s l b _ m i s s _ r e a l m o d e
# ifdef _ _ D I S A B L E D _ _
slb_miss_user_iseries :
std r10 ,P A C A _ E X G E N + E X _ R 1 0 ( r13 )
std r11 ,P A C A _ E X G E N + E X _ R 1 1 ( r13 )
std r12 ,P A C A _ E X G E N + E X _ R 1 2 ( r13 )
mfspr r10 ,S P R G 1
ld r11 ,P A C A _ E X S L B + E X _ R 9 ( r13 )
ld r12 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
std r10 ,P A C A _ E X G E N + E X _ R 1 3 ( r13 )
std r11 ,P A C A _ E X G E N + E X _ R 9 ( r13 )
std r12 ,P A C A _ E X G E N + E X _ R 3 ( r13 )
EXCEPTION_ P R O L O G _ I S E R I E S _ 2
b s l b _ m i s s _ u s e r _ c o m m o n
# endif
2005-09-26 10:04:21 +04:00
MASKABLE_ E X C E P T I O N _ I S E R I E S ( 0 x50 0 , h a r d w a r e _ i n t e r r u p t )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x60 0 , a l i g n m e n t , P A C A _ E X G E N )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x70 0 , p r o g r a m _ c h e c k , P A C A _ E X G E N )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x80 0 , f p _ u n a v a i l a b l e , P A C A _ E X G E N )
MASKABLE_ E X C E P T I O N _ I S E R I E S ( 0 x90 0 , d e c r e m e n t e r )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x a00 , t r a p _ 0 a , P A C A _ E X G E N )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x b00 , t r a p _ 0 b , P A C A _ E X G E N )
.globl system_call_iSeries
system_call_iSeries :
mr r9 ,r13
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3
2005-09-26 10:04:21 +04:00
EXCEPTION_ P R O L O G _ I S E R I E S _ 2
b s y s t e m _ c a l l _ c o m m o n
STD_ E X C E P T I O N _ I S E R I E S ( 0 x d00 , s i n g l e _ s t e p , P A C A _ E X G E N )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x e 0 0 , t r a p _ 0 e , P A C A _ E X G E N )
STD_ E X C E P T I O N _ I S E R I E S ( 0 x f00 , p e r f o r m a n c e _ m o n i t o r , P A C A _ E X G E N )
.globl system_reset_iSeries
system_reset_iSeries :
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3 / * G e t p a c a a d d r e s s * /
2005-09-26 10:04:21 +04:00
mfmsr r24
ori r24 ,r24 ,M S R _ R I
mtmsrd r24 / * R I o n * /
lhz r24 ,P A C A P A C A I N D E X ( r13 ) / * G e t p r o c e s s o r # * /
cmpwi 0 ,r24 ,0 / * A r e w e p r o c e s s o r 0 ? * /
beq . _ _ s t a r t _ i n i t i a l i z a t i o n _ i S e r i e s / * S t a r t u p t h e f i r s t p r o c e s s o r * /
mfspr r4 ,S P R N _ C T R L F
li r5 ,C T R L _ R U N L A T C H / * T u r n o f f t h e r u n l i g h t * /
andc r4 ,r4 ,r5
mtspr S P R N _ C T R L T ,r4
1 :
HMT_ L O W
# ifdef C O N F I G _ S M P
lbz r23 ,P A C A P R O C S T A R T ( r13 ) / * T e s t i f t h i s p r o c e s s o r
* should s t a r t * /
sync
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r3 ,c u r r e n t _ s e t )
2005-09-26 10:04:21 +04:00
sldi r28 ,r24 ,3 / * g e t c u r r e n t _ s e t [ c p u #] * /
ldx r3 ,r3 ,r28
addi r1 ,r3 ,T H R E A D _ S I Z E
subi r1 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
cmpwi 0 ,r23 ,0
beq i S e r i e s _ s e c o n d a r y _ s m p _ l o o p / * L o o p u n t i l t o l d t o g o * /
bne . _ _ s e c o n d a r y _ s t a r t / * L o o p u n t i l t o l d t o g o * /
iSeries_secondary_smp_loop :
/* Let the Hypervisor know we are alive */
/* 8002 is a call to HvCallCfg::getLps, a harmless Hypervisor function */
lis r3 ,0 x80 0 2
rldicr r3 ,r3 ,3 2 ,1 5 / * r0 = ( r3 < < 3 2 ) & 0 x f f f f00 0 0 0 0 0 0 0 0 0 0 * /
# else / * C O N F I G _ S M P * /
/ * Yield t h e p r o c e s s o r . T h i s i s r e q u i r e d f o r n o n - S M P k e r n e l s
which a r e r u n n i n g o n m u l t i - t h r e a d e d m a c h i n e s . * /
lis r3 ,0 x80 0 0
rldicr r3 ,r3 ,3 2 ,1 5 / * r3 = ( r3 < < 3 2 ) & 0 x f f f f00 0 0 0 0 0 0 0 0 0 0 * /
addi r3 ,r3 ,1 8 / * r3 = 0 x80 0 0 0 0 0 0 0 0 0 0 0 0 1 2 w h i c h i s " y i e l d " * /
li r4 ,0 / * " y i e l d t i m e d " * /
li r5 ,- 1 / * " y i e l d f o r e v e r " * /
# endif / * C O N F I G _ S M P * /
li r0 ,- 1 / * r0 = - 1 i n d i c a t e s a H y p e r v i s o r c a l l * /
sc / * I n v o k e t h e h y p e r v i s o r v i a a s y s t e m c a l l * /
2005-10-10 08:01:07 +04:00
mfspr r13 ,S P R N _ S P R G 3 / * P u t r13 b a c k ? ? ? ? * /
2005-09-26 10:04:21 +04:00
b 1 b / * I f S M P n o t c o n f i g u r e d , s e c o n d a r i e s
* loop f o r e v e r * /
.globl decrementer_iSeries_masked
decrementer_iSeries_masked :
2006-02-07 05:26:14 +03:00
/* We may not have a valid TOC pointer in here. */
2005-09-26 10:04:21 +04:00
li r11 ,1
2006-01-13 02:26:42 +03:00
ld r12 ,P A C A L P P A C A P T R ( r13 )
stb r11 ,L P P A C A D E C R I N T ( r12 )
2006-02-07 05:26:14 +03:00
LOAD_ R E G _ I M M E D I A T E ( r12 , t b _ t i c k s _ p e r _ j i f f y )
lwz r12 ,0 ( r12 )
2005-09-26 10:04:21 +04:00
mtspr S P R N _ D E C ,r12
/* fall through */
.globl hardware_interrupt_iSeries_masked
hardware_interrupt_iSeries_masked :
mtcrf 0 x80 ,r9 / * R e s t o r e r e g s * /
2006-01-13 02:26:42 +03:00
ld r12 ,P A C A L P P A C A P T R ( r13 )
ld r11 ,L P P A C A S R R 0 ( r12 )
ld r12 ,L P P A C A S R R 1 ( r12 )
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 0 ,r11
mtspr S P R N _ S R R 1 ,r12
2005-09-26 10:04:21 +04:00
ld r9 ,P A C A _ E X G E N + E X _ R 9 ( r13 )
ld r10 ,P A C A _ E X G E N + E X _ R 1 0 ( r13 )
ld r11 ,P A C A _ E X G E N + E X _ R 1 1 ( r13 )
ld r12 ,P A C A _ E X G E N + E X _ R 1 2 ( r13 )
ld r13 ,P A C A _ E X G E N + E X _ R 1 3 ( r13 )
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
# endif / * C O N F I G _ P P C _ I S E R I E S * /
/*** Common interrupt handlers ***/
STD_ E X C E P T I O N _ C O M M O N ( 0 x10 0 , s y s t e m _ r e s e t , . s y s t e m _ r e s e t _ e x c e p t i o n )
/ *
* Machine c h e c k i s d i f f e r e n t b e c a u s e w e u s e a d i f f e r e n t
* save a r e a : P A C A _ E X M C i n s t e a d o f P A C A _ E X G E N .
* /
.align 7
.globl machine_check_common
machine_check_common :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x20 0 , P A C A _ E X M C )
2006-04-18 15:49:11 +04:00
FINISH_ N A P
2005-09-26 10:04:21 +04:00
DISABLE_ I N T S
bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . m a c h i n e _ c h e c k _ e x c e p t i o n
b . r e t _ f r o m _ e x c e p t
STD_ E X C E P T I O N _ C O M M O N _ L I T E ( 0 x90 0 , d e c r e m e n t e r , . t i m e r _ i n t e r r u p t )
STD_ E X C E P T I O N _ C O M M O N ( 0 x a00 , t r a p _ 0 a , . u n k n o w n _ e x c e p t i o n )
STD_ E X C E P T I O N _ C O M M O N ( 0 x b00 , t r a p _ 0 b , . u n k n o w n _ e x c e p t i o n )
STD_ E X C E P T I O N _ C O M M O N ( 0 x d00 , s i n g l e _ s t e p , . s i n g l e _ s t e p _ e x c e p t i o n )
STD_ E X C E P T I O N _ C O M M O N ( 0 x e 0 0 , t r a p _ 0 e , . u n k n o w n _ e x c e p t i o n )
2006-04-18 15:49:11 +04:00
STD_ E X C E P T I O N _ C O M M O N _ I D L E ( 0 x f00 , p e r f o r m a n c e _ m o n i t o r , . p e r f o r m a n c e _ m o n i t o r _ e x c e p t i o n )
2005-09-26 10:04:21 +04:00
STD_ E X C E P T I O N _ C O M M O N ( 0 x13 0 0 , i n s t r u c t i o n _ b r e a k p o i n t , . i n s t r u c t i o n _ b r e a k p o i n t _ e x c e p t i o n )
# ifdef C O N F I G _ A L T I V E C
STD_ E X C E P T I O N _ C O M M O N ( 0 x17 0 0 , a l t i v e c _ a s s i s t , . a l t i v e c _ a s s i s t _ e x c e p t i o n )
# else
STD_ E X C E P T I O N _ C O M M O N ( 0 x17 0 0 , a l t i v e c _ a s s i s t , . u n k n o w n _ e x c e p t i o n )
# endif
2006-06-19 22:33:16 +04:00
# ifdef C O N F I G _ C B E _ R A S
STD_ E X C E P T I O N _ C O M M O N ( 0 x12 0 0 , c b e _ s y s t e m _ e r r o r , . c b e _ s y s t e m _ e r r o r _ e x c e p t i o n )
STD_ E X C E P T I O N _ C O M M O N ( 0 x16 0 0 , c b e _ m a i n t e n a n c e , . c b e _ m a i n t e n a n c e _ e x c e p t i o n )
STD_ E X C E P T I O N _ C O M M O N ( 0 x18 0 0 , c b e _ t h e r m a l , . c b e _ t h e r m a l _ e x c e p t i o n )
# endif / * C O N F I G _ C B E _ R A S * /
2005-09-26 10:04:21 +04:00
/ *
* Here w e h a v e d e t e c t e d t h a t t h e k e r n e l s t a c k p o i n t e r i s b a d .
* R9 c o n t a i n s t h e s a v e d C R , r13 p o i n t s t o t h e p a c a ,
* r1 0 c o n t a i n s t h e ( b a d ) k e r n e l s t a c k p o i n t e r ,
* r1 1 a n d r12 c o n t a i n t h e s a v e d S R R 0 a n d S R R 1 .
* We s w i t c h t o u s i n g a n e m e r g e n c y s t a c k , s a v e t h e r e g i s t e r s t h e r e ,
* and c a l l k e r n e l _ b a d _ s t a c k ( ) , w h i c h p a n i c s .
* /
bad_stack :
ld r1 ,P A C A E M E R G S P ( r13 )
subi r1 ,r1 ,6 4 + I N T _ F R A M E _ S I Z E
std r9 ,_ C C R ( r1 )
std r10 ,G P R 1 ( r1 )
std r11 ,_ N I P ( r1 )
std r12 ,_ M S R ( r1 )
2005-10-10 08:01:07 +04:00
mfspr r11 ,S P R N _ D A R
mfspr r12 ,S P R N _ D S I S R
2005-09-26 10:04:21 +04:00
std r11 ,_ D A R ( r1 )
std r12 ,_ D S I S R ( r1 )
mflr r10
mfctr r11
mfxer r12
std r10 ,_ L I N K ( r1 )
std r11 ,_ C T R ( r1 )
std r12 ,_ X E R ( r1 )
SAVE_ G P R ( 0 ,r1 )
SAVE_ G P R ( 2 ,r1 )
SAVE_ 4 G P R S ( 3 ,r1 )
SAVE_ 2 G P R S ( 7 ,r1 )
SAVE_ 1 0 G P R S ( 1 2 ,r1 )
SAVE_ 1 0 G P R S ( 2 2 ,r1 )
addi r11 ,r1 ,I N T _ F R A M E _ S I Z E
std r11 ,0 ( r1 )
li r12 ,0
std r12 ,0 ( r11 )
ld r2 ,P A C A T O C ( r13 )
1 : addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . k e r n e l _ b a d _ s t a c k
b 1 b
/ *
* Return f r o m a n e x c e p t i o n w i t h m i n i m a l c h e c k s .
* The c a l l e r i s a s s u m e d t o h a v e d o n e E X C E P T I O N _ P R O L O G _ C O M M O N .
* If i n t e r r u p t s h a v e b e e n e n a b l e d , o r a n y t h i n g h a s b e e n
* done t h a t m i g h t h a v e c h a n g e d t h e s c h e d u l i n g s t a t u s o f
* any t a s k o r s e n t a n y t a s k a s i g n a l , y o u s h o u l d u s e
* ret_ f r o m _ e x c e p t o r r e t _ f r o m _ e x c e p t _ l i t e i n s t e a d o f t h i s .
* /
2005-10-10 16:50:37 +04:00
.globl fast_exception_return
2005-09-26 10:04:21 +04:00
fast_exception_return :
ld r12 ,_ M S R ( r1 )
ld r11 ,_ N I P ( r1 )
andi. r3 ,r12 ,M S R _ R I / * c h e c k i f R I i s s e t * /
beq- u n r e c o v _ f e r
powerpc: Implement accurate task and CPU time accounting
This implements accurate task and cpu time accounting for 64-bit
powerpc kernels. Instead of accounting a whole jiffy of time to a
task on a timer interrupt because that task happened to be running at
the time, we now account time in units of timebase ticks according to
the actual time spent by the task in user mode and kernel mode. We
also count the time spent processing hardware and software interrupts
accurately. This is conditional on CONFIG_VIRT_CPU_ACCOUNTING. If
that is not set, we do tick-based approximate accounting as before.
To get this accurate information, we read either the PURR (processor
utilization of resources register) on POWER5 machines, or the timebase
on other machines on
* each entry to the kernel from usermode
* each exit to usermode
* transitions between process context, hard irq context and soft irq
context in kernel mode
* context switches.
On POWER5 systems with shared-processor logical partitioning we also
read both the PURR and the timebase at each timer interrupt and
context switch in order to determine how much time has been taken by
the hypervisor to run other partitions ("steal" time). Unfortunately,
since we need values of the PURR on both threads at the same time to
accurately calculate the steal time, and since we can only calculate
steal time on a per-core basis, the apportioning of the steal time
between idle time (time which we ceded to the hypervisor in the idle
loop) and actual stolen time is somewhat approximate at the moment.
This is all based quite heavily on what s390 does, and it uses the
generic interfaces that were added by the s390 developers,
i.e. account_system_time(), account_user_time(), etc.
This patch doesn't add any new interfaces between the kernel and
userspace, and doesn't change the units in which time is reported to
userspace by things such as /proc/stat, /proc/<pid>/stat, getrusage(),
times(), etc. Internally the various task and cpu times are stored in
timebase units, but they are converted to USER_HZ units (1/100th of a
second) when reported to userspace. Some precision is therefore lost
but there should not be any accumulating error, since the internal
accumulation is at full precision.
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-02-24 02:06:59 +03:00
# ifdef C O N F I G _ V I R T _ C P U _ A C C O U N T I N G
andi. r3 ,r12 ,M S R _ P R
beq 2 f
ACCOUNT_ C P U _ U S E R _ E X I T ( r3 , r4 )
2 :
# endif
2005-09-26 10:04:21 +04:00
ld r3 ,_ C C R ( r1 )
ld r4 ,_ L I N K ( r1 )
ld r5 ,_ C T R ( r1 )
ld r6 ,_ X E R ( r1 )
mtcr r3
mtlr r4
mtctr r5
mtxer r6
REST_ G P R ( 0 , r1 )
REST_ 8 G P R S ( 2 , r1 )
mfmsr r10
clrrdi r10 ,r10 ,2 / * c l e a r R I ( L E i s 0 a l r e a d y ) * /
mtmsrd r10 ,1
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 1 ,r12
mtspr S P R N _ S R R 0 ,r11
2005-09-26 10:04:21 +04:00
REST_ 4 G P R S ( 1 0 , r1 )
ld r1 ,G P R 1 ( r1 )
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
unrecov_fer :
bl . s a v e _ n v g p r s
1 : addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . u n r e c o v e r a b l e _ e x c e p t i o n
b 1 b
/ *
* Here r13 p o i n t s t o t h e p a c a , r9 c o n t a i n s t h e s a v e d C R ,
* SRR0 a n d S R R 1 a r e s a v e d i n r11 a n d r12 ,
* r9 - r13 a r e s a v e d i n p a c a - > e x g e n .
* /
.align 7
.globl data_access_common
data_access_common :
2005-10-10 08:01:07 +04:00
mfspr r10 ,S P R N _ D A R
2005-09-26 10:04:21 +04:00
std r10 ,P A C A _ E X G E N + E X _ D A R ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r10 ,S P R N _ D S I S R
2005-09-26 10:04:21 +04:00
stw r10 ,P A C A _ E X G E N + E X _ D S I S R ( r13 )
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x30 0 , P A C A _ E X G E N )
ld r3 ,P A C A _ E X G E N + E X _ D A R ( r13 )
lwz r4 ,P A C A _ E X G E N + E X _ D S I S R ( r13 )
li r5 ,0 x30 0
b . d o _ h a s h _ p a g e / * T r y t o h a n d l e a s h p t e f a u l t * /
.align 7
.globl instruction_access_common
instruction_access_common :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x40 0 , P A C A _ E X G E N )
ld r3 ,_ N I P ( r1 )
andis. r4 ,r12 ,0 x58 2 0
li r5 ,0 x40 0
b . d o _ h a s h _ p a g e / * T r y t o h a n d l e a s h p t e f a u l t * /
2005-11-07 03:06:55 +03:00
/ *
* Here i s t h e c o m m o n S L B m i s s u s e r t h a t i s u s e d w h e n g o i n g t o v i r t u a l
* mode f o r S L B m i s s e s , t h a t i s c u r r e n t l y n o t u s e d
* /
# ifdef _ _ D I S A B L E D _ _
.align 7
.globl slb_miss_user_common
slb_miss_user_common :
mflr r10
std r3 ,P A C A _ E X G E N + E X _ D A R ( r13 )
stw r9 ,P A C A _ E X G E N + E X _ C C R ( r13 )
std r10 ,P A C A _ E X G E N + E X _ L R ( r13 )
std r11 ,P A C A _ E X G E N + E X _ S R R 0 ( r13 )
bl . s l b _ a l l o c a t e _ u s e r
ld r10 ,P A C A _ E X G E N + E X _ L R ( r13 )
ld r3 ,P A C A _ E X G E N + E X _ R 3 ( r13 )
lwz r9 ,P A C A _ E X G E N + E X _ C C R ( r13 )
ld r11 ,P A C A _ E X G E N + E X _ S R R 0 ( r13 )
mtlr r10
beq- s l b _ m i s s _ f a u l t
andi. r10 ,r12 ,M S R _ R I / * c h e c k f o r u n r e c o v e r a b l e e x c e p t i o n * /
beq- u n r e c o v _ u s e r _ s l b
mfmsr r10
.machine push
.machine " power4 "
mtcrf 0 x80 ,r9
.machine pop
clrrdi r10 ,r10 ,2 / * c l e a r R I b e f o r e s e t t i n g S R R 0 / 1 * /
mtmsrd r10 ,1
mtspr S R R 0 ,r11
mtspr S R R 1 ,r12
ld r9 ,P A C A _ E X G E N + E X _ R 9 ( r13 )
ld r10 ,P A C A _ E X G E N + E X _ R 1 0 ( r13 )
ld r11 ,P A C A _ E X G E N + E X _ R 1 1 ( r13 )
ld r12 ,P A C A _ E X G E N + E X _ R 1 2 ( r13 )
ld r13 ,P A C A _ E X G E N + E X _ R 1 3 ( r13 )
rfid
b .
slb_miss_fault :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x38 0 , P A C A _ E X G E N )
ld r4 ,P A C A _ E X G E N + E X _ D A R ( r13 )
li r5 ,0
std r4 ,_ D A R ( r1 )
std r5 ,_ D S I S R ( r1 )
b . h a n d l e _ p a g e _ f a u l t
unrecov_user_slb :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x42 0 0 , P A C A _ E X G E N )
DISABLE_ I N T S
bl . s a v e _ n v g p r s
1 : addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . u n r e c o v e r a b l e _ e x c e p t i o n
b 1 b
# endif / * _ _ D I S A B L E D _ _ * /
/ *
* r1 3 p o i n t s t o t h e P A C A , r9 c o n t a i n s t h e s a v e d C R ,
* r1 2 c o n t a i n t h e s a v e d S R R 1 , S R R 0 i s s t i l l r e a d y f o r r e t u r n
* r3 h a s t h e f a u l t i n g a d d r e s s
* r9 - r13 a r e s a v e d i n p a c a - > e x s l b .
* r3 i s s a v e d i n p a c a - > s l b _ r3
* We a s s u m e w e a r e n ' t g o i n g t o t a k e a n y e x c e p t i o n s d u r i n g t h i s p r o c e d u r e .
* /
_ GLOBAL( s l b _ m i s s _ r e a l m o d e )
mflr r10
stw r9 ,P A C A _ E X S L B + E X _ C C R ( r13 ) / * s a v e C R i n e x c . f r a m e * /
std r10 ,P A C A _ E X S L B + E X _ L R ( r13 ) / * s a v e L R * /
bl . s l b _ a l l o c a t e _ r e a l m o d e
/* All done -- return from exception. */
ld r10 ,P A C A _ E X S L B + E X _ L R ( r13 )
ld r3 ,P A C A _ E X S L B + E X _ R 3 ( r13 )
lwz r9 ,P A C A _ E X S L B + E X _ C C R ( r13 ) / * g e t s a v e d C R * /
# ifdef C O N F I G _ P P C _ I S E R I E S
2006-01-13 02:26:42 +03:00
ld r11 ,P A C A L P P A C A P T R ( r13 )
ld r11 ,L P P A C A S R R 0 ( r11 ) / * g e t S R R 0 v a l u e * /
2005-11-07 03:06:55 +03:00
# endif / * C O N F I G _ P P C _ I S E R I E S * /
mtlr r10
andi. r10 ,r12 ,M S R _ R I / * c h e c k f o r u n r e c o v e r a b l e e x c e p t i o n * /
beq- u n r e c o v _ s l b
.machine push
.machine " power4 "
mtcrf 0 x80 ,r9
mtcrf 0 x01 ,r9 / * s l b _ a l l o c a t e u s e s c r0 a n d c r7 * /
.machine pop
# ifdef C O N F I G _ P P C _ I S E R I E S
mtspr S P R N _ S R R 0 ,r11
mtspr S P R N _ S R R 1 ,r12
# endif / * C O N F I G _ P P C _ I S E R I E S * /
ld r9 ,P A C A _ E X S L B + E X _ R 9 ( r13 )
ld r10 ,P A C A _ E X S L B + E X _ R 1 0 ( r13 )
ld r11 ,P A C A _ E X S L B + E X _ R 1 1 ( r13 )
ld r12 ,P A C A _ E X S L B + E X _ R 1 2 ( r13 )
ld r13 ,P A C A _ E X S L B + E X _ R 1 3 ( r13 )
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
unrecov_slb :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x41 0 0 , P A C A _ E X S L B )
DISABLE_ I N T S
bl . s a v e _ n v g p r s
1 : addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . u n r e c o v e r a b l e _ e x c e p t i o n
b 1 b
2005-09-26 10:04:21 +04:00
.align 7
.globl hardware_interrupt_common
.globl hardware_interrupt_entry
hardware_interrupt_common :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x50 0 , P A C A _ E X G E N )
2006-04-18 15:49:11 +04:00
FINISH_ N A P
2005-09-26 10:04:21 +04:00
hardware_interrupt_entry :
DISABLE_ I N T S
2006-02-13 06:48:35 +03:00
bl . p p c64 _ r u n l a t c h _ o n
2005-09-26 10:04:21 +04:00
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . d o _ I R Q
b . r e t _ f r o m _ e x c e p t _ l i t e
2006-04-18 15:49:11 +04:00
# ifdef C O N F I G _ P P C _ 9 7 0 _ N A P
power4_fixup_nap :
andc r9 ,r9 ,r10
std r9 ,T I _ L O C A L _ F L A G S ( r11 )
ld r10 ,_ L I N K ( r1 ) / * m a k e i d l e t a s k d o t h e * /
std r10 ,_ N I P ( r1 ) / * e q u i v a l e n t o f a b l r * /
blr
# endif
2005-09-26 10:04:21 +04:00
.align 7
.globl alignment_common
alignment_common :
2005-10-10 08:01:07 +04:00
mfspr r10 ,S P R N _ D A R
2005-09-26 10:04:21 +04:00
std r10 ,P A C A _ E X G E N + E X _ D A R ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r10 ,S P R N _ D S I S R
2005-09-26 10:04:21 +04:00
stw r10 ,P A C A _ E X G E N + E X _ D S I S R ( r13 )
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x60 0 , P A C A _ E X G E N )
ld r3 ,P A C A _ E X G E N + E X _ D A R ( r13 )
lwz r4 ,P A C A _ E X G E N + E X _ D S I S R ( r13 )
std r3 ,_ D A R ( r1 )
std r4 ,_ D S I S R ( r1 )
bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
ENABLE_ I N T S
bl . a l i g n m e n t _ e x c e p t i o n
b . r e t _ f r o m _ e x c e p t
.align 7
.globl program_check_common
program_check_common :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x70 0 , P A C A _ E X G E N )
bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
ENABLE_ I N T S
bl . p r o g r a m _ c h e c k _ e x c e p t i o n
b . r e t _ f r o m _ e x c e p t
.align 7
.globl fp_unavailable_common
fp_unavailable_common :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x80 0 , P A C A _ E X G E N )
bne . l o a d _ u p _ f p u / * i f f r o m u s e r , j u s t l o a d i t u p * /
bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
ENABLE_ I N T S
bl . k e r n e l _ f p _ u n a v a i l a b l e _ e x c e p t i o n
BUG_ O P C O D E
.align 7
.globl altivec_unavailable_common
altivec_unavailable_common :
EXCEPTION_ P R O L O G _ C O M M O N ( 0 x f20 , P A C A _ E X G E N )
# ifdef C O N F I G _ A L T I V E C
BEGIN_ F T R _ S E C T I O N
bne . l o a d _ u p _ a l t i v e c / * i f f r o m u s e r , j u s t l o a d i t u p * /
END_ F T R _ S E C T I O N _ I F S E T ( C P U _ F T R _ A L T I V E C )
# endif
bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
ENABLE_ I N T S
bl . a l t i v e c _ u n a v a i l a b l e _ e x c e p t i o n
b . r e t _ f r o m _ e x c e p t
# ifdef C O N F I G _ A L T I V E C
/ *
* load_ u p _ a l t i v e c ( u n u s e d , u n u s e d , t s k )
* Disable V M X f o r t h e t a s k w h i c h h a d i t p r e v i o u s l y ,
* and s a v e i t s v e c t o r r e g i s t e r s i n i t s t h r e a d _ s t r u c t .
* Enables t h e V M X f o r u s e i n t h e k e r n e l o n r e t u r n .
* On S M P w e k n o w t h e V M X i s f r e e , s i n c e w e g i v e i t u p e v e r y
* switch ( i e , n o l a z y s a v e o f t h e v e c t o r r e g i s t e r s ) .
* On e n t r y : r13 = = ' c u r r e n t ' & & l a s t _ t a s k _ u s e d _ a l t i v e c ! = ' c u r r e n t '
* /
_ STATIC( l o a d _ u p _ a l t i v e c )
mfmsr r5 / * g r a b t h e c u r r e n t M S R * /
oris r5 ,r5 ,M S R _ V E C @h
mtmsrd r5 / * e n a b l e u s e o f V M X n o w * /
isync
/ *
* For S M P , w e d o n ' t d o l a z y V M X s w i t c h i n g b e c a u s e i t j u s t g e t s t o o
* horrendously c o m p l e x , e s p e c i a l l y w h e n a t a s k s w i t c h e s f r o m o n e C P U
* to a n o t h e r . I n s t e a d w e c a l l g i v e u p _ a l t v e c i n s w i t c h _ t o .
* VRSAVE i s n ' t d e a l t w i t h h e r e , t h a t i s d o n e i n t h e n o r m a l c o n t e x t
* switch c o d e . N o t e t h a t w e c o u l d r e l y o n v r s a v e v a l u e t o e v e n t u a l l y
* avoid s a v i n g a l l o f t h e V R E G s h e r e . . .
* /
# ifndef C O N F I G _ S M P
ld r3 ,l a s t _ t a s k _ u s e d _ a l t i v e c @got(r2)
ld r4 ,0 ( r3 )
cmpdi 0 ,r4 ,0
beq 1 f
/* Save VMX state to last_task_used_altivec's THREAD struct */
addi r4 ,r4 ,T H R E A D
SAVE_ 3 2 V R S ( 0 ,r5 ,r4 )
mfvscr v r0
li r10 ,T H R E A D _ V S C R
stvx v r0 ,r10 ,r4
/* Disable VMX for last_task_used_altivec */
ld r5 ,P T _ R E G S ( r4 )
ld r4 ,_ M S R - S T A C K _ F R A M E _ O V E R H E A D ( r5 )
lis r6 ,M S R _ V E C @h
andc r4 ,r4 ,r6
std r4 ,_ M S R - S T A C K _ F R A M E _ O V E R H E A D ( r5 )
1 :
# endif / * C O N F I G _ S M P * /
/ * Hack : if w e g e t a n a l t i v e c u n a v a i l a b l e t r a p w i t h V R S A V E
* set t o a l l z e r o s , w e a s s u m e t h i s i s a b r o k e n a p p l i c a t i o n
* that f a i l s t o s e t i t p r o p e r l y , a n d t h u s w e s w i t c h i t t o
* all 1 ' s
* /
mfspr r4 ,S P R N _ V R S A V E
cmpdi 0 ,r4 ,0
bne+ 1 f
li r4 ,- 1
mtspr S P R N _ V R S A V E ,r4
1 :
/* enable use of VMX after return */
ld r4 ,P A C A C U R R E N T ( r13 )
addi r5 ,r4 ,T H R E A D / * G e t T H R E A D * /
oris r12 ,r12 ,M S R _ V E C @h
std r12 ,_ M S R ( r1 )
li r4 ,1
li r10 ,T H R E A D _ V S C R
stw r4 ,T H R E A D _ U S E D _ V R ( r5 )
lvx v r0 ,r10 ,r5
mtvscr v r0
REST_ 3 2 V R S ( 0 ,r4 ,r5 )
# ifndef C O N F I G _ S M P
/* Update last_task_used_math to 'current' */
subi r4 ,r5 ,T H R E A D / * B a c k t o ' c u r r e n t ' * /
std r4 ,0 ( r3 )
# endif / * C O N F I G _ S M P * /
/* restore registers and return */
b f a s t _ e x c e p t i o n _ r e t u r n
# endif / * C O N F I G _ A L T I V E C * /
/ *
* Hash t a b l e s t u f f
* /
.align 7
_ GLOBAL( d o _ h a s h _ p a g e )
std r3 ,_ D A R ( r1 )
std r4 ,_ D S I S R ( r1 )
andis. r0 ,r4 ,0 x a45 0 / * w e i r d e r r o r ? * /
bne- . h a n d l e _ p a g e _ f a u l t / * i f n o t , t r y t o i n s e r t a H P T E * /
BEGIN_ F T R _ S E C T I O N
andis. r0 ,r4 ,0 x00 2 0 / * I s i t a s e g m e n t t a b l e f a u l t ? * /
bne- . d o _ s t e _ a l l o c / * I f s o h a n d l e i t * /
END_ F T R _ S E C T I O N _ I F C L R ( C P U _ F T R _ S L B )
/ *
* We n e e d t o s e t t h e _ P A G E _ U S E R b i t i f M S R _ P R i s s e t o r i f w e a r e
* accessing a u s e r s p a c e s e g m e n t ( e v e n f r o m t h e k e r n e l ) . W e a s s u m e
* kernel a d d r e s s e s a l w a y s h a v e t h e h i g h b i t s e t .
* /
rlwinm r4 ,r4 ,3 2 - 2 5 + 9 ,3 1 - 9 ,3 1 - 9 / * D S I S R _ S T O R E - > _ P A G E _ R W * /
rotldi r0 ,r3 ,1 5 / * M o v e h i g h b i t i n t o M S R _ P R p o s n * /
orc r0 ,r12 ,r0 / * M S R _ P R | ~ h i g h _ b i t * /
rlwimi r4 ,r0 ,3 2 - 1 3 ,3 0 ,3 0 / * b e c o m e s _ P A G E _ U S E R a c c e s s b i t * /
ori r4 ,r4 ,1 / * a d d _ P A G E _ P R E S E N T * /
rlwimi r4 ,r5 ,2 2 + 2 ,3 1 - 2 ,3 1 - 2 / * S e t _ P A G E _ E X E C i f t r a p i s 0 x40 0 * /
/ *
* On i S e r i e s , w e s o f t - d i s a b l e i n t e r r u p t s h e r e , t h e n
* hard- e n a b l e i n t e r r u p t s s o t h a t t h e h a s h _ p a g e c o d e c a n s p i n o n
* the h a s h _ t a b l e _ l o c k w i t h o u t p r o b l e m s o n a s h a r e d p r o c e s s o r .
* /
DISABLE_ I N T S
/ *
* r3 c o n t a i n s t h e f a u l t i n g a d d r e s s
* r4 c o n t a i n s t h e r e q u i r e d a c c e s s p e r m i s s i o n s
* r5 c o n t a i n s t h e t r a p n u m b e r
*
* at r e t u r n r3 = 0 f o r s u c c e s s
* /
bl . h a s h _ p a g e / * b u i l d H P T E i f p o s s i b l e * /
cmpdi r3 ,0 / * s e e i f h a s h _ p a g e s u c c e e d e d * /
# ifdef D O _ S O F T _ D I S A B L E
/ *
* If w e h a d i n t e r r u p t s s o f t - e n a b l e d a t t h e p o i n t w h e r e t h e
* DSI/ I S I o c c u r r e d , a n d a n i n t e r r u p t c a m e i n d u r i n g h a s h _ p a g e ,
* handle i t n o w .
* We j u m p t o r e t _ f r o m _ e x c e p t _ l i t e r a t h e r t h a n f a s t _ e x c e p t i o n _ r e t u r n
* because r e t _ f r o m _ e x c e p t _ l i t e w i l l c h e c k f o r a n d h a n d l e p e n d i n g
* interrupts i f n e c e s s a r y .
* /
beq . r e t _ f r o m _ e x c e p t _ l i t e
/* For a hash failure, we don't bother re-enabling interrupts */
ble- 1 2 f
/ *
* hash_ p a g e c o u l d n ' t h a n d l e i t , s e t s o f t i n t e r r u p t e n a b l e b a c k
* to w h a t i t w a s b e f o r e t h e t r a p . N o t e t h a t . l o c a l _ i r q _ r e s t o r e
* handles a n y i n t e r r u p t s p e n d i n g a t t h i s p o i n t .
* /
ld r3 ,S O F T E ( r1 )
bl . l o c a l _ i r q _ r e s t o r e
b 1 1 f
# else
beq f a s t _ e x c e p t i o n _ r e t u r n / * R e t u r n f r o m e x c e p t i o n o n s u c c e s s * /
ble- 1 2 f / * F a i l u r e r e t u r n f r o m h a s h _ p a g e * /
/* fall through */
# endif
/* Here we have a page fault that hash_page can't handle. */
_ GLOBAL( h a n d l e _ p a g e _ f a u l t )
ENABLE_ I N T S
11 : ld r4 ,_ D A R ( r1 )
ld r5 ,_ D S I S R ( r1 )
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . d o _ p a g e _ f a u l t
cmpdi r3 ,0
beq+ . r e t _ f r o m _ e x c e p t _ l i t e
bl . s a v e _ n v g p r s
mr r5 ,r3
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
lwz r4 ,_ D A R ( r1 )
bl . b a d _ p a g e _ f a u l t
b . r e t _ f r o m _ e x c e p t
/ * We h a v e a p a g e f a u l t t h a t h a s h _ p a g e c o u l d h a n d l e b u t H V r e f u s e d
* the P T E i n s e r t i o n
* /
12 : bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
lwz r4 ,_ D A R ( r1 )
bl . l o w _ h a s h _ f a u l t
b . r e t _ f r o m _ e x c e p t
/* here we have a segment miss */
_ GLOBAL( d o _ s t e _ a l l o c )
bl . s t e _ a l l o c a t e / * t r y t o i n s e r t s t a b e n t r y * /
cmpdi r3 ,0
beq+ f a s t _ e x c e p t i o n _ r e t u r n
b . h a n d l e _ p a g e _ f a u l t
/ *
* r1 3 p o i n t s t o t h e P A C A , r9 c o n t a i n s t h e s a v e d C R ,
* r1 1 a n d r12 c o n t a i n t h e s a v e d S R R 0 a n d S R R 1 .
* r9 - r13 a r e s a v e d i n p a c a - > e x s l b .
* We a s s u m e w e a r e n ' t g o i n g t o t a k e a n y e x c e p t i o n s d u r i n g t h i s p r o c e d u r e .
* We a s s u m e ( D A R > > 6 0 ) = = 0 x c .
* /
.align 7
_ GLOBAL( d o _ s t a b _ b o l t e d )
stw r9 ,P A C A _ E X S L B + E X _ C C R ( r13 ) / * s a v e C R i n e x c . f r a m e * /
std r11 ,P A C A _ E X S L B + E X _ S R R 0 ( r13 ) / * s a v e S R R 0 i n e x c . f r a m e * /
/* Hash to the primary group */
ld r10 ,P A C A S T A B V I R T ( r13 )
2005-10-10 08:01:07 +04:00
mfspr r11 ,S P R N _ D A R
2005-09-26 10:04:21 +04:00
srdi r11 ,r11 ,2 8
rldimi r10 ,r11 ,7 ,5 2 / * r10 = f i r s t s t e o f t h e g r o u p * /
/* Calculate VSID */
/* This is a kernel address, so protovsid = ESID */
ASM_ V S I D _ S C R A M B L E ( r11 , r9 )
rldic r9 ,r11 ,1 2 ,1 6 / * r9 = v s i d < < 1 2 * /
/* Search the primary group for a free entry */
1 : ld r11 ,0 ( r10 ) / * T e s t v a l i d b i t o f t h e c u r r e n t s t e * /
andi. r11 ,r11 ,0 x80
beq 2 f
addi r10 ,r10 ,1 6
andi. r11 ,r10 ,0 x70
bne 1 b
/* Stick for only searching the primary group for now. */
/* At least for now, we use a very simple random castout scheme */
/* Use the TB as a random number ; OR in 1 to avoid entry 0 */
mftb r11
rldic r11 ,r11 ,4 ,5 7 / * r11 = ( r11 < < 4 ) & 0 x70 * /
ori r11 ,r11 ,0 x10
/* r10 currently points to an ste one past the group of interest */
/* make it point to the randomly selected entry */
subi r10 ,r10 ,1 2 8
or r10 ,r10 ,r11 / * r10 i s t h e e n t r y t o i n v a l i d a t e * /
isync / * m a r k t h e e n t r y i n v a l i d * /
ld r11 ,0 ( r10 )
rldicl r11 ,r11 ,5 6 ,1 / * c l e a r t h e v a l i d b i t * /
rotldi r11 ,r11 ,8
std r11 ,0 ( r10 )
sync
clrrdi r11 ,r11 ,2 8 / * G e t t h e e s i d p a r t o f t h e s t e * /
slbie r11
2 : std r9 ,8 ( r10 ) / * S t o r e t h e v s i d p a r t o f t h e s t e * /
eieio
2005-10-10 08:01:07 +04:00
mfspr r11 ,S P R N _ D A R / * G e t t h e n e w e s i d * /
2005-09-26 10:04:21 +04:00
clrrdi r11 ,r11 ,2 8 / * P e r m i t s a f u l l 3 2 b o f E S I D * /
ori r11 ,r11 ,0 x90 / * T u r n o n v a l i d a n d k p * /
std r11 ,0 ( r10 ) / * P u t n e w e n t r y b a c k i n t o t h e s t a b * /
sync
/* All done -- return from exception. */
lwz r9 ,P A C A _ E X S L B + E X _ C C R ( r13 ) / * g e t s a v e d C R * /
ld r11 ,P A C A _ E X S L B + E X _ S R R 0 ( r13 ) / * g e t s a v e d S R R 0 * /
andi. r10 ,r12 ,M S R _ R I
beq- u n r e c o v _ s l b
mtcrf 0 x80 ,r9 / * r e s t o r e C R * /
mfmsr r10
clrrdi r10 ,r10 ,2
mtmsrd r10 ,1
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 0 ,r11
mtspr S P R N _ S R R 1 ,r12
2005-09-26 10:04:21 +04:00
ld r9 ,P A C A _ E X S L B + E X _ R 9 ( r13 )
ld r10 ,P A C A _ E X S L B + E X _ R 1 0 ( r13 )
ld r11 ,P A C A _ E X S L B + E X _ R 1 1 ( r13 )
ld r12 ,P A C A _ E X S L B + E X _ R 1 2 ( r13 )
ld r13 ,P A C A _ E X S L B + E X _ R 1 3 ( r13 )
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
/ *
* Space f o r C P U 0 ' s s e g m e n t t a b l e .
*
* On i S e r i e s , t h e h y p e r v i s o r m u s t f i l l i n a t l e a s t o n e e n t r y b e f o r e
* we g e t c o n t r o l ( w i t h r e l o c a t e o n ) . T h e a d d r e s s i s g i v e t o t h e h v
2005-09-29 05:50:22 +04:00
* as a p a g e n u m b e r ( s e e x L p a r M a p i n l p a r d a t a . c ) , s o t h i s m u s t b e a t a
2005-09-26 10:04:21 +04:00
* fixed a d d r e s s ( t h e l i n k e r c a n ' t c o m p u t e ( u 6 4 ) & i n i t i a l _ s t a b > >
* PAGE_ S H I F T ) .
* /
2005-12-06 00:49:00 +03:00
. = STAB0 _ O F F S E T / * 0 x60 0 0 * /
2005-09-26 10:04:21 +04:00
.globl initial_stab
initial_stab :
.space 4096
/ *
* Data a r e a r e s e r v e d f o r F W N M I o p t i o n .
* This a d d r e s s ( 0 x70 0 0 ) i s f i x e d b y t h e R P A .
* /
. = 0 x7 0 0 0
.globl fwnmi_data_area
fwnmi_data_area :
/ * iSeries d o e s n o t u s e t h e F W N M I s t u f f , s o i t i s s a f e t o p u t
* this h e r e , e v e n i f w e l a t e r a l l o w k e r n e l s t h a t w i l l b o o t o n
* both p S e r i e s a n d i S e r i e s * /
# ifdef C O N F I G _ P P C _ I S E R I E S
. = LPARMAP_ P H Y S
# include " l p a r m a p . s "
/ *
* This " . t e x t " i s h e r e f o r o l d c o m p i l e r s t h a t g e n e r a t e a t r a i l i n g
* .note section when c o m p i l i n g . c f i l e s t o . s
* /
.text
# endif / * C O N F I G _ P P C _ I S E R I E S * /
. = 0 x8 0 0 0
/ *
* On p S e r i e s , s e c o n d a r y p r o c e s s o r s s p i n i n t h e f o l l o w i n g c o d e .
* At e n t r y , r3 = t h i s p r o c e s s o r ' s n u m b e r ( p h y s i c a l c p u i d )
* /
_ GLOBAL( p S e r i e s _ s e c o n d a r y _ s m p _ i n i t )
mr r24 ,r3
/* turn on 64-bit mode */
bl . e n a b l e _ 6 4 b _ m o d e
isync
/* Copy some CPU settings from CPU 0 */
bl . _ _ r e s t o r e _ c p u _ s e t u p
/ * Set u p a p a c a v a l u e f o r t h i s p r o c e s s o r . S i n c e w e h a v e t h e
* physical c p u i d i n r24 , w e n e e d t o s e a r c h t h e p a c a s t o f i n d
* which l o g i c a l i d m a p s t o o u r p h y s i c a l o n e .
* /
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r13 , p a c a ) / * G e t b a s e v a d d r o f p a c a a r r a y * /
2005-09-26 10:04:21 +04:00
li r5 ,0 / * l o g i c a l c p u i d * /
1 : lhz r6 ,P A C A H W C P U I D ( r13 ) / * L o a d H W p r o c i d f r o m p a c a * /
cmpw r6 ,r24 / * C o m p a r e t o o u r i d * /
beq 2 f
addi r13 ,r13 ,P A C A _ S I Z E / * L o o p t o n e x t P A C A o n m i s s * /
addi r5 ,r5 ,1
cmpwi r5 ,N R _ C P U S
blt 1 b
mr r3 ,r24 / * n o t f o u n d , c o p y p h y s t o r3 * /
b . k e x e c _ w a i t / * n e x t k e r n e l m i g h t d o b e t t e r * /
2005-10-10 08:01:07 +04:00
2 : mtspr S P R N _ S P R G 3 ,r13 / * S a v e v a d d r o f p a c a i n S P R G 3 * /
2005-09-26 10:04:21 +04:00
/* From now on, r24 is expected to be logical cpuid */
mr r24 ,r5
3 : HMT_ L O W
lbz r23 ,P A C A P R O C S T A R T ( r13 ) / * T e s t i f t h i s p r o c e s s o r s h o u l d * /
/* start. */
sync
/* Create a temp kernel stack for use before relocation is on. */
ld r1 ,P A C A E M E R G S P ( r13 )
subi r1 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
cmpwi 0 ,r23 ,0
# ifdef C O N F I G _ S M P
bne . _ _ s e c o n d a r y _ s t a r t
# endif
b 3 b / * L o o p u n t i l t o l d t o g o * /
# ifdef C O N F I G _ P P C _ I S E R I E S
_ STATIC( _ _ s t a r t _ i n i t i a l i z a t i o n _ i S e r i e s )
/* Clear out the BSS */
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r11 ,_ _ b s s _ s t o p )
LOAD_ R E G _ I M M E D I A T E ( r8 ,_ _ b s s _ s t a r t )
2005-09-26 10:04:21 +04:00
sub r11 ,r11 ,r8 / * b s s s i z e * /
addi r11 ,r11 ,7 / * r o u n d u p t o a n e v e n d o u b l e w o r d * /
rldicl. r11 ,r11 ,6 1 ,3 / * s h i f t r i g h t b y 3 * /
beq 4 f
addi r8 ,r8 ,- 8
li r0 ,0
mtctr r11 / * z e r o t h i s m a n y d o u b l e w o r d s * /
3 : stdu r0 ,8 ( r8 )
bdnz 3 b
4 :
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r1 ,i n i t _ t h r e a d _ u n i o n )
2005-09-26 10:04:21 +04:00
addi r1 ,r1 ,T H R E A D _ S I Z E
li r0 ,0
stdu r0 ,- S T A C K _ F R A M E _ O V E R H E A D ( r1 )
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r3 ,c p u _ s p e c s )
LOAD_ R E G _ I M M E D I A T E ( r4 ,c u r _ c p u _ s p e c )
2005-09-26 10:04:21 +04:00
li r5 ,0
bl . i d e n t i f y _ c p u
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r2 ,_ _ t o c _ s t a r t )
2005-09-26 10:04:21 +04:00
addi r2 ,r2 ,0 x40 0 0
addi r2 ,r2 ,0 x40 0 0
bl . i S e r i e s _ e a r l y _ s e t u p
2005-09-29 05:50:22 +04:00
bl . e a r l y _ s e t u p
2005-09-26 10:04:21 +04:00
/* relocation is on at this point */
b . s t a r t _ h e r e _ c o m m o n
# endif / * C O N F I G _ P P C _ I S E R I E S * /
# ifdef C O N F I G _ P P C _ M U L T I P L A T F O R M
_ STATIC( _ _ m m u _ o f f )
mfmsr r3
andi. r0 ,r3 ,M S R _ I R | M S R _ D R
beqlr
andc r3 ,r3 ,r0
mtspr S P R N _ S R R 0 ,r4
mtspr S P R N _ S R R 1 ,r3
sync
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
/ *
* Here i s o u r m a i n k e r n e l e n t r y p o i n t . W e s u p p o r t c u r r e n t l y 2 k i n d o f e n t r i e s
* depending o n t h e v a l u e o f r5 .
*
* r5 ! = N U L L - > O F e n t r y , w e g o t o p r o m _ i n i t , " l e g a c y " p a r a m e t e r c o n t e n t
* in r3 . . . r7
*
* r5 = = N U L L - > k e x e c s t y l e e n t r y . r3 i s a p h y s i c a l p o i n t e r t o t h e
* DT b l o c k , r4 i s a p h y s i c a l p o i n t e r t o t h e k e r n e l i t s e l f
*
* /
_ GLOBAL( _ _ s t a r t _ i n i t i a l i z a t i o n _ m u l t i p l a t f o r m )
2006-01-09 13:32:42 +03:00
# ifdef C O N F I G _ P P C _ M U L T I P L A T F O R M
2005-09-26 10:04:21 +04:00
/ *
* Are w e b o o t e d f r o m a P R O M O f - t y p e c l i e n t - i n t e r f a c e ?
* /
cmpldi c r0 ,r5 ,0
bne . _ _ b o o t _ f r o m _ p r o m / * y e s - > p r o m * /
2006-01-09 13:32:42 +03:00
# endif
2005-09-26 10:04:21 +04:00
/* Save parameters */
mr r31 ,r3
mr r30 ,r4
/* Make sure we are running in 64 bits mode */
bl . e n a b l e _ 6 4 b _ m o d e
/* Setup some critical 970 SPRs before switching MMU off */
bl . _ _ 9 7 0 _ c p u _ p r e i n i t
/* cpu # */
li r24 ,0
/* Switch off MMU if not already */
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r4 , . _ _ a f t e r _ p r o m _ s t a r t - K E R N E L B A S E )
2005-09-26 10:04:21 +04:00
add r4 ,r4 ,r30
bl . _ _ m m u _ o f f
b . _ _ a f t e r _ p r o m _ s t a r t
2006-01-09 13:32:42 +03:00
# ifdef C O N F I G _ P P C _ M U L T I P L A T F O R M
2005-09-26 10:04:21 +04:00
_ STATIC( _ _ b o o t _ f r o m _ p r o m )
/* Save parameters */
mr r31 ,r3
mr r30 ,r4
mr r29 ,r5
mr r28 ,r6
mr r27 ,r7
[PATCH] correct the comment about stackpointer alignment in __boot_from_prom
The address of variable val in prom_init_stdout is passed to prom_getprop.
prom_getprop casts the pointer to u32 and passes it to call_prom in the hope
that OpenFirmware stores something there.
But the pointer is truncated in the lower bits and the expected value is
stored somewhere else.
In my testing I had a stackpointer of 0x0023e6b4. val was at offset 120,
wich has address 0x0023e72c. But the value passed to OF was 0x0023e728.
c00000000040b710: 3b 01 00 78 addi r24,r1,120
...
c00000000040b754: 57 08 00 38 rlwinm r8,r24,0,0,28
...
c00000000040b784: 80 01 00 78 lwz r0,120(r1)
...
c00000000040b798: 90 1b 00 0c stw r0,12(r27)
...
The stackpointer came from 32bit code.
The chain was yaboot -> zImage -> vmlinux
PowerMac OpenFirmware does appearently not handle the ELF sections
correctly. If yaboot was compiled in
/usr/src/packages/BUILD/lilo-10.1.1/yaboot, then the stackpointer is
unaligned. But the stackpointer is correct if yaboot is compiled in
/tmp/yaboot.
This bug triggered since 2.6.15, now prom_getprop is an inline
function. gcc clears the lower bits, instead of just clearing the
upper 32 bits.
Signed-off-by: Olaf Hering <olh@suse.de>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-03-23 23:50:59 +03:00
/ *
* Align t h e s t a c k t o 1 6 - b y t e b o u n d a r y
* Depending o n t h e s i z e a n d l a y o u t o f t h e E L F s e c t i o n s i n t h e i n i t i a l
* boot b i n a r y , t h e s t a c k p o i n t e r w i l l b e u n a l i g n e t o n P o w e r M a c
* /
2006-03-05 02:00:45 +03:00
rldicr r1 ,r1 ,0 ,5 9
2005-09-26 10:04:21 +04:00
/* Make sure we are running in 64 bits mode */
bl . e n a b l e _ 6 4 b _ m o d e
/* put a relocation offset into r3 */
bl . r e l o c _ o f f s e t
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r2 ,_ _ t o c _ s t a r t )
2005-09-26 10:04:21 +04:00
addi r2 ,r2 ,0 x40 0 0
addi r2 ,r2 ,0 x40 0 0
/* Relocate the TOC from a virt addr to a real addr */
2005-10-10 16:41:25 +04:00
add r2 ,r2 ,r3
2005-09-26 10:04:21 +04:00
/* Restore parameters */
mr r3 ,r31
mr r4 ,r30
mr r5 ,r29
mr r6 ,r28
mr r7 ,r27
/* Do all of the interaction with OF client interface */
bl . p r o m _ i n i t
/* We never return */
trap
2006-01-09 13:32:42 +03:00
# endif
2005-09-26 10:04:21 +04:00
/ *
* At t h i s p o i n t , r3 c o n t a i n s t h e p h y s i c a l a d d r e s s w e a r e r u n n i n g a t ,
* returned b y p r o m _ i n i t ( )
* /
_ STATIC( _ _ a f t e r _ p r o m _ s t a r t )
/ *
2005-12-06 00:49:00 +03:00
* We n e e d t o r u n w i t h _ _ s t a r t a t p h y s i c a l a d d r e s s P H Y S I C A L _ S T A R T .
2005-09-26 10:04:21 +04:00
* This w i l l l e a v e s o m e c o d e i n t h e f i r s t 2 5 6 B o f
* real m e m o r y , w h i c h a r e r e s e r v e d f o r s o f t w a r e u s e .
* The r e m a i n d e r o f t h e f i r s t p a g e i s l o a d e d w i t h t h e f i x e d
* interrupt v e c t o r s . T h e n e x t t w o p a g e s a r e f i l l e d w i t h
* unknown e x c e p t i o n p l a c e h o l d e r s .
*
* Note : This p r o c e s s o v e r w r i t e s t h e O F e x c e p t i o n v e c t o r s .
* r2 6 = = r e l o c a t i o n o f f s e t
* r2 7 = = K E R N E L B A S E
* /
bl . r e l o c _ o f f s e t
mr r26 ,r3
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r27 , K E R N E L B A S E )
2005-09-26 10:04:21 +04:00
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r3 , P H Y S I C A L _ S T A R T ) / * t a r g e t a d d r * /
2005-09-26 10:04:21 +04:00
/ / XXX F I X M E : U s e p h y s r e t u r n e d b y O F ( r30 )
2005-10-10 16:41:25 +04:00
add r4 ,r27 ,r26 / * s o u r c e a d d r * /
2005-09-26 10:04:21 +04:00
/* current address of _start */
/* i.e. where we are running */
/* the source addr */
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r5 ,c o p y _ t o _ h e r e ) / * # b y t e s o f m e m o r y t o c o p y * /
2005-09-26 10:04:21 +04:00
sub r5 ,r5 ,r27
li r6 ,0 x10 0 / * S t a r t o f f s e t , t h e f i r s t 0 x10 0 * /
/* bytes were copied earlier. */
bl . c o p y _ a n d _ f l u s h / * c o p y t h e f i r s t n b y t e s * /
/* this includes the code being */
/* executed here. */
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r0 , 4 f ) / * J u m p t o t h e c o p y o f t h i s c o d e * /
2005-09-26 10:04:21 +04:00
mtctr r0 / * t h a t w e j u s t m a d e / r e l o c a t e d * /
bctr
2006-01-13 06:56:25 +03:00
4 : LOAD_ R E G _ I M M E D I A T E ( r5 ,k l i m i t )
2005-10-10 16:41:25 +04:00
add r5 ,r5 ,r26
2005-09-26 10:04:21 +04:00
ld r5 ,0 ( r5 ) / * g e t t h e v a l u e o f k l i m i t * /
sub r5 ,r5 ,r27
bl . c o p y _ a n d _ f l u s h / * c o p y t h e r e s t * /
b . s t a r t _ h e r e _ m u l t i p l a t f o r m
# endif / * C O N F I G _ P P C _ M U L T I P L A T F O R M * /
/ *
* Copy r o u t i n e u s e d t o c o p y t h e k e r n e l t o s t a r t a t p h y s i c a l a d d r e s s 0
* and f l u s h a n d i n v a l i d a t e t h e c a c h e s a s n e e d e d .
* r3 = d e s t a d d r , r4 = s o u r c e a d d r , r5 = c o p y l i m i t , r6 = s t a r t o f f s e t
* on e x i t , r3 , r4 , r5 a r e u n c h a n g e d , r6 i s u p d a t e d t o b e > = r5 .
*
* Note : this r o u t i n e * o n l y * c l o b b e r s r0 , r6 a n d l r
* /
_ GLOBAL( c o p y _ a n d _ f l u s h )
addi r5 ,r5 ,- 8
addi r6 ,r6 ,- 8
4 : li r0 ,1 6 / * U s e t h e l e a s t c o m m o n * /
/* denominator cache line */
/* size. This results in */
/* extra cache line flushes */
/* but operation is correct. */
/* Can't get cache line size */
/* from NACA as it is being */
/* moved too. */
mtctr r0 / * p u t # w o r d s / l i n e i n c t r * /
3 : addi r6 ,r6 ,8 / * c o p y a c a c h e l i n e * /
ldx r0 ,r6 ,r4
stdx r0 ,r6 ,r3
bdnz 3 b
dcbst r6 ,r3 / * w r i t e i t t o m e m o r y * /
sync
icbi r6 ,r3 / * f l u s h t h e i c a c h e l i n e * /
cmpld 0 ,r6 ,r5
blt 4 b
sync
addi r5 ,r5 ,8
addi r6 ,r6 ,8
blr
.align 8
copy_to_here :
# ifdef C O N F I G _ S M P
# ifdef C O N F I G _ P P C _ P M A C
/ *
* On P o w e r M a c , s e c o n d a r y p r o c e s s o r s s t a r t s f r o m t h e r e s e t v e c t o r , w h i c h
* is t e m p o r a r i l y t u r n e d i n t o a c a l l t o o n e o f t h e f u n c t i o n s b e l o w .
* /
.section " .text " ;
.align 2 ;
2005-10-22 10:02:39 +04:00
.globl __secondary_start_pmac_0
__secondary_start_pmac_0 :
/* NB the entries for cpus 0, 1, 2 must each occupy 8 bytes. */
li r24 ,0
b 1 f
li r24 ,1
b 1 f
li r24 ,2
b 1 f
li r24 ,3
1 :
2005-09-26 10:04:21 +04:00
_ GLOBAL( p m a c _ s e c o n d a r y _ s t a r t )
/* turn on 64-bit mode */
bl . e n a b l e _ 6 4 b _ m o d e
isync
/* Copy some CPU settings from CPU 0 */
bl . _ _ r e s t o r e _ c p u _ s e t u p
/* pSeries do that early though I don't think we really need it */
mfmsr r3
ori r3 ,r3 ,M S R _ R I
mtmsrd r3 / * R I o n * /
/* Set up a paca value for this processor. */
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r4 , p a c a ) / * G e t b a s e v a d d r o f p a c a a r r a y * /
2005-09-26 10:04:21 +04:00
mulli r13 ,r24 ,P A C A _ S I Z E / * C a l c u l a t e v a d d r o f r i g h t p a c a * /
add r13 ,r13 ,r4 / * f o r t h i s p r o c e s s o r . * /
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 3 ,r13 / * S a v e v a d d r o f p a c a i n S P R G 3 * /
2005-09-26 10:04:21 +04:00
/* Create a temp kernel stack for use before relocation is on. */
ld r1 ,P A C A E M E R G S P ( r13 )
subi r1 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
b . _ _ s e c o n d a r y _ s t a r t
# endif / * C O N F I G _ P P C _ P M A C * /
/ *
* This f u n c t i o n i s c a l l e d a f t e r t h e m a s t e r C P U h a s r e l e a s e d t h e
* secondary p r o c e s s o r s . T h e e x e c u t i o n e n v i r o n m e n t i s r e l o c a t i o n o f f .
* The p a c a f o r t h i s p r o c e s s o r h a s t h e f o l l o w i n g f i e l d s i n i t i a l i z e d a t
* this p o i n t :
* 1 . Processor n u m b e r
* 2 . Segment t a b l e p o i n t e r ( v i r t u a l a d d r e s s )
* On e n t r y t h e f o l l o w i n g a r e s e t :
* r1 = s t a c k p o i n t e r . v a d d r f o r i S e r i e s , r a d d r ( t e m p s t a c k ) f o r p S e r i e s
* r2 4 = c p u # ( i n L i n u x t e r m s )
* r1 3 = p a c a v i r t u a l a d d r e s s
* SPRG3 = p a c a v i r t u a l a d d r e s s
* /
_ GLOBAL( _ _ s e c o n d a r y _ s t a r t )
2005-11-10 05:37:51 +03:00
/* Set thread priority to MEDIUM */
HMT_ M E D I U M
2005-09-26 10:04:21 +04:00
2005-11-10 05:37:51 +03:00
/* Load TOC */
2005-09-26 10:04:21 +04:00
ld r2 ,P A C A T O C ( r13 )
2005-11-10 05:37:51 +03:00
/* Do early setup for that CPU (stab, slb, hash table pointer) */
bl . e a r l y _ s e t u p _ s e c o n d a r y
2005-09-26 10:04:21 +04:00
/* Initialize the kernel stack. Just a repeat for iSeries. */
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ A D D R ( r3 , c u r r e n t _ s e t )
2005-09-26 10:04:21 +04:00
sldi r28 ,r24 ,3 / * g e t c u r r e n t _ s e t [ c p u #] * /
ldx r1 ,r3 ,r28
addi r1 ,r1 ,T H R E A D _ S I Z E - S T A C K _ F R A M E _ O V E R H E A D
std r1 ,P A C A K S A V E ( r13 )
2005-11-10 05:37:51 +03:00
/* Clear backchain so we get nice backtraces */
2005-09-26 10:04:21 +04:00
li r7 ,0
mtlr r7
/* enable MMU and jump to start_secondary */
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ A D D R ( r3 , . s t a r t _ s e c o n d a r y _ p r o l o g )
LOAD_ R E G _ I M M E D I A T E ( r4 , M S R _ K E R N E L )
2005-09-26 10:04:21 +04:00
# ifdef D O _ S O F T _ D I S A B L E
ori r4 ,r4 ,M S R _ E E
# endif
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 0 ,r3
mtspr S P R N _ S R R 1 ,r4
2005-09-26 10:04:21 +04:00
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
/ *
* Running w i t h r e l o c a t i o n o n a t t h i s p o i n t . A l l w e w a n t t o d o i s
* zero t h e s t a c k b a c k - c h a i n p o i n t e r b e f o r e g o i n g i n t o C c o d e .
* /
_ GLOBAL( s t a r t _ s e c o n d a r y _ p r o l o g )
li r3 ,0
std r3 ,0 ( r1 ) / * Z e r o t h e s t a c k f r a m e p o i n t e r * /
bl . s t a r t _ s e c o n d a r y
2005-11-10 05:37:51 +03:00
b .
2005-09-26 10:04:21 +04:00
# endif
/ *
* This s u b r o u t i n e c l o b b e r s r11 a n d r12
* /
_ GLOBAL( e n a b l e _ 6 4 b _ m o d e )
mfmsr r11 / * g r a b t h e c u r r e n t M S R * /
li r12 ,1
rldicr r12 ,r12 ,M S R _ S F _ L G ,( 6 3 - M S R _ S F _ L G )
or r11 ,r11 ,r12
li r12 ,1
rldicr r12 ,r12 ,M S R _ I S F _ L G ,( 6 3 - M S R _ I S F _ L G )
or r11 ,r11 ,r12
mtmsrd r11
isync
blr
# ifdef C O N F I G _ P P C _ M U L T I P L A T F O R M
/ *
* This i s w h e r e t h e m a i n k e r n e l c o d e s t a r t s .
* /
_ STATIC( s t a r t _ h e r e _ m u l t i p l a t f o r m )
/* get a new offset, now that the kernel has moved. */
bl . r e l o c _ o f f s e t
mr r26 ,r3
/ * Clear o u t t h e B S S . I t m a y h a v e b e e n d o n e i n p r o m _ i n i t ,
* already b u t t h a t ' s i r r e l e v a n t s i n c e p r o m _ i n i t w i l l s o o n
* be d e t a c h e d f r o m t h e k e r n e l c o m p l e t e l y . B e s i d e s , w e n e e d
* to c l e a r i t n o w f o r k e x e c - s t y l e e n t r y .
* /
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r11 ,_ _ b s s _ s t o p )
LOAD_ R E G _ I M M E D I A T E ( r8 ,_ _ b s s _ s t a r t )
2005-09-26 10:04:21 +04:00
sub r11 ,r11 ,r8 / * b s s s i z e * /
addi r11 ,r11 ,7 / * r o u n d u p t o a n e v e n d o u b l e w o r d * /
rldicl. r11 ,r11 ,6 1 ,3 / * s h i f t r i g h t b y 3 * /
beq 4 f
addi r8 ,r8 ,- 8
li r0 ,0
mtctr r11 / * z e r o t h i s m a n y d o u b l e w o r d s * /
3 : stdu r0 ,8 ( r8 )
bdnz 3 b
4 :
mfmsr r6
ori r6 ,r6 ,M S R _ R I
mtmsrd r6 / * R I o n * /
/* The following gets the stack and TOC set up with the regs */
/* pointing to the real addr of the kernel stack. This is */
/* all done to support the C function call below which sets */
/* up the htab. This is done because we have relocated the */
/* kernel but are still running in real mode. */
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r3 ,i n i t _ t h r e a d _ u n i o n )
2005-10-10 16:41:25 +04:00
add r3 ,r3 ,r26
2005-09-26 10:04:21 +04:00
/* set up a stack pointer (physical address) */
addi r1 ,r3 ,T H R E A D _ S I Z E
li r0 ,0
stdu r0 ,- S T A C K _ F R A M E _ O V E R H E A D ( r1 )
/* set up the TOC (physical address) */
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r2 ,_ _ t o c _ s t a r t )
2005-09-26 10:04:21 +04:00
addi r2 ,r2 ,0 x40 0 0
addi r2 ,r2 ,0 x40 0 0
2005-10-10 16:41:25 +04:00
add r2 ,r2 ,r26
2005-09-26 10:04:21 +04:00
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r3 , c p u _ s p e c s )
2005-10-10 16:41:25 +04:00
add r3 ,r3 ,r26
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r4 ,c u r _ c p u _ s p e c )
2005-10-10 16:41:25 +04:00
add r4 ,r4 ,r26
2005-09-26 10:04:21 +04:00
mr r5 ,r26
bl . i d e n t i f y _ c p u
/ * Save s o m e l o w l e v e l c o n f i g H I D s o f C P U 0 t o b e c o p i e d t o
* other C P U s l a t e r o n , o r u s e d f o r s u s p e n d / r e s u m e
* /
bl . _ _ s a v e _ c p u _ s e t u p
sync
/ * Do v e r y e a r l y k e r n e l i n i t i a l i z a t i o n s , i n c l u d i n g i n i t i a l h a s h t a b l e ,
* stab a n d s l b s e t u p b e f o r e w e t u r n o n r e l o c a t i o n . * /
/* Restore parameters passed from prom_init/kexec */
mr r3 ,r31
bl . e a r l y _ s e t u p
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r3 , . s t a r t _ h e r e _ c o m m o n )
LOAD_ R E G _ I M M E D I A T E ( r4 , M S R _ K E R N E L )
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S R R 0 ,r3
mtspr S P R N _ S R R 1 ,r4
2005-09-26 10:04:21 +04:00
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
# endif / * C O N F I G _ P P C _ M U L T I P L A T F O R M * /
/* This is where all platforms converge execution */
_ STATIC( s t a r t _ h e r e _ c o m m o n )
/* relocation is on at this point */
/* The following code sets up the SP and TOC now that we are */
/* running with translation enabled. */
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r3 ,i n i t _ t h r e a d _ u n i o n )
2005-09-26 10:04:21 +04:00
/* set up the stack */
addi r1 ,r3 ,T H R E A D _ S I Z E
li r0 ,0
stdu r0 ,- S T A C K _ F R A M E _ O V E R H E A D ( r1 )
/ * Apply t h e C P U s - s p e c i f i c f i x u p s ( n o p o u t s e c t i o n s n o t r e l e v a n t
* to t h i s C P U
* /
li r3 ,0
bl . d o _ c p u _ f t r _ f i x u p s
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r26 , b o o t _ c p u i d )
2005-09-26 10:04:21 +04:00
lwz r26 ,0 ( r26 )
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r24 , p a c a ) / * G e t b a s e v a d d r o f p a c a a r r a y * /
2005-09-26 10:04:21 +04:00
mulli r13 ,r26 ,P A C A _ S I Z E / * C a l c u l a t e v a d d r o f r i g h t p a c a * /
add r13 ,r13 ,r24 / * f o r t h i s p r o c e s s o r . * /
2005-10-10 08:01:07 +04:00
mtspr S P R N _ S P R G 3 ,r13
2005-09-26 10:04:21 +04:00
/* ptr to current */
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r4 , i n i t _ t a s k )
2005-09-26 10:04:21 +04:00
std r4 ,P A C A C U R R E N T ( r13 )
/* Load the TOC */
ld r2 ,P A C A T O C ( r13 )
std r1 ,P A C A K S A V E ( r13 )
bl . s e t u p _ s y s t e m
/* Load up the kernel context */
5 :
# ifdef D O _ S O F T _ D I S A B L E
li r5 ,0
stb r5 ,P A C A P R O C E N A B L E D ( r13 ) / * S o f t D i s a b l e d * /
mfmsr r5
ori r5 ,r5 ,M S R _ E E / * H a r d E n a b l e d * /
mtmsrd r5
# endif
bl . s t a r t _ k e r n e l
2006-02-13 10:11:13 +03:00
/* Not reached */
BUG_ O P C O D E
2005-09-26 10:04:21 +04:00
2006-03-25 09:25:17 +03:00
/* Put the paca pointer into r13 and SPRG3 */
_ GLOBAL( s e t u p _ b o o t _ p a c a )
LOAD_ R E G _ I M M E D I A T E ( r3 , b o o t _ c p u i d )
lwz r3 ,0 ( r3 )
LOAD_ R E G _ I M M E D I A T E ( r4 , p a c a ) / * G e t b a s e v a d d r o f p a c a a r r a y * /
mulli r3 ,r3 ,P A C A _ S I Z E / * C a l c u l a t e v a d d r o f r i g h t p a c a * /
add r13 ,r3 ,r4 / * f o r t h i s p r o c e s s o r . * /
mtspr S P R N _ S P R G 3 ,r13
blr
2005-09-26 10:04:21 +04:00
/ *
* We p u t a f e w t h i n g s h e r e t h a t h a v e t o b e p a g e - a l i g n e d .
* This s t u f f g o e s a t t h e b e g i n n i n g o f t h e b s s , w h i c h i s p a g e - a l i g n e d .
* /
.section " .bss "
.align PAGE_SHIFT
.globl empty_zero_page
empty_zero_page :
.space PAGE_SIZE
.globl swapper_pg_dir
swapper_pg_dir :
.space PAGE_SIZE
/ *
* This s p a c e g e t s a c o p y o f o p t i o n a l i n f o p a s s e d t o u s b y t h e b o o t s t r a p
* Used t o p a s s p a r a m e t e r s i n t o t h e k e r n e l l i k e r o o t = / d e v / s d a1 , e t c .
* /
.globl cmd_line
cmd_line :
.space COMMAND_LINE_SIZE
