2005-10-10 16:36:14 +04:00
/ *
* PowerPC v e r s i o n
* Copyright ( C ) 1 9 9 5 - 1 9 9 6 G a r y T h o m a s ( g d t @linuxppc.org)
* Rewritten b y C o r t D o u g a n ( c o r t @cs.nmt.edu) for PReP
* Copyright ( C ) 1 9 9 6 C o r t D o u g a n < c o r t @cs.nmt.edu>
* Adapted f o r P o w e r M a c i n t o s h b y P a u l M a c k e r r a s .
* Low- l e v e l e x c e p t i o n h a n d l e r s a n d M M U s u p p o r t
* rewritten b y P a u l M a c k e r r a s .
* Copyright ( C ) 1 9 9 6 P a u l M a c k e r r a s .
* MPC8 x x m o d i f i c a t i o n s C o p y r i g h t ( C ) 1 9 9 7 D a n M a l e k ( d m a l e k @jlc.net).
*
* This f i l e c o n t a i n s t h e s y s t e m c a l l e n t r y c o d e , c o n t e x t s w i t c h
* code, a n d e x c e p t i o n / i n t e r r u p t r e t u r n c o d e f o r P o w e r P C .
*
* This p r o g r a m i s f r e e s o f t w a r e ; you can redistribute it and/or
* modify i t u n d e r t h e t e r m s o f t h e G N U G e n e r a l P u b l i c L i c e n s e
* as p u b l i s h e d b y t h e F r e e S o f t w a r e F o u n d a t i o n ; either version
* 2 of t h e L i c e n s e , o r ( a t y o u r o p t i o n ) a n y l a t e r v e r s i o n .
* /
# include < l i n u x / e r r n o . h >
# include < a s m / u n i s t d . h >
# include < a s m / p r o c e s s o r . h >
# include < a s m / p a g e . h >
# include < a s m / m m u . h >
# include < a s m / t h r e a d _ i n f o . h >
# include < a s m / p p c _ a s m . h >
# include < a s m / a s m - o f f s e t s . h >
# include < a s m / c p u t a b l e . h >
2006-09-25 12:19:00 +04:00
# include < a s m / f i r m w a r e . h >
2007-01-01 21:45:34 +03:00
# include < a s m / b u g . h >
2005-10-10 16:36:14 +04:00
/ *
* System c a l l s .
* /
.section " .toc " , " aw"
.SYS_CALL_TABLE :
.tc .sys_call_table [ TC] ,. s y s _ c a l l _ t a b l e
/* This value is used to mark exception frames on the stack. */
exception_marker :
.tc _ 7 2 6 5 6 7 7 3 _ 6 8 6 5 7 2 6 5 [ T C ] ,0 x72 6 5 6 7 7 3 6 8 6 5 7 2 6 5
.section " .text "
.align 7
# undef S H O W _ S Y S C A L L S
.globl system_call_common
system_call_common :
andi. r10 ,r12 ,M S R _ P R
mr r10 ,r1
addi r1 ,r1 ,- I N T _ F R A M E _ S I Z E
beq- 1 f
ld r1 ,P A C A K S A V E ( r13 )
1 : std r10 ,0 ( r1 )
2006-06-10 19:15:55 +04:00
crclr s o
2005-10-10 16:36:14 +04:00
std r11 ,_ N I P ( r1 )
std r12 ,_ M S R ( r1 )
std r0 ,G P R 0 ( r1 )
std r10 ,G P R 1 ( r1 )
powerpc: Implement accurate task and CPU time accounting
This implements accurate task and cpu time accounting for 64-bit
powerpc kernels. Instead of accounting a whole jiffy of time to a
task on a timer interrupt because that task happened to be running at
the time, we now account time in units of timebase ticks according to
the actual time spent by the task in user mode and kernel mode. We
also count the time spent processing hardware and software interrupts
accurately. This is conditional on CONFIG_VIRT_CPU_ACCOUNTING. If
that is not set, we do tick-based approximate accounting as before.
To get this accurate information, we read either the PURR (processor
utilization of resources register) on POWER5 machines, or the timebase
on other machines on
* each entry to the kernel from usermode
* each exit to usermode
* transitions between process context, hard irq context and soft irq
context in kernel mode
* context switches.
On POWER5 systems with shared-processor logical partitioning we also
read both the PURR and the timebase at each timer interrupt and
context switch in order to determine how much time has been taken by
the hypervisor to run other partitions ("steal" time). Unfortunately,
since we need values of the PURR on both threads at the same time to
accurately calculate the steal time, and since we can only calculate
steal time on a per-core basis, the apportioning of the steal time
between idle time (time which we ceded to the hypervisor in the idle
loop) and actual stolen time is somewhat approximate at the moment.
This is all based quite heavily on what s390 does, and it uses the
generic interfaces that were added by the s390 developers,
i.e. account_system_time(), account_user_time(), etc.
This patch doesn't add any new interfaces between the kernel and
userspace, and doesn't change the units in which time is reported to
userspace by things such as /proc/stat, /proc/<pid>/stat, getrusage(),
times(), etc. Internally the various task and cpu times are stored in
timebase units, but they are converted to USER_HZ units (1/100th of a
second) when reported to userspace. Some precision is therefore lost
but there should not be any accumulating error, since the internal
accumulation is at full precision.
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-02-24 02:06:59 +03:00
ACCOUNT_ C P U _ U S E R _ E N T R Y ( r10 , r11 )
2005-10-10 16:36:14 +04:00
std r2 ,G P R 2 ( r1 )
std r3 ,G P R 3 ( r1 )
std r4 ,G P R 4 ( r1 )
std r5 ,G P R 5 ( r1 )
std r6 ,G P R 6 ( r1 )
std r7 ,G P R 7 ( r1 )
std r8 ,G P R 8 ( r1 )
li r11 ,0
std r11 ,G P R 9 ( r1 )
std r11 ,G P R 1 0 ( r1 )
std r11 ,G P R 1 1 ( r1 )
std r11 ,G P R 1 2 ( r1 )
std r9 ,G P R 1 3 ( r1 )
mfcr r9
mflr r10
li r11 ,0 x c01
std r9 ,_ C C R ( r1 )
std r10 ,_ L I N K ( r1 )
std r11 ,_ T R A P ( r1 )
mfxer r9
mfctr r10
std r9 ,_ X E R ( r1 )
std r10 ,_ C T R ( r1 )
std r3 ,O R I G _ G P R 3 ( r1 )
ld r2 ,P A C A T O C ( r13 )
addi r9 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
ld r11 ,e x c e p t i o n _ m a r k e r @toc(r2)
std r11 ,- 1 6 ( r9 ) / * " r e g s h e r e " m a r k e r * /
[POWERPC] Lazy interrupt disabling for 64-bit machines
This implements a lazy strategy for disabling interrupts. This means
that local_irq_disable() et al. just clear the 'interrupts are
enabled' flag in the paca. If an interrupt comes along, the interrupt
entry code notices that interrupts are supposed to be disabled, and
clears the EE bit in SRR1, clears the 'interrupts are hard-enabled'
flag in the paca, and returns. This means that interrupts only
actually get disabled in the processor when an interrupt comes along.
When interrupts are enabled by local_irq_enable() et al., the code
sets the interrupts-enabled flag in the paca, and then checks whether
interrupts got hard-disabled. If so, it also sets the EE bit in the
MSR to hard-enable the interrupts.
This has the potential to improve performance, and also makes it
easier to make a kernel that can boot on iSeries and on other 64-bit
machines, since this lazy-disable strategy is very similar to the
soft-disable strategy that iSeries already uses.
This version renames paca->proc_enabled to paca->soft_enabled, and
changes a couple of soft-disables in the kexec code to hard-disables,
which should fix the crash that Michael Ellerman saw. This doesn't
yet use a reserved CR field for the soft_enabled and hard_enabled
flags. This applies on top of Stephen Rothwell's patches to make it
possible to build a combined iSeries/other kernel.
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-10-04 10:47:49 +04:00
li r10 ,1
stb r10 ,P A C A S O F T I R Q E N ( r13 )
stb r10 ,P A C A H A R D I R Q E N ( r13 )
std r10 ,S O F T E ( r1 )
2005-10-10 16:36:14 +04:00
# ifdef C O N F I G _ P P C _ I S E R I E S
2006-09-25 12:19:00 +04:00
BEGIN_ F W _ F T R _ S E C T I O N
2005-10-10 16:36:14 +04:00
/* Hack for handling interrupts when soft-enabling on iSeries */
cmpdi c r1 ,r0 ,0 x55 5 5 / * s y s c a l l 0 x55 5 5 * /
andi. r10 ,r12 ,M S R _ P R / * f r o m k e r n e l * /
crand 4 * c r0 + e q ,4 * c r1 + e q ,4 * c r0 + e q
2006-11-27 06:59:50 +03:00
bne 2 f
b h a r d w a r e _ i n t e r r u p t _ e n t r y
2 :
2006-09-25 12:19:00 +04:00
END_ F W _ F T R _ S E C T I O N _ I F S E T ( F W _ F E A T U R E _ I S E R I E S )
2005-10-10 16:36:14 +04:00
# endif
mfmsr r11
ori r11 ,r11 ,M S R _ E E
mtmsrd r11 ,1
# ifdef S H O W _ S Y S C A L L S
bl . d o _ s h o w _ s y s c a l l
REST_ G P R ( 0 ,r1 )
REST_ 4 G P R S ( 3 ,r1 )
REST_ 2 G P R S ( 7 ,r1 )
addi r9 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
# endif
clrrdi r11 ,r1 ,T H R E A D _ S H I F T
ld r10 ,T I _ F L A G S ( r11 )
andi. r11 ,r10 ,_ T I F _ S Y S C A L L _ T _ O R _ A
bne- s y s c a l l _ d o t r a c e
syscall_dotrace_cont :
cmpldi 0 ,r0 ,N R _ s y s c a l l s
bge- s y s c a l l _ e n o s y s
system_call : /* label this so stack traces look sane */
/ *
* Need t o v e c t o r t o 3 2 B i t o r d e f a u l t s y s _ c a l l _ t a b l e h e r e ,
* based o n c a l l e r ' s r u n - m o d e / p e r s o n a l i t y .
* /
ld r11 ,. S Y S _ C A L L _ T A B L E @toc(2)
andi. r10 ,r10 ,_ T I F _ 3 2 B I T
beq 1 5 f
addi r11 ,r11 ,8 / * u s e 3 2 - b i t s y s c a l l e n t r i e s * /
clrldi r3 ,r3 ,3 2
clrldi r4 ,r4 ,3 2
clrldi r5 ,r5 ,3 2
clrldi r6 ,r6 ,3 2
clrldi r7 ,r7 ,3 2
clrldi r8 ,r8 ,3 2
15 :
slwi r0 ,r0 ,4
ldx r10 ,r11 ,r0 / * F e t c h s y s t e m c a l l h a n d l e r [ p t r ] * /
mtctr r10
bctrl / * C a l l h a n d l e r * /
syscall_exit :
[PATCH] syscall entry/exit revamp
This cleanup patch speeds up the null syscall path on ppc64 by about 3%,
and brings the ppc32 and ppc64 code slightly closer together.
The ppc64 code was checking current_thread_info()->flags twice in the
syscall exit path; once for TIF_SYSCALL_T_OR_A before disabling
interrupts, and then again for TIF_SIGPENDING|TIF_NEED_RESCHED etc after
disabling interrupts. Now we do the same as ppc32 -- check the flags
only once in the fast path, and re-enable interrupts if necessary in the
ptrace case.
The patch abolishes the 'syscall_noerror' member of struct thread_info
and replaces it with a TIF_NOERROR bit in the flags, which is handled in
the slow path. This shortens the syscall entry code, which no longer
needs to clear syscall_noerror.
The patch adds a TIF_SAVE_NVGPRS flag which causes the syscall exit slow
path to save the non-volatile GPRs into a signal frame. This removes the
need for the assembly wrappers around sys_sigsuspend(),
sys_rt_sigsuspend(), et al which existed solely to save those registers
in advance. It also means I don't have to add new wrappers for ppoll()
and pselect(), which is what I was supposed to be doing when I got
distracted into this...
Finally, it unifies the ppc64 and ppc32 methods of handling syscall exit
directly into a signal handler (as required by sigsuspend et al) by
introducing a TIF_RESTOREALL flag which causes _all_ the registers to be
reloaded from the pt_regs by taking the ret_from_exception path, instead
of the normal syscall exit path which stomps on the callee-saved GPRs.
It appears to pass an LTP test run on ppc64, and passes basic testing on
ppc32 too. Brief tests of ptrace functionality with strace and gdb also
appear OK. I wouldn't send it to Linus for 2.6.15 just yet though :)
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-15 21:52:18 +03:00
std r3 ,R E S U L T ( r1 )
2005-10-10 16:36:14 +04:00
# ifdef S H O W _ S Y S C A L L S
bl . d o _ s h o w _ s y s c a l l _ e x i t
[PATCH] syscall entry/exit revamp
This cleanup patch speeds up the null syscall path on ppc64 by about 3%,
and brings the ppc32 and ppc64 code slightly closer together.
The ppc64 code was checking current_thread_info()->flags twice in the
syscall exit path; once for TIF_SYSCALL_T_OR_A before disabling
interrupts, and then again for TIF_SIGPENDING|TIF_NEED_RESCHED etc after
disabling interrupts. Now we do the same as ppc32 -- check the flags
only once in the fast path, and re-enable interrupts if necessary in the
ptrace case.
The patch abolishes the 'syscall_noerror' member of struct thread_info
and replaces it with a TIF_NOERROR bit in the flags, which is handled in
the slow path. This shortens the syscall entry code, which no longer
needs to clear syscall_noerror.
The patch adds a TIF_SAVE_NVGPRS flag which causes the syscall exit slow
path to save the non-volatile GPRs into a signal frame. This removes the
need for the assembly wrappers around sys_sigsuspend(),
sys_rt_sigsuspend(), et al which existed solely to save those registers
in advance. It also means I don't have to add new wrappers for ppoll()
and pselect(), which is what I was supposed to be doing when I got
distracted into this...
Finally, it unifies the ppc64 and ppc32 methods of handling syscall exit
directly into a signal handler (as required by sigsuspend et al) by
introducing a TIF_RESTOREALL flag which causes _all_ the registers to be
reloaded from the pt_regs by taking the ret_from_exception path, instead
of the normal syscall exit path which stomps on the callee-saved GPRs.
It appears to pass an LTP test run on ppc64, and passes basic testing on
ppc32 too. Brief tests of ptrace functionality with strace and gdb also
appear OK. I wouldn't send it to Linus for 2.6.15 just yet though :)
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-15 21:52:18 +03:00
ld r3 ,R E S U L T ( r1 )
2005-10-10 16:36:14 +04:00
# endif
clrrdi r12 ,r1 ,T H R E A D _ S H I F T
/ * disable i n t e r r u p t s s o c u r r e n t _ t h r e a d _ i n f o ( ) - > f l a g s c a n ' t c h a n g e ,
and s o t h a t w e d o n ' t g e t i n t e r r u p t e d a f t e r l o a d i n g S R R 0 / 1 . * /
ld r8 ,_ M S R ( r1 )
andi. r10 ,r8 ,M S R _ R I
beq- u n r e c o v _ r e s t o r e
mfmsr r10
rldicl r10 ,r10 ,4 8 ,1
rotldi r10 ,r10 ,1 6
mtmsrd r10 ,1
ld r9 ,T I _ F L A G S ( r12 )
[PATCH] syscall entry/exit revamp
This cleanup patch speeds up the null syscall path on ppc64 by about 3%,
and brings the ppc32 and ppc64 code slightly closer together.
The ppc64 code was checking current_thread_info()->flags twice in the
syscall exit path; once for TIF_SYSCALL_T_OR_A before disabling
interrupts, and then again for TIF_SIGPENDING|TIF_NEED_RESCHED etc after
disabling interrupts. Now we do the same as ppc32 -- check the flags
only once in the fast path, and re-enable interrupts if necessary in the
ptrace case.
The patch abolishes the 'syscall_noerror' member of struct thread_info
and replaces it with a TIF_NOERROR bit in the flags, which is handled in
the slow path. This shortens the syscall entry code, which no longer
needs to clear syscall_noerror.
The patch adds a TIF_SAVE_NVGPRS flag which causes the syscall exit slow
path to save the non-volatile GPRs into a signal frame. This removes the
need for the assembly wrappers around sys_sigsuspend(),
sys_rt_sigsuspend(), et al which existed solely to save those registers
in advance. It also means I don't have to add new wrappers for ppoll()
and pselect(), which is what I was supposed to be doing when I got
distracted into this...
Finally, it unifies the ppc64 and ppc32 methods of handling syscall exit
directly into a signal handler (as required by sigsuspend et al) by
introducing a TIF_RESTOREALL flag which causes _all_ the registers to be
reloaded from the pt_regs by taking the ret_from_exception path, instead
of the normal syscall exit path which stomps on the callee-saved GPRs.
It appears to pass an LTP test run on ppc64, and passes basic testing on
ppc32 too. Brief tests of ptrace functionality with strace and gdb also
appear OK. I wouldn't send it to Linus for 2.6.15 just yet though :)
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-15 21:52:18 +03:00
li r11 ,- _ L A S T _ E R R N O
2006-03-08 05:24:22 +03:00
andi. r0 ,r9 ,( _ T I F _ S Y S C A L L _ T _ O R _ A | _ T I F _ S I N G L E S T E P | _ T I F _ U S E R _ W O R K _ M A S K | _ T I F _ P E R S Y S C A L L _ M A S K )
2005-10-10 16:36:14 +04:00
bne- s y s c a l l _ e x i t _ w o r k
[PATCH] syscall entry/exit revamp
This cleanup patch speeds up the null syscall path on ppc64 by about 3%,
and brings the ppc32 and ppc64 code slightly closer together.
The ppc64 code was checking current_thread_info()->flags twice in the
syscall exit path; once for TIF_SYSCALL_T_OR_A before disabling
interrupts, and then again for TIF_SIGPENDING|TIF_NEED_RESCHED etc after
disabling interrupts. Now we do the same as ppc32 -- check the flags
only once in the fast path, and re-enable interrupts if necessary in the
ptrace case.
The patch abolishes the 'syscall_noerror' member of struct thread_info
and replaces it with a TIF_NOERROR bit in the flags, which is handled in
the slow path. This shortens the syscall entry code, which no longer
needs to clear syscall_noerror.
The patch adds a TIF_SAVE_NVGPRS flag which causes the syscall exit slow
path to save the non-volatile GPRs into a signal frame. This removes the
need for the assembly wrappers around sys_sigsuspend(),
sys_rt_sigsuspend(), et al which existed solely to save those registers
in advance. It also means I don't have to add new wrappers for ppoll()
and pselect(), which is what I was supposed to be doing when I got
distracted into this...
Finally, it unifies the ppc64 and ppc32 methods of handling syscall exit
directly into a signal handler (as required by sigsuspend et al) by
introducing a TIF_RESTOREALL flag which causes _all_ the registers to be
reloaded from the pt_regs by taking the ret_from_exception path, instead
of the normal syscall exit path which stomps on the callee-saved GPRs.
It appears to pass an LTP test run on ppc64, and passes basic testing on
ppc32 too. Brief tests of ptrace functionality with strace and gdb also
appear OK. I wouldn't send it to Linus for 2.6.15 just yet though :)
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-15 21:52:18 +03:00
cmpld r3 ,r11
ld r5 ,_ C C R ( r1 )
bge- s y s c a l l _ e r r o r
syscall_error_cont :
2005-10-10 16:36:14 +04:00
ld r7 ,_ N I P ( r1 )
stdcx. r0 ,0 ,r1 / * t o c l e a r t h e r e s e r v a t i o n * /
andi. r6 ,r8 ,M S R _ P R
ld r4 ,_ L I N K ( r1 )
2007-02-07 05:13:26 +03:00
/ *
* Clear R I b e f o r e r e s t o r i n g r13 . I f w e a r e r e t u r n i n g t o
* userspace a n d w e t a k e a n e x c e p t i o n a f t e r r e s t o r i n g r13 ,
* we e n d u p c o r r u p t i n g t h e u s e r s p a c e r13 v a l u e .
* /
li r12 ,M S R _ R I
andc r11 ,r10 ,r12
mtmsrd r11 ,1 / * c l e a r M S R . R I * /
powerpc: Implement accurate task and CPU time accounting
This implements accurate task and cpu time accounting for 64-bit
powerpc kernels. Instead of accounting a whole jiffy of time to a
task on a timer interrupt because that task happened to be running at
the time, we now account time in units of timebase ticks according to
the actual time spent by the task in user mode and kernel mode. We
also count the time spent processing hardware and software interrupts
accurately. This is conditional on CONFIG_VIRT_CPU_ACCOUNTING. If
that is not set, we do tick-based approximate accounting as before.
To get this accurate information, we read either the PURR (processor
utilization of resources register) on POWER5 machines, or the timebase
on other machines on
* each entry to the kernel from usermode
* each exit to usermode
* transitions between process context, hard irq context and soft irq
context in kernel mode
* context switches.
On POWER5 systems with shared-processor logical partitioning we also
read both the PURR and the timebase at each timer interrupt and
context switch in order to determine how much time has been taken by
the hypervisor to run other partitions ("steal" time). Unfortunately,
since we need values of the PURR on both threads at the same time to
accurately calculate the steal time, and since we can only calculate
steal time on a per-core basis, the apportioning of the steal time
between idle time (time which we ceded to the hypervisor in the idle
loop) and actual stolen time is somewhat approximate at the moment.
This is all based quite heavily on what s390 does, and it uses the
generic interfaces that were added by the s390 developers,
i.e. account_system_time(), account_user_time(), etc.
This patch doesn't add any new interfaces between the kernel and
userspace, and doesn't change the units in which time is reported to
userspace by things such as /proc/stat, /proc/<pid>/stat, getrusage(),
times(), etc. Internally the various task and cpu times are stored in
timebase units, but they are converted to USER_HZ units (1/100th of a
second) when reported to userspace. Some precision is therefore lost
but there should not be any accumulating error, since the internal
accumulation is at full precision.
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-02-24 02:06:59 +03:00
beq- 1 f
ACCOUNT_ C P U _ U S E R _ E X I T ( r11 , r12 )
ld r13 ,G P R 1 3 ( r1 ) / * o n l y r e s t o r e r13 i f r e t u r n i n g t o u s e r m o d e * /
2005-10-10 16:36:14 +04:00
1 : ld r2 ,G P R 2 ( r1 )
ld r1 ,G P R 1 ( r1 )
mtlr r4
mtcr r5
mtspr S P R N _ S R R 0 ,r7
mtspr S P R N _ S R R 1 ,r8
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
[PATCH] syscall entry/exit revamp
This cleanup patch speeds up the null syscall path on ppc64 by about 3%,
and brings the ppc32 and ppc64 code slightly closer together.
The ppc64 code was checking current_thread_info()->flags twice in the
syscall exit path; once for TIF_SYSCALL_T_OR_A before disabling
interrupts, and then again for TIF_SIGPENDING|TIF_NEED_RESCHED etc after
disabling interrupts. Now we do the same as ppc32 -- check the flags
only once in the fast path, and re-enable interrupts if necessary in the
ptrace case.
The patch abolishes the 'syscall_noerror' member of struct thread_info
and replaces it with a TIF_NOERROR bit in the flags, which is handled in
the slow path. This shortens the syscall entry code, which no longer
needs to clear syscall_noerror.
The patch adds a TIF_SAVE_NVGPRS flag which causes the syscall exit slow
path to save the non-volatile GPRs into a signal frame. This removes the
need for the assembly wrappers around sys_sigsuspend(),
sys_rt_sigsuspend(), et al which existed solely to save those registers
in advance. It also means I don't have to add new wrappers for ppoll()
and pselect(), which is what I was supposed to be doing when I got
distracted into this...
Finally, it unifies the ppc64 and ppc32 methods of handling syscall exit
directly into a signal handler (as required by sigsuspend et al) by
introducing a TIF_RESTOREALL flag which causes _all_ the registers to be
reloaded from the pt_regs by taking the ret_from_exception path, instead
of the normal syscall exit path which stomps on the callee-saved GPRs.
It appears to pass an LTP test run on ppc64, and passes basic testing on
ppc32 too. Brief tests of ptrace functionality with strace and gdb also
appear OK. I wouldn't send it to Linus for 2.6.15 just yet though :)
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-15 21:52:18 +03:00
syscall_error :
2005-10-10 16:36:14 +04:00
oris r5 ,r5 ,0 x10 0 0 / * S e t S O b i t i n C R * /
[PATCH] syscall entry/exit revamp
This cleanup patch speeds up the null syscall path on ppc64 by about 3%,
and brings the ppc32 and ppc64 code slightly closer together.
The ppc64 code was checking current_thread_info()->flags twice in the
syscall exit path; once for TIF_SYSCALL_T_OR_A before disabling
interrupts, and then again for TIF_SIGPENDING|TIF_NEED_RESCHED etc after
disabling interrupts. Now we do the same as ppc32 -- check the flags
only once in the fast path, and re-enable interrupts if necessary in the
ptrace case.
The patch abolishes the 'syscall_noerror' member of struct thread_info
and replaces it with a TIF_NOERROR bit in the flags, which is handled in
the slow path. This shortens the syscall entry code, which no longer
needs to clear syscall_noerror.
The patch adds a TIF_SAVE_NVGPRS flag which causes the syscall exit slow
path to save the non-volatile GPRs into a signal frame. This removes the
need for the assembly wrappers around sys_sigsuspend(),
sys_rt_sigsuspend(), et al which existed solely to save those registers
in advance. It also means I don't have to add new wrappers for ppoll()
and pselect(), which is what I was supposed to be doing when I got
distracted into this...
Finally, it unifies the ppc64 and ppc32 methods of handling syscall exit
directly into a signal handler (as required by sigsuspend et al) by
introducing a TIF_RESTOREALL flag which causes _all_ the registers to be
reloaded from the pt_regs by taking the ret_from_exception path, instead
of the normal syscall exit path which stomps on the callee-saved GPRs.
It appears to pass an LTP test run on ppc64, and passes basic testing on
ppc32 too. Brief tests of ptrace functionality with strace and gdb also
appear OK. I wouldn't send it to Linus for 2.6.15 just yet though :)
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-15 21:52:18 +03:00
neg r3 ,r3
2005-10-10 16:36:14 +04:00
std r5 ,_ C C R ( r1 )
b s y s c a l l _ e r r o r _ c o n t
[PATCH] syscall entry/exit revamp
This cleanup patch speeds up the null syscall path on ppc64 by about 3%,
and brings the ppc32 and ppc64 code slightly closer together.
The ppc64 code was checking current_thread_info()->flags twice in the
syscall exit path; once for TIF_SYSCALL_T_OR_A before disabling
interrupts, and then again for TIF_SIGPENDING|TIF_NEED_RESCHED etc after
disabling interrupts. Now we do the same as ppc32 -- check the flags
only once in the fast path, and re-enable interrupts if necessary in the
ptrace case.
The patch abolishes the 'syscall_noerror' member of struct thread_info
and replaces it with a TIF_NOERROR bit in the flags, which is handled in
the slow path. This shortens the syscall entry code, which no longer
needs to clear syscall_noerror.
The patch adds a TIF_SAVE_NVGPRS flag which causes the syscall exit slow
path to save the non-volatile GPRs into a signal frame. This removes the
need for the assembly wrappers around sys_sigsuspend(),
sys_rt_sigsuspend(), et al which existed solely to save those registers
in advance. It also means I don't have to add new wrappers for ppoll()
and pselect(), which is what I was supposed to be doing when I got
distracted into this...
Finally, it unifies the ppc64 and ppc32 methods of handling syscall exit
directly into a signal handler (as required by sigsuspend et al) by
introducing a TIF_RESTOREALL flag which causes _all_ the registers to be
reloaded from the pt_regs by taking the ret_from_exception path, instead
of the normal syscall exit path which stomps on the callee-saved GPRs.
It appears to pass an LTP test run on ppc64, and passes basic testing on
ppc32 too. Brief tests of ptrace functionality with strace and gdb also
appear OK. I wouldn't send it to Linus for 2.6.15 just yet though :)
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-15 21:52:18 +03:00
2005-10-10 16:36:14 +04:00
/* Traced system call support */
syscall_dotrace :
bl . s a v e _ n v g p r s
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . d o _ s y s c a l l _ t r a c e _ e n t e r
ld r0 ,G P R 0 ( r1 ) / * R e s t o r e o r i g i n a l r e g i s t e r s * /
ld r3 ,G P R 3 ( r1 )
ld r4 ,G P R 4 ( r1 )
ld r5 ,G P R 5 ( r1 )
ld r6 ,G P R 6 ( r1 )
ld r7 ,G P R 7 ( r1 )
ld r8 ,G P R 8 ( r1 )
addi r9 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
clrrdi r10 ,r1 ,T H R E A D _ S H I F T
ld r10 ,T I _ F L A G S ( r10 )
b s y s c a l l _ d o t r a c e _ c o n t
[PATCH] syscall entry/exit revamp
This cleanup patch speeds up the null syscall path on ppc64 by about 3%,
and brings the ppc32 and ppc64 code slightly closer together.
The ppc64 code was checking current_thread_info()->flags twice in the
syscall exit path; once for TIF_SYSCALL_T_OR_A before disabling
interrupts, and then again for TIF_SIGPENDING|TIF_NEED_RESCHED etc after
disabling interrupts. Now we do the same as ppc32 -- check the flags
only once in the fast path, and re-enable interrupts if necessary in the
ptrace case.
The patch abolishes the 'syscall_noerror' member of struct thread_info
and replaces it with a TIF_NOERROR bit in the flags, which is handled in
the slow path. This shortens the syscall entry code, which no longer
needs to clear syscall_noerror.
The patch adds a TIF_SAVE_NVGPRS flag which causes the syscall exit slow
path to save the non-volatile GPRs into a signal frame. This removes the
need for the assembly wrappers around sys_sigsuspend(),
sys_rt_sigsuspend(), et al which existed solely to save those registers
in advance. It also means I don't have to add new wrappers for ppoll()
and pselect(), which is what I was supposed to be doing when I got
distracted into this...
Finally, it unifies the ppc64 and ppc32 methods of handling syscall exit
directly into a signal handler (as required by sigsuspend et al) by
introducing a TIF_RESTOREALL flag which causes _all_ the registers to be
reloaded from the pt_regs by taking the ret_from_exception path, instead
of the normal syscall exit path which stomps on the callee-saved GPRs.
It appears to pass an LTP test run on ppc64, and passes basic testing on
ppc32 too. Brief tests of ptrace functionality with strace and gdb also
appear OK. I wouldn't send it to Linus for 2.6.15 just yet though :)
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-15 21:52:18 +03:00
syscall_enosys :
li r3 ,- E N O S Y S
b s y s c a l l _ e x i t
syscall_exit_work :
/ * If T I F _ R E S T O R E A L L i s s e t , d o n ' t s c r i b b l e o n e i t h e r r3 o r c c r .
If T I F _ N O E R R O R i s s e t , j u s t s a v e r3 a s i t i s . * /
andi. r0 ,r9 ,_ T I F _ R E S T O R E A L L
2006-03-08 05:24:22 +03:00
beq+ 0 f
REST_ N V G P R S ( r1 )
b 2 f
0 : cmpld r3 ,r11 / * r10 i s - L A S T _ E R R N O * /
[PATCH] syscall entry/exit revamp
This cleanup patch speeds up the null syscall path on ppc64 by about 3%,
and brings the ppc32 and ppc64 code slightly closer together.
The ppc64 code was checking current_thread_info()->flags twice in the
syscall exit path; once for TIF_SYSCALL_T_OR_A before disabling
interrupts, and then again for TIF_SIGPENDING|TIF_NEED_RESCHED etc after
disabling interrupts. Now we do the same as ppc32 -- check the flags
only once in the fast path, and re-enable interrupts if necessary in the
ptrace case.
The patch abolishes the 'syscall_noerror' member of struct thread_info
and replaces it with a TIF_NOERROR bit in the flags, which is handled in
the slow path. This shortens the syscall entry code, which no longer
needs to clear syscall_noerror.
The patch adds a TIF_SAVE_NVGPRS flag which causes the syscall exit slow
path to save the non-volatile GPRs into a signal frame. This removes the
need for the assembly wrappers around sys_sigsuspend(),
sys_rt_sigsuspend(), et al which existed solely to save those registers
in advance. It also means I don't have to add new wrappers for ppoll()
and pselect(), which is what I was supposed to be doing when I got
distracted into this...
Finally, it unifies the ppc64 and ppc32 methods of handling syscall exit
directly into a signal handler (as required by sigsuspend et al) by
introducing a TIF_RESTOREALL flag which causes _all_ the registers to be
reloaded from the pt_regs by taking the ret_from_exception path, instead
of the normal syscall exit path which stomps on the callee-saved GPRs.
It appears to pass an LTP test run on ppc64, and passes basic testing on
ppc32 too. Brief tests of ptrace functionality with strace and gdb also
appear OK. I wouldn't send it to Linus for 2.6.15 just yet though :)
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-15 21:52:18 +03:00
blt+ 1 f
andi. r0 ,r9 ,_ T I F _ N O E R R O R
bne- 1 f
ld r5 ,_ C C R ( r1 )
neg r3 ,r3
oris r5 ,r5 ,0 x10 0 0 / * S e t S O b i t i n C R * /
std r5 ,_ C C R ( r1 )
1 : std r3 ,G P R 3 ( r1 )
2 : andi. r0 ,r9 ,( _ T I F _ P E R S Y S C A L L _ M A S K )
beq 4 f
2006-03-08 05:24:22 +03:00
/* Clear per-syscall TIF flags if any are set. */
[PATCH] syscall entry/exit revamp
This cleanup patch speeds up the null syscall path on ppc64 by about 3%,
and brings the ppc32 and ppc64 code slightly closer together.
The ppc64 code was checking current_thread_info()->flags twice in the
syscall exit path; once for TIF_SYSCALL_T_OR_A before disabling
interrupts, and then again for TIF_SIGPENDING|TIF_NEED_RESCHED etc after
disabling interrupts. Now we do the same as ppc32 -- check the flags
only once in the fast path, and re-enable interrupts if necessary in the
ptrace case.
The patch abolishes the 'syscall_noerror' member of struct thread_info
and replaces it with a TIF_NOERROR bit in the flags, which is handled in
the slow path. This shortens the syscall entry code, which no longer
needs to clear syscall_noerror.
The patch adds a TIF_SAVE_NVGPRS flag which causes the syscall exit slow
path to save the non-volatile GPRs into a signal frame. This removes the
need for the assembly wrappers around sys_sigsuspend(),
sys_rt_sigsuspend(), et al which existed solely to save those registers
in advance. It also means I don't have to add new wrappers for ppoll()
and pselect(), which is what I was supposed to be doing when I got
distracted into this...
Finally, it unifies the ppc64 and ppc32 methods of handling syscall exit
directly into a signal handler (as required by sigsuspend et al) by
introducing a TIF_RESTOREALL flag which causes _all_ the registers to be
reloaded from the pt_regs by taking the ret_from_exception path, instead
of the normal syscall exit path which stomps on the callee-saved GPRs.
It appears to pass an LTP test run on ppc64, and passes basic testing on
ppc32 too. Brief tests of ptrace functionality with strace and gdb also
appear OK. I wouldn't send it to Linus for 2.6.15 just yet though :)
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-15 21:52:18 +03:00
li r11 ,_ T I F _ P E R S Y S C A L L _ M A S K
addi r12 ,r12 ,T I _ F L A G S
3 : ldarx r10 ,0 ,r12
andc r10 ,r10 ,r11
stdcx. r10 ,0 ,r12
bne- 3 b
subi r12 ,r12 ,T I _ F L A G S
2006-03-08 05:24:22 +03:00
4 : /* Anything else left to do? */
andi. r0 ,r9 ,( _ T I F _ S Y S C A L L _ T _ O R _ A | _ T I F _ S I N G L E S T E P )
[PATCH] syscall entry/exit revamp
This cleanup patch speeds up the null syscall path on ppc64 by about 3%,
and brings the ppc32 and ppc64 code slightly closer together.
The ppc64 code was checking current_thread_info()->flags twice in the
syscall exit path; once for TIF_SYSCALL_T_OR_A before disabling
interrupts, and then again for TIF_SIGPENDING|TIF_NEED_RESCHED etc after
disabling interrupts. Now we do the same as ppc32 -- check the flags
only once in the fast path, and re-enable interrupts if necessary in the
ptrace case.
The patch abolishes the 'syscall_noerror' member of struct thread_info
and replaces it with a TIF_NOERROR bit in the flags, which is handled in
the slow path. This shortens the syscall entry code, which no longer
needs to clear syscall_noerror.
The patch adds a TIF_SAVE_NVGPRS flag which causes the syscall exit slow
path to save the non-volatile GPRs into a signal frame. This removes the
need for the assembly wrappers around sys_sigsuspend(),
sys_rt_sigsuspend(), et al which existed solely to save those registers
in advance. It also means I don't have to add new wrappers for ppoll()
and pselect(), which is what I was supposed to be doing when I got
distracted into this...
Finally, it unifies the ppc64 and ppc32 methods of handling syscall exit
directly into a signal handler (as required by sigsuspend et al) by
introducing a TIF_RESTOREALL flag which causes _all_ the registers to be
reloaded from the pt_regs by taking the ret_from_exception path, instead
of the normal syscall exit path which stomps on the callee-saved GPRs.
It appears to pass an LTP test run on ppc64, and passes basic testing on
ppc32 too. Brief tests of ptrace functionality with strace and gdb also
appear OK. I wouldn't send it to Linus for 2.6.15 just yet though :)
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-15 21:52:18 +03:00
beq . r e t _ f r o m _ e x c e p t _ l i t e
/* Re-enable interrupts */
mfmsr r10
ori r10 ,r10 ,M S R _ E E
mtmsrd r10 ,1
2006-03-08 05:24:22 +03:00
bl . s a v e _ n v g p r s
2005-10-10 16:36:14 +04:00
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . d o _ s y s c a l l _ t r a c e _ l e a v e
2006-03-08 05:24:22 +03:00
b . r e t _ f r o m _ e x c e p t
2005-10-10 16:36:14 +04:00
/* Save non-volatile GPRs, if not already saved. */
_ GLOBAL( s a v e _ n v g p r s )
ld r11 ,_ T R A P ( r1 )
andi. r0 ,r11 ,1
beqlr-
SAVE_ N V G P R S ( r1 )
clrrdi r0 ,r11 ,1
std r0 ,_ T R A P ( r1 )
blr
[PATCH] syscall entry/exit revamp
This cleanup patch speeds up the null syscall path on ppc64 by about 3%,
and brings the ppc32 and ppc64 code slightly closer together.
The ppc64 code was checking current_thread_info()->flags twice in the
syscall exit path; once for TIF_SYSCALL_T_OR_A before disabling
interrupts, and then again for TIF_SIGPENDING|TIF_NEED_RESCHED etc after
disabling interrupts. Now we do the same as ppc32 -- check the flags
only once in the fast path, and re-enable interrupts if necessary in the
ptrace case.
The patch abolishes the 'syscall_noerror' member of struct thread_info
and replaces it with a TIF_NOERROR bit in the flags, which is handled in
the slow path. This shortens the syscall entry code, which no longer
needs to clear syscall_noerror.
The patch adds a TIF_SAVE_NVGPRS flag which causes the syscall exit slow
path to save the non-volatile GPRs into a signal frame. This removes the
need for the assembly wrappers around sys_sigsuspend(),
sys_rt_sigsuspend(), et al which existed solely to save those registers
in advance. It also means I don't have to add new wrappers for ppoll()
and pselect(), which is what I was supposed to be doing when I got
distracted into this...
Finally, it unifies the ppc64 and ppc32 methods of handling syscall exit
directly into a signal handler (as required by sigsuspend et al) by
introducing a TIF_RESTOREALL flag which causes _all_ the registers to be
reloaded from the pt_regs by taking the ret_from_exception path, instead
of the normal syscall exit path which stomps on the callee-saved GPRs.
It appears to pass an LTP test run on ppc64, and passes basic testing on
ppc32 too. Brief tests of ptrace functionality with strace and gdb also
appear OK. I wouldn't send it to Linus for 2.6.15 just yet though :)
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2005-11-15 21:52:18 +03:00
2005-10-10 16:36:14 +04:00
/ *
* The s i g s u s p e n d a n d r t _ s i g s u s p e n d s y s t e m c a l l s c a n c a l l d o _ s i g n a l
* and t h u s p u t t h e p r o c e s s i n t o t h e s t o p p e d s t a t e w h e r e w e m i g h t
* want t o e x a m i n e i t s u s e r s t a t e w i t h p t r a c e . T h e r e f o r e w e n e e d
* to s a v e a l l t h e n o n v o l a t i l e r e g i s t e r s ( r14 - r31 ) b e f o r e c a l l i n g
* the C c o d e . S i m i l a r l y , f o r k , v f o r k a n d c l o n e n e e d t h e f u l l
* register s t a t e o n t h e s t a c k s o t h a t i t c a n b e c o p i e d t o t h e c h i l d .
* /
_ GLOBAL( p p c _ f o r k )
bl . s a v e _ n v g p r s
bl . s y s _ f o r k
b s y s c a l l _ e x i t
_ GLOBAL( p p c _ v f o r k )
bl . s a v e _ n v g p r s
bl . s y s _ v f o r k
b s y s c a l l _ e x i t
_ GLOBAL( p p c _ c l o n e )
bl . s a v e _ n v g p r s
bl . s y s _ c l o n e
b s y s c a l l _ e x i t
2006-03-08 05:24:22 +03:00
_ GLOBAL( p p c32 _ s w a p c o n t e x t )
bl . s a v e _ n v g p r s
bl . c o m p a t _ s y s _ s w a p c o n t e x t
b s y s c a l l _ e x i t
_ GLOBAL( p p c64 _ s w a p c o n t e x t )
bl . s a v e _ n v g p r s
bl . s y s _ s w a p c o n t e x t
b s y s c a l l _ e x i t
2005-10-10 16:36:14 +04:00
_ GLOBAL( r e t _ f r o m _ f o r k )
bl . s c h e d u l e _ t a i l
REST_ N V G P R S ( r1 )
li r3 ,0
b s y s c a l l _ e x i t
/ *
* This r o u t i n e s w i t c h e s b e t w e e n t w o d i f f e r e n t t a s k s . T h e p r o c e s s
* state o f o n e i s s a v e d o n i t s k e r n e l s t a c k . T h e n t h e s t a t e
* of t h e o t h e r i s r e s t o r e d f r o m i t s k e r n e l s t a c k . T h e m e m o r y
* management h a r d w a r e i s u p d a t e d t o t h e s e c o n d p r o c e s s ' s s t a t e .
* Finally, w e c a n r e t u r n t o t h e s e c o n d p r o c e s s , v i a r e t _ f r o m _ e x c e p t .
* On e n t r y , r3 p o i n t s t o t h e T H R E A D f o r t h e c u r r e n t t a s k , r4
* points t o t h e T H R E A D f o r t h e n e w t a s k .
*
* Note : there a r e t w o w a y s t o g e t t o t h e " g o i n g o u t " p o r t i o n
* of t h i s c o d e ; either by coming in via the entry (_switch)
* or v i a " f o r k " w h i c h m u s t s e t u p a n e n v i r o n m e n t e q u i v a l e n t
* to t h e " _ s w i t c h " p a t h . I f y o u c h a n g e t h i s y o u ' l l h a v e t o c h a n g e
* the f o r k c o d e a l s o .
*
* The c o d e w h i c h c r e a t e s t h e n e w t a s k c o n t e x t i s i n ' c o p y _ t h r e a d '
2006-01-23 19:58:20 +03:00
* in a r c h / p o w e r p c / k e r n e l / p r o c e s s . c
2005-10-10 16:36:14 +04:00
* /
.align 7
_ GLOBAL( _ s w i t c h )
mflr r0
std r0 ,1 6 ( r1 )
stdu r1 ,- S W I T C H _ F R A M E _ S I Z E ( r1 )
/* r3-r13 are caller saved -- Cort */
SAVE_ 8 G P R S ( 1 4 , r1 )
SAVE_ 1 0 G P R S ( 2 2 , r1 )
mflr r20 / * R e t u r n t o s w i t c h c a l l e r * /
mfmsr r22
li r0 , M S R _ F P
# ifdef C O N F I G _ A L T I V E C
BEGIN_ F T R _ S E C T I O N
oris r0 ,r0 ,M S R _ V E C @h /* Disable altivec */
mfspr r24 ,S P R N _ V R S A V E / * s a v e v r s a v e r e g i s t e r v a l u e * /
std r24 ,T H R E A D _ V R S A V E ( r3 )
END_ F T R _ S E C T I O N _ I F S E T ( C P U _ F T R _ A L T I V E C )
# endif / * C O N F I G _ A L T I V E C * /
and. r0 ,r0 ,r22
beq+ 1 f
andc r22 ,r22 ,r0
mtmsrd r22
isync
1 : std r20 ,_ N I P ( r1 )
mfcr r23
std r23 ,_ C C R ( r1 )
std r1 ,K S P ( r3 ) / * S e t o l d s t a c k p o i n t e r * /
# ifdef C O N F I G _ S M P
/ * We n e e d a s y n c s o m e w h e r e h e r e t o m a k e s u r e t h a t i f t h e
* previous t a s k g e t s r e s c h e d u l e d o n a n o t h e r C P U , i t s e e s a l l
* stores i t h a s p e r f o r m e d o n t h i s o n e .
* /
sync
# endif / * C O N F I G _ S M P * /
addi r6 ,r4 ,- T H R E A D / * C o n v e r t T H R E A D t o ' c u r r e n t ' * /
std r6 ,P A C A C U R R E N T ( r13 ) / * S e t n e w ' c u r r e n t ' * /
ld r8 ,K S P ( r4 ) / * n e w s t a c k p o i n t e r * /
BEGIN_ F T R _ S E C T I O N
clrrdi r6 ,r8 ,2 8 / * g e t i t s E S I D * /
clrrdi r9 ,r1 ,2 8 / * g e t c u r r e n t s p E S I D * /
clrldi. r0 ,r6 ,2 / * i s n e w E S I D c00 0 0 0 0 0 0 ? * /
cmpd c r1 ,r6 ,r9 / * o r i s n e w E S I D t h e s a m e a s c u r r e n t E S I D ? * /
cror e q ,4 * c r1 + e q ,e q
beq 2 f / * i f y e s , d o n ' t s l b i e i t * /
/* Bolt in the new stack SLB entry */
ld r7 ,K S P _ V S I D ( r4 ) / * G e t n e w s t a c k ' s V S I D * /
oris r0 ,r6 ,( S L B _ E S I D _ V ) @h
ori r0 ,r0 ,( S L B _ N U M _ B O L T E D - 1 ) @l
2006-08-07 10:19:19 +04:00
/* Update the last bolted SLB */
ld r9 ,P A C A _ S L B S H A D O W P T R ( r13 )
2006-08-09 11:00:30 +04:00
li r12 ,0
std r12 ,S L B S H A D O W _ S T A C K E S I D ( r9 ) / * C l e a r E S I D * /
2007-08-03 05:55:39 +04:00
eieio
2006-08-09 11:00:30 +04:00
std r7 ,S L B S H A D O W _ S T A C K V S I D ( r9 ) / * S a v e V S I D * /
2007-08-03 05:55:39 +04:00
eieio
2006-08-09 11:00:30 +04:00
std r0 ,S L B S H A D O W _ S T A C K E S I D ( r9 ) / * S a v e E S I D * /
2007-08-03 05:55:39 +04:00
eieio
2006-08-07 10:19:19 +04:00
2005-10-10 16:36:14 +04:00
slbie r6
slbie r6 / * W o r k a r o u n d P O W E R 5 < D D 2 . 1 i s s u e * /
slbmte r7 ,r0
isync
2 :
END_ F T R _ S E C T I O N _ I F S E T ( C P U _ F T R _ S L B )
clrrdi r7 ,r8 ,T H R E A D _ S H I F T / * b a s e o f n e w s t a c k * /
/ * Note : this u s e s S W I T C H _ F R A M E _ S I Z E r a t h e r t h a n I N T _ F R A M E _ S I Z E
because w e d o n ' t n e e d t o l e a v e t h e 2 8 8 - b y t e A B I g a p a t t h e
top o f t h e k e r n e l s t a c k . * /
addi r7 ,r7 ,T H R E A D _ S I Z E - S W I T C H _ F R A M E _ S I Z E
mr r1 ,r8 / * s t a r t u s i n g n e w s t a c k p o i n t e r * /
std r7 ,P A C A K S A V E ( r13 )
ld r6 ,_ C C R ( r1 )
mtcrf 0 x F F ,r6
# ifdef C O N F I G _ A L T I V E C
BEGIN_ F T R _ S E C T I O N
ld r0 ,T H R E A D _ V R S A V E ( r4 )
mtspr S P R N _ V R S A V E ,r0 / * i f G 4 , r e s t o r e V R S A V E r e g * /
END_ F T R _ S E C T I O N _ I F S E T ( C P U _ F T R _ A L T I V E C )
# endif / * C O N F I G _ A L T I V E C * /
/* r3-r13 are destroyed -- Cort */
REST_ 8 G P R S ( 1 4 , r1 )
REST_ 1 0 G P R S ( 2 2 , r1 )
/* convert old thread to its task_struct for return value */
addi r3 ,r3 ,- T H R E A D
ld r7 ,_ N I P ( r1 ) / * R e t u r n t o _ s w i t c h c a l l e r i n n e w t a s k * /
mtlr r7
addi r1 ,r1 ,S W I T C H _ F R A M E _ S I Z E
blr
.align 7
_ GLOBAL( r e t _ f r o m _ e x c e p t )
ld r11 ,_ T R A P ( r1 )
andi. r0 ,r11 ,1
bne . r e t _ f r o m _ e x c e p t _ l i t e
REST_ N V G P R S ( r1 )
_ GLOBAL( r e t _ f r o m _ e x c e p t _ l i t e )
/ *
* Disable i n t e r r u p t s s o t h a t c u r r e n t _ t h r e a d _ i n f o ( ) - > f l a g s
* can' t c h a n g e b e t w e e n w h e n w e t e s t i t a n d w h e n w e r e t u r n
* from t h e i n t e r r u p t .
* /
mfmsr r10 / * G e t c u r r e n t i n t e r r u p t s t a t e * /
rldicl r9 ,r10 ,4 8 ,1 / * c l e a r M S R _ E E * /
rotldi r9 ,r9 ,1 6
mtmsrd r9 ,1 / * U p d a t e m a c h i n e s t a t e * /
# ifdef C O N F I G _ P R E E M P T
clrrdi r9 ,r1 ,T H R E A D _ S H I F T / * c u r r e n t _ t h r e a d _ i n f o ( ) * /
li r0 ,_ T I F _ N E E D _ R E S C H E D / * b i t s t o c h e c k * /
ld r3 ,_ M S R ( r1 )
ld r4 ,T I _ F L A G S ( r9 )
/* Move MSR_PR bit in r3 to _TIF_SIGPENDING position in r0 */
rlwimi r0 ,r3 ,3 2 + T I F _ S I G P E N D I N G - M S R _ P R _ L G ,_ T I F _ S I G P E N D I N G
and. r0 ,r4 ,r0 / * c h e c k N E E D _ R E S C H E D a n d m a y b e S I G P E N D I N G * /
bne d o _ w o r k
# else / * ! C O N F I G _ P R E E M P T * /
ld r3 ,_ M S R ( r1 ) / * R e t u r n i n g t o u s e r m o d e ? * /
andi. r3 ,r3 ,M S R _ P R
beq r e s t o r e / * i f n o t , j u s t r e s t o r e r e g s a n d r e t u r n * /
/* Check current_thread_info()->flags */
clrrdi r9 ,r1 ,T H R E A D _ S H I F T
ld r4 ,T I _ F L A G S ( r9 )
andi. r0 ,r4 ,_ T I F _ U S E R _ W O R K _ M A S K
bne d o _ w o r k
# endif
restore :
[POWERPC] Lazy interrupt disabling for 64-bit machines
This implements a lazy strategy for disabling interrupts. This means
that local_irq_disable() et al. just clear the 'interrupts are
enabled' flag in the paca. If an interrupt comes along, the interrupt
entry code notices that interrupts are supposed to be disabled, and
clears the EE bit in SRR1, clears the 'interrupts are hard-enabled'
flag in the paca, and returns. This means that interrupts only
actually get disabled in the processor when an interrupt comes along.
When interrupts are enabled by local_irq_enable() et al., the code
sets the interrupts-enabled flag in the paca, and then checks whether
interrupts got hard-disabled. If so, it also sets the EE bit in the
MSR to hard-enable the interrupts.
This has the potential to improve performance, and also makes it
easier to make a kernel that can boot on iSeries and on other 64-bit
machines, since this lazy-disable strategy is very similar to the
soft-disable strategy that iSeries already uses.
This version renames paca->proc_enabled to paca->soft_enabled, and
changes a couple of soft-disables in the kexec code to hard-disables,
which should fix the crash that Michael Ellerman saw. This doesn't
yet use a reserved CR field for the soft_enabled and hard_enabled
flags. This applies on top of Stephen Rothwell's patches to make it
possible to build a combined iSeries/other kernel.
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-10-04 10:47:49 +04:00
ld r5 ,S O F T E ( r1 )
2005-10-10 16:36:14 +04:00
# ifdef C O N F I G _ P P C _ I S E R I E S
2006-09-25 12:19:00 +04:00
BEGIN_ F W _ F T R _ S E C T I O N
2005-10-10 16:36:14 +04:00
cmpdi 0 ,r5 ,0
beq 4 f
/* Check for pending interrupts (iSeries) */
2006-01-13 02:26:42 +03:00
ld r3 ,P A C A L P P A C A P T R ( r13 )
ld r3 ,L P P A C A A N Y I N T ( r3 )
2005-10-10 16:36:14 +04:00
cmpdi r3 ,0
beq+ 4 f / * s k i p d o _ I R Q i f n o i n t e r r u p t s * /
li r3 ,0
[POWERPC] Lazy interrupt disabling for 64-bit machines
This implements a lazy strategy for disabling interrupts. This means
that local_irq_disable() et al. just clear the 'interrupts are
enabled' flag in the paca. If an interrupt comes along, the interrupt
entry code notices that interrupts are supposed to be disabled, and
clears the EE bit in SRR1, clears the 'interrupts are hard-enabled'
flag in the paca, and returns. This means that interrupts only
actually get disabled in the processor when an interrupt comes along.
When interrupts are enabled by local_irq_enable() et al., the code
sets the interrupts-enabled flag in the paca, and then checks whether
interrupts got hard-disabled. If so, it also sets the EE bit in the
MSR to hard-enable the interrupts.
This has the potential to improve performance, and also makes it
easier to make a kernel that can boot on iSeries and on other 64-bit
machines, since this lazy-disable strategy is very similar to the
soft-disable strategy that iSeries already uses.
This version renames paca->proc_enabled to paca->soft_enabled, and
changes a couple of soft-disables in the kexec code to hard-disables,
which should fix the crash that Michael Ellerman saw. This doesn't
yet use a reserved CR field for the soft_enabled and hard_enabled
flags. This applies on top of Stephen Rothwell's patches to make it
possible to build a combined iSeries/other kernel.
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-10-04 10:47:49 +04:00
stb r3 ,P A C A S O F T I R Q E N ( r13 ) / * e n s u r e w e a r e s o f t - d i s a b l e d * /
2005-10-10 16:36:14 +04:00
ori r10 ,r10 ,M S R _ E E
mtmsrd r10 / * h a r d - e n a b l e a g a i n * /
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . d o _ I R Q
b . r e t _ f r o m _ e x c e p t _ l i t e / * l o o p b a c k a n d h a n d l e m o r e * /
[POWERPC] Lazy interrupt disabling for 64-bit machines
This implements a lazy strategy for disabling interrupts. This means
that local_irq_disable() et al. just clear the 'interrupts are
enabled' flag in the paca. If an interrupt comes along, the interrupt
entry code notices that interrupts are supposed to be disabled, and
clears the EE bit in SRR1, clears the 'interrupts are hard-enabled'
flag in the paca, and returns. This means that interrupts only
actually get disabled in the processor when an interrupt comes along.
When interrupts are enabled by local_irq_enable() et al., the code
sets the interrupts-enabled flag in the paca, and then checks whether
interrupts got hard-disabled. If so, it also sets the EE bit in the
MSR to hard-enable the interrupts.
This has the potential to improve performance, and also makes it
easier to make a kernel that can boot on iSeries and on other 64-bit
machines, since this lazy-disable strategy is very similar to the
soft-disable strategy that iSeries already uses.
This version renames paca->proc_enabled to paca->soft_enabled, and
changes a couple of soft-disables in the kexec code to hard-disables,
which should fix the crash that Michael Ellerman saw. This doesn't
yet use a reserved CR field for the soft_enabled and hard_enabled
flags. This applies on top of Stephen Rothwell's patches to make it
possible to build a combined iSeries/other kernel.
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-10-04 10:47:49 +04:00
4 :
2006-09-25 12:19:00 +04:00
END_ F W _ F T R _ S E C T I O N _ I F S E T ( F W _ F E A T U R E _ I S E R I E S )
2005-10-10 16:36:14 +04:00
# endif
[POWERPC] Lazy interrupt disabling for 64-bit machines
This implements a lazy strategy for disabling interrupts. This means
that local_irq_disable() et al. just clear the 'interrupts are
enabled' flag in the paca. If an interrupt comes along, the interrupt
entry code notices that interrupts are supposed to be disabled, and
clears the EE bit in SRR1, clears the 'interrupts are hard-enabled'
flag in the paca, and returns. This means that interrupts only
actually get disabled in the processor when an interrupt comes along.
When interrupts are enabled by local_irq_enable() et al., the code
sets the interrupts-enabled flag in the paca, and then checks whether
interrupts got hard-disabled. If so, it also sets the EE bit in the
MSR to hard-enable the interrupts.
This has the potential to improve performance, and also makes it
easier to make a kernel that can boot on iSeries and on other 64-bit
machines, since this lazy-disable strategy is very similar to the
soft-disable strategy that iSeries already uses.
This version renames paca->proc_enabled to paca->soft_enabled, and
changes a couple of soft-disables in the kexec code to hard-disables,
which should fix the crash that Michael Ellerman saw. This doesn't
yet use a reserved CR field for the soft_enabled and hard_enabled
flags. This applies on top of Stephen Rothwell's patches to make it
possible to build a combined iSeries/other kernel.
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-10-04 10:47:49 +04:00
stb r5 ,P A C A S O F T I R Q E N ( r13 )
2005-10-10 16:36:14 +04:00
2007-02-07 05:13:26 +03:00
/* extract EE bit and use it to restore paca->hard_enabled */
2005-10-10 16:36:14 +04:00
ld r3 ,_ M S R ( r1 )
2007-02-07 05:13:26 +03:00
rldicl r4 ,r3 ,4 9 ,6 3 / * r0 = ( r3 > > 1 5 ) & 1 * /
stb r4 ,P A C A H A R D I R Q E N ( r13 )
ld r4 ,_ C T R ( r1 )
ld r0 ,_ L I N K ( r1 )
mtctr r4
mtlr r0
ld r4 ,_ X E R ( r1 )
mtspr S P R N _ X E R ,r4
REST_ 8 G P R S ( 5 , r1 )
2005-10-10 16:36:14 +04:00
andi. r0 ,r3 ,M S R _ R I
beq- u n r e c o v _ r e s t o r e
2007-02-07 05:13:26 +03:00
stdcx. r0 ,0 ,r1 / * t o c l e a r t h e r e s e r v a t i o n * /
2006-10-18 04:11:22 +04:00
2007-02-07 05:13:26 +03:00
/ *
* Clear R I b e f o r e r e s t o r i n g r13 . I f w e a r e r e t u r n i n g t o
* userspace a n d w e t a k e a n e x c e p t i o n a f t e r r e s t o r i n g r13 ,
* we e n d u p c o r r u p t i n g t h e u s e r s p a c e r13 v a l u e .
* /
mfmsr r4
andc r4 ,r4 ,r0 / * r0 c o n t a i n s M S R _ R I h e r e * /
mtmsrd r4 ,1
2005-10-10 16:36:14 +04:00
/ *
* r1 3 i s o u r p e r c p u a r e a , o n l y r e s t o r e i t i f w e a r e r e t u r n i n g t o
* userspace
* /
2007-02-07 05:13:26 +03:00
andi. r0 ,r3 ,M S R _ P R
2005-10-10 16:36:14 +04:00
beq 1 f
2007-02-07 05:13:26 +03:00
ACCOUNT_ C P U _ U S E R _ E X I T ( r2 , r4 )
2005-10-10 16:36:14 +04:00
REST_ G P R ( 1 3 , r1 )
1 :
2007-02-07 05:13:26 +03:00
mtspr S P R N _ S R R 1 ,r3
2005-10-10 16:36:14 +04:00
ld r2 ,_ C C R ( r1 )
mtcrf 0 x F F ,r2
ld r2 ,_ N I P ( r1 )
mtspr S P R N _ S R R 0 ,r2
ld r0 ,G P R 0 ( r1 )
ld r2 ,G P R 2 ( r1 )
ld r3 ,G P R 3 ( r1 )
ld r4 ,G P R 4 ( r1 )
ld r1 ,G P R 1 ( r1 )
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
do_work :
# ifdef C O N F I G _ P R E E M P T
andi. r0 ,r3 ,M S R _ P R / * R e t u r n i n g t o u s e r m o d e ? * /
bne u s e r _ w o r k
/* Check that preempt_count() == 0 and interrupts are enabled */
lwz r8 ,T I _ P R E E M P T ( r9 )
cmpwi c r1 ,r8 ,0
ld r0 ,S O F T E ( r1 )
cmpdi r0 ,0
crandc e q ,c r1 * 4 + e q ,e q
bne r e s t o r e
/* here we are preempting the current task */
1 :
li r0 ,1
[POWERPC] Lazy interrupt disabling for 64-bit machines
This implements a lazy strategy for disabling interrupts. This means
that local_irq_disable() et al. just clear the 'interrupts are
enabled' flag in the paca. If an interrupt comes along, the interrupt
entry code notices that interrupts are supposed to be disabled, and
clears the EE bit in SRR1, clears the 'interrupts are hard-enabled'
flag in the paca, and returns. This means that interrupts only
actually get disabled in the processor when an interrupt comes along.
When interrupts are enabled by local_irq_enable() et al., the code
sets the interrupts-enabled flag in the paca, and then checks whether
interrupts got hard-disabled. If so, it also sets the EE bit in the
MSR to hard-enable the interrupts.
This has the potential to improve performance, and also makes it
easier to make a kernel that can boot on iSeries and on other 64-bit
machines, since this lazy-disable strategy is very similar to the
soft-disable strategy that iSeries already uses.
This version renames paca->proc_enabled to paca->soft_enabled, and
changes a couple of soft-disables in the kexec code to hard-disables,
which should fix the crash that Michael Ellerman saw. This doesn't
yet use a reserved CR field for the soft_enabled and hard_enabled
flags. This applies on top of Stephen Rothwell's patches to make it
possible to build a combined iSeries/other kernel.
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-10-04 10:47:49 +04:00
stb r0 ,P A C A S O F T I R Q E N ( r13 )
stb r0 ,P A C A H A R D I R Q E N ( r13 )
2005-10-10 16:36:14 +04:00
ori r10 ,r10 ,M S R _ E E
mtmsrd r10 ,1 / * r e e n a b l e i n t e r r u p t s * /
bl . p r e e m p t _ s c h e d u l e
mfmsr r10
clrrdi r9 ,r1 ,T H R E A D _ S H I F T
rldicl r10 ,r10 ,4 8 ,1 / * d i s a b l e i n t e r r u p t s a g a i n * /
rotldi r10 ,r10 ,1 6
mtmsrd r10 ,1
ld r4 ,T I _ F L A G S ( r9 )
andi. r0 ,r4 ,_ T I F _ N E E D _ R E S C H E D
bne 1 b
b r e s t o r e
user_work :
# endif
/* Enable interrupts */
ori r10 ,r10 ,M S R _ E E
mtmsrd r10 ,1
andi. r0 ,r4 ,_ T I F _ N E E D _ R E S C H E D
beq 1 f
bl . s c h e d u l e
b . r e t _ f r o m _ e x c e p t _ l i t e
1 : bl . s a v e _ n v g p r s
li r3 ,0
addi r4 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . d o _ s i g n a l
b . r e t _ f r o m _ e x c e p t
unrecov_restore :
addi r3 ,r1 ,S T A C K _ F R A M E _ O V E R H E A D
bl . u n r e c o v e r a b l e _ e x c e p t i o n
b u n r e c o v _ r e s t o r e
# ifdef C O N F I G _ P P C _ R T A S
/ *
* On C H R P , t h e R u n - T i m e A b s t r a c t i o n S e r v i c e s ( R T A S ) h a v e t o b e
* called w i t h t h e M M U o f f .
*
* In a d d i t i o n , w e n e e d t o b e i n 3 2 b m o d e , a t l e a s t f o r n o w .
*
* Note : r3 i s a n i n p u t p a r a m e t e r t o r t a s , s o d o n ' t t r a s h i t . . .
* /
_ GLOBAL( e n t e r _ r t a s )
mflr r0
std r0 ,1 6 ( r1 )
stdu r1 ,- R T A S _ F R A M E _ S I Z E ( r1 ) / * S a v e S P a n d c r e a t e s t a c k s p a c e . * /
/ * Because R T A S i s r u n n i n g i n 3 2 b m o d e , i t c l o b b e r s t h e h i g h o r d e r h a l f
* of a l l r e g i s t e r s t h a t i t s a v e s . W e t h e r e f o r e s a v e t h o s e r e g i s t e r s
* RTAS m i g h t t o u c h t o t h e s t a c k . ( r0 , r3 - r13 a r e c a l l e r s a v e d )
* /
SAVE_ G P R ( 2 , r1 ) / * S a v e t h e T O C * /
SAVE_ G P R ( 1 3 , r1 ) / * S a v e p a c a * /
SAVE_ 8 G P R S ( 1 4 , r1 ) / * S a v e t h e n o n - v o l a t i l e s * /
SAVE_ 1 0 G P R S ( 2 2 , r1 ) / * d i t t o * /
mfcr r4
std r4 ,_ C C R ( r1 )
mfctr r5
std r5 ,_ C T R ( r1 )
mfspr r6 ,S P R N _ X E R
std r6 ,_ X E R ( r1 )
mfdar r7
std r7 ,_ D A R ( r1 )
mfdsisr r8
std r8 ,_ D S I S R ( r1 )
mfsrr0 r9
std r9 ,_ S R R 0 ( r1 )
mfsrr1 r10
std r10 ,_ S R R 1 ( r1 )
2006-03-28 03:20:00 +04:00
/ * Temporary w o r k a r o u n d t o c l e a r C R u n t i l R T A S c a n b e m o d i f i e d t o
* ignore a l l b i t s .
* /
li r0 ,0
mtcr r0
2007-01-01 21:45:34 +03:00
# ifdef C O N F I G _ B U G
2005-10-10 16:36:14 +04:00
/ * There i s n o w a y i t i s a c c e p t a b l e t o g e t h e r e w i t h i n t e r r u p t s e n a b l e d ,
* check i t w i t h t h e a s m e q u i v a l e n t o f W A R N _ O N
* /
[POWERPC] Lazy interrupt disabling for 64-bit machines
This implements a lazy strategy for disabling interrupts. This means
that local_irq_disable() et al. just clear the 'interrupts are
enabled' flag in the paca. If an interrupt comes along, the interrupt
entry code notices that interrupts are supposed to be disabled, and
clears the EE bit in SRR1, clears the 'interrupts are hard-enabled'
flag in the paca, and returns. This means that interrupts only
actually get disabled in the processor when an interrupt comes along.
When interrupts are enabled by local_irq_enable() et al., the code
sets the interrupts-enabled flag in the paca, and then checks whether
interrupts got hard-disabled. If so, it also sets the EE bit in the
MSR to hard-enable the interrupts.
This has the potential to improve performance, and also makes it
easier to make a kernel that can boot on iSeries and on other 64-bit
machines, since this lazy-disable strategy is very similar to the
soft-disable strategy that iSeries already uses.
This version renames paca->proc_enabled to paca->soft_enabled, and
changes a couple of soft-disables in the kexec code to hard-disables,
which should fix the crash that Michael Ellerman saw. This doesn't
yet use a reserved CR field for the soft_enabled and hard_enabled
flags. This applies on top of Stephen Rothwell's patches to make it
possible to build a combined iSeries/other kernel.
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-10-04 10:47:49 +04:00
lbz r0 ,P A C A S O F T I R Q E N ( r13 )
2005-10-10 16:36:14 +04:00
1 : tdnei r0 ,0
2007-01-01 21:45:34 +03:00
EMIT_ B U G _ E N T R Y 1 b ,_ _ F I L E _ _ ,_ _ L I N E _ _ ,B U G F L A G _ W A R N I N G
# endif
[POWERPC] Lazy interrupt disabling for 64-bit machines
This implements a lazy strategy for disabling interrupts. This means
that local_irq_disable() et al. just clear the 'interrupts are
enabled' flag in the paca. If an interrupt comes along, the interrupt
entry code notices that interrupts are supposed to be disabled, and
clears the EE bit in SRR1, clears the 'interrupts are hard-enabled'
flag in the paca, and returns. This means that interrupts only
actually get disabled in the processor when an interrupt comes along.
When interrupts are enabled by local_irq_enable() et al., the code
sets the interrupts-enabled flag in the paca, and then checks whether
interrupts got hard-disabled. If so, it also sets the EE bit in the
MSR to hard-enable the interrupts.
This has the potential to improve performance, and also makes it
easier to make a kernel that can boot on iSeries and on other 64-bit
machines, since this lazy-disable strategy is very similar to the
soft-disable strategy that iSeries already uses.
This version renames paca->proc_enabled to paca->soft_enabled, and
changes a couple of soft-disables in the kexec code to hard-disables,
which should fix the crash that Michael Ellerman saw. This doesn't
yet use a reserved CR field for the soft_enabled and hard_enabled
flags. This applies on top of Stephen Rothwell's patches to make it
possible to build a combined iSeries/other kernel.
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-10-04 10:47:49 +04:00
/* Hard-disable interrupts */
mfmsr r6
rldicl r7 ,r6 ,4 8 ,1
rotldi r7 ,r7 ,1 6
mtmsrd r7 ,1
2005-10-10 16:36:14 +04:00
/ * Unfortunately, t h e s t a c k p o i n t e r a n d t h e M S R a r e a l s o c l o b b e r e d ,
* so t h e y a r e s a v e d i n t h e P A C A w h i c h a l l o w s u s t o r e s t o r e
* our o r i g i n a l s t a t e a f t e r R T A S r e t u r n s .
* /
std r1 ,P A C A R 1 ( r13 )
std r6 ,P A C A S A V E D M S R ( r13 )
/* Setup our real return addr */
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ A D D R ( r4 ,. r t a s _ r e t u r n _ l o c )
clrldi r4 ,r4 ,2 / * c o n v e r t t o r e a l m o d e a d d r e s s * /
2005-10-10 16:36:14 +04:00
mtlr r4
li r0 ,0
ori r0 ,r0 ,M S R _ E E | M S R _ S E | M S R _ B E | M S R _ R I
andc r0 ,r6 ,r0
li r9 ,1
rldicr r9 ,r9 ,M S R _ S F _ L G ,( 6 3 - M S R _ S F _ L G )
ori r9 ,r9 ,M S R _ I R | M S R _ D R | M S R _ F E 0 | M S R _ F E 1 | M S R _ F P
andc r6 ,r0 ,r9
ori r6 ,r6 ,M S R _ R I
sync / * d i s a b l e i n t e r r u p t s s o S R R 0 / 1 * /
mtmsrd r0 / * d o n ' t g e t t r a s h e d * /
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ A D D R ( r4 , r t a s )
2005-10-10 16:36:14 +04:00
ld r5 ,R T A S E N T R Y ( r4 ) / * g e t t h e r t a s - > e n t r y v a l u e * /
ld r4 ,R T A S B A S E ( r4 ) / * g e t t h e r t a s - > b a s e v a l u e * /
mtspr S P R N _ S R R 0 ,r5
mtspr S P R N _ S R R 1 ,r6
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
_ STATIC( r t a s _ r e t u r n _ l o c )
/* relocation is off at this point */
mfspr r4 ,S P R N _ S P R G 3 / * G e t P A C A * /
2006-01-13 06:56:25 +03:00
clrldi r4 ,r4 ,2 / * c o n v e r t t o r e a l m o d e a d d r e s s * /
2005-10-10 16:36:14 +04:00
mfmsr r6
li r0 ,M S R _ R I
andc r6 ,r6 ,r0
sync
mtmsrd r6
ld r1 ,P A C A R 1 ( r4 ) / * R e s t o r e o u r S P * /
2006-01-13 06:56:25 +03:00
LOAD_ R E G _ I M M E D I A T E ( r3 ,. r t a s _ r e s t o r e _ r e g s )
2005-10-10 16:36:14 +04:00
ld r4 ,P A C A S A V E D M S R ( r4 ) / * R e s t o r e o u r M S R * /
mtspr S P R N _ S R R 0 ,r3
mtspr S P R N _ S R R 1 ,r4
rfid
b . / * p r e v e n t s p e c u l a t i v e e x e c u t i o n * /
_ STATIC( r t a s _ r e s t o r e _ r e g s )
/* relocation is on at this point */
REST_ G P R ( 2 , r1 ) / * R e s t o r e t h e T O C * /
REST_ G P R ( 1 3 , r1 ) / * R e s t o r e p a c a * /
REST_ 8 G P R S ( 1 4 , r1 ) / * R e s t o r e t h e n o n - v o l a t i l e s * /
REST_ 1 0 G P R S ( 2 2 , r1 ) / * d i t t o * /
mfspr r13 ,S P R N _ S P R G 3
ld r4 ,_ C C R ( r1 )
mtcr r4
ld r5 ,_ C T R ( r1 )
mtctr r5
ld r6 ,_ X E R ( r1 )
mtspr S P R N _ X E R ,r6
ld r7 ,_ D A R ( r1 )
mtdar r7
ld r8 ,_ D S I S R ( r1 )
mtdsisr r8
ld r9 ,_ S R R 0 ( r1 )
mtsrr0 r9
ld r10 ,_ S R R 1 ( r1 )
mtsrr1 r10
addi r1 ,r1 ,R T A S _ F R A M E _ S I Z E / * U n s t a c k o u r f r a m e * /
ld r0 ,1 6 ( r1 ) / * g e t r e t u r n a d d r e s s * /
mtlr r0
blr / * r e t u r n t o c a l l e r * /
# endif / * C O N F I G _ P P C _ R T A S * /
_ GLOBAL( e n t e r _ p r o m )
mflr r0
std r0 ,1 6 ( r1 )
stdu r1 ,- P R O M _ F R A M E _ S I Z E ( r1 ) / * S a v e S P a n d c r e a t e s t a c k s p a c e * /
/ * Because P R O M i s r u n n i n g i n 3 2 b m o d e , i t c l o b b e r s t h e h i g h o r d e r h a l f
* of a l l r e g i s t e r s t h a t i t s a v e s . W e t h e r e f o r e s a v e t h o s e r e g i s t e r s
* PROM m i g h t t o u c h t o t h e s t a c k . ( r0 , r3 - r13 a r e c a l l e r s a v e d )
* /
SAVE_ 8 G P R S ( 2 , r1 )
SAVE_ G P R ( 1 3 , r1 )
SAVE_ 8 G P R S ( 1 4 , r1 )
SAVE_ 1 0 G P R S ( 2 2 , r1 )
mfcr r4
std r4 ,_ C C R ( r1 )
mfctr r5
std r5 ,_ C T R ( r1 )
mfspr r6 ,S P R N _ X E R
std r6 ,_ X E R ( r1 )
mfdar r7
std r7 ,_ D A R ( r1 )
mfdsisr r8
std r8 ,_ D S I S R ( r1 )
mfsrr0 r9
std r9 ,_ S R R 0 ( r1 )
mfsrr1 r10
std r10 ,_ S R R 1 ( r1 )
mfmsr r11
std r11 ,_ M S R ( r1 )
/* Get the PROM entrypoint */
ld r0 ,G P R 4 ( r1 )
mtlr r0
/ * Switch M S R t o 3 2 b i t s m o d e
* /
mfmsr r11
li r12 ,1
rldicr r12 ,r12 ,M S R _ S F _ L G ,( 6 3 - M S R _ S F _ L G )
andc r11 ,r11 ,r12
li r12 ,1
rldicr r12 ,r12 ,M S R _ I S F _ L G ,( 6 3 - M S R _ I S F _ L G )
andc r11 ,r11 ,r12
mtmsrd r11
isync
/* Restore arguments & enter PROM here... */
ld r3 ,G P R 3 ( r1 )
blrl
/ * Just m a k e s u r e t h a t r1 t o p 3 2 b i t s d i d n ' t g e t
* corrupt b y O F
* /
rldicl r1 ,r1 ,0 ,3 2
/* Restore the MSR (back to 64 bits) */
ld r0 ,_ M S R ( r1 )
mtmsrd r0
isync
/* Restore other registers */
REST_ G P R ( 2 , r1 )
REST_ G P R ( 1 3 , r1 )
REST_ 8 G P R S ( 1 4 , r1 )
REST_ 1 0 G P R S ( 2 2 , r1 )
ld r4 ,_ C C R ( r1 )
mtcr r4
ld r5 ,_ C T R ( r1 )
mtctr r5
ld r6 ,_ X E R ( r1 )
mtspr S P R N _ X E R ,r6
ld r7 ,_ D A R ( r1 )
mtdar r7
ld r8 ,_ D S I S R ( r1 )
mtdsisr r8
ld r9 ,_ S R R 0 ( r1 )
mtsrr0 r9
ld r10 ,_ S R R 1 ( r1 )
mtsrr1 r10
addi r1 ,r1 ,P R O M _ F R A M E _ S I Z E
ld r0 ,1 6 ( r1 )
mtlr r0
blr
