6c62985d57
x86/paravirt: Change vcp_is_preempted() arg type to long
...
The cpu argument in the function prototype of vcpu_is_preempted()
is changed from int to long. That makes it easier to provide a better
optimized assembly version of that function.
For Xen, vcpu_is_preempted(long) calls xen_vcpu_stolen(int), the
downcast from long to int is not a problem as vCPU number won't exceed
32 bits.
Signed-off-by: Waiman Long <longman@redhat.com >
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com >
2017-02-21 12:48:06 +01:00
3cded41794
x86/paravirt: Optimize native pv_lock_ops.vcpu_is_preempted()
...
Avoid the pointless function call to pv_lock_ops.vcpu_is_preempted()
when a paravirt spinlock enabled kernel is ran on native hardware.
Do this by patching out the CALL instruction with "XOR %RAX,%RAX"
which has the same effect (0 return value).
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: David.Laight@ACULAB.COM
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Pan Xinhui <xinhui.pan@linux.vnet.ibm.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: benh@kernel.crashing.org
Cc: boqun.feng@gmail.com
Cc: borntraeger@de.ibm.com
Cc: bsingharora@gmail.com
Cc: dave@stgolabs.net
Cc: jgross@suse.com
Cc: kernellwp@gmail.com
Cc: konrad.wilk@oracle.com
Cc: mpe@ellerman.id.au
Cc: paulmck@linux.vnet.ibm.com
Cc: paulus@samba.org
Cc: pbonzini@redhat.com
Cc: rkrcmar@redhat.com
Cc: will.deacon@arm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2016-11-22 12:48:11 +01:00
a6b277857f
locking/qspinlock/x86: Only emit the test-and-set fallback when building guest support
...
Only emit the test-and-set fallback for Hypervisors lacking
PARAVIRT_SPINLOCKS support when building for guests.
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: linux-kernel@vger.kernel.org
Cc: stable@vger.kernel.org # 4.2
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-09-11 07:50:12 +02:00
43b3f02899
locking/qspinlock/x86: Fix performance regression under unaccelerated VMs
...
Dave ran into horrible performance on a VM without PARAVIRT_SPINLOCKS
set and Linus noted that the test-and-set implementation was retarded.
One should spin on the variable with a load, not a RMW.
While there, remove 'queued' from the name, as the lock isn't queued
at all, but a simple test-and-set.
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org >
Reported-by: Dave Chinner <david@fromorbit.com >
Tested-by: Dave Chinner <david@fromorbit.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: Waiman Long <Waiman.Long@hp.com >
Cc: stable@vger.kernel.org # v4.2+
Link: http://lkml.kernel.org/r/20150904152523.GR18673@twins.programming.kicks-ass.net
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-09-11 07:49:42 +02:00
f233f7f158
locking/pvqspinlock, x86: Implement the paravirt qspinlock call patching
...
We use the regular paravirt call patching to switch between:
native_queued_spin_lock_slowpath() __pv_queued_spin_lock_slowpath()
native_queued_spin_unlock() __pv_queued_spin_unlock()
We use a callee saved call for the unlock function which reduces the
i-cache footprint and allows 'inlining' of SPIN_UNLOCK functions
again.
We further optimize the unlock path by patching the direct call with a
"movb $0,%arg1" if we are indeed using the native unlock code. This
makes the unlock code almost as fast as the !PARAVIRT case.
This significantly lowers the overhead of having
CONFIG_PARAVIRT_SPINLOCKS enabled, even for native code.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Signed-off-by: Waiman Long <Waiman.Long@hp.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: Andrew Morton <akpm@linux-foundation.org >
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com >
Cc: Borislav Petkov <bp@alien8.de >
Cc: Daniel J Blueman <daniel@numascale.com >
Cc: David Vrabel <david.vrabel@citrix.com >
Cc: Douglas Hatch <doug.hatch@hp.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Paolo Bonzini <paolo.bonzini@gmail.com >
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com >
Cc: Rik van Riel <riel@redhat.com >
Cc: Scott J Norton <scott.norton@hp.com >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: virtualization@lists.linux-foundation.org
Cc: xen-devel@lists.xenproject.org
Link: http://lkml.kernel.org/r/1429901803-29771-10-git-send-email-Waiman.Long@hp.com
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-08 12:37:09 +02:00
2aa79af642
locking/qspinlock: Revert to test-and-set on hypervisors
...
When we detect a hypervisor (!paravirt, see qspinlock paravirt support
patches), revert to a simple test-and-set lock to avoid the horrors
of queue preemption.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Signed-off-by: Waiman Long <Waiman.Long@hp.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: Andrew Morton <akpm@linux-foundation.org >
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com >
Cc: Borislav Petkov <bp@alien8.de >
Cc: Daniel J Blueman <daniel@numascale.com >
Cc: David Vrabel <david.vrabel@citrix.com >
Cc: Douglas Hatch <doug.hatch@hp.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Paolo Bonzini <paolo.bonzini@gmail.com >
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com >
Cc: Rik van Riel <riel@redhat.com >
Cc: Scott J Norton <scott.norton@hp.com >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: virtualization@lists.linux-foundation.org
Cc: xen-devel@lists.xenproject.org
Link: http://lkml.kernel.org/r/1429901803-29771-8-git-send-email-Waiman.Long@hp.com
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-08 12:36:58 +02:00
d73a33973f
locking/qspinlock, x86: Enable x86-64 to use queued spinlocks
...
This patch makes the necessary changes at the x86 architecture
specific layer to enable the use of queued spinlocks for x86-64. As
x86-32 machines are typically not multi-socket. The benefit of queue
spinlock may not be apparent. So queued spinlocks are not enabled.
Currently, there is some incompatibilities between the para-virtualized
spinlock code (which hard-codes the use of ticket spinlock) and the
queued spinlocks. Therefore, the use of queued spinlocks is disabled
when the para-virtualized spinlock is enabled.
The arch/x86/include/asm/qspinlock.h header file includes some x86
specific optimization which will make the queueds spinlock code
perform better than the generic implementation.
Signed-off-by: Waiman Long <Waiman.Long@hp.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: Andrew Morton <akpm@linux-foundation.org >
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com >
Cc: Borislav Petkov <bp@alien8.de >
Cc: Daniel J Blueman <daniel@numascale.com >
Cc: David Vrabel <david.vrabel@citrix.com >
Cc: Douglas Hatch <doug.hatch@hp.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Paolo Bonzini <paolo.bonzini@gmail.com >
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com >
Cc: Rik van Riel <riel@redhat.com >
Cc: Scott J Norton <scott.norton@hp.com >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: virtualization@lists.linux-foundation.org
Cc: xen-devel@lists.xenproject.org
Link: http://lkml.kernel.org/r/1429901803-29771-3-git-send-email-Waiman.Long@hp.com
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-08 12:36:26 +02:00
