linux/kernel/locking
Waiman Long ffffeaf318 locking/qrwlock: Reduce reader/writer to reader lock transfer latency
Currently, a reader will check first to make sure that the writer mode
byte is cleared before incrementing the reader count. That waiting is
not really necessary. It increases the latency in the reader/writer
to reader transition and reduces readers performance.

This patch eliminates that waiting. It also has the side effect
of reducing the chance of writer lock stealing and improving the
fairness of the lock. Using a locking microbenchmark, a 10-threads 5M
locking loop of mostly readers (RW ratio = 10,000:1) has the following
performance numbers in a Haswell-EX box:

        Kernel          Locking Rate (Kops/s)
        ------          ---------------------
        4.1.1               15,063,081
        4.1.1+patch         17,241,552  (+14.4%)

Signed-off-by: Waiman Long <Waiman.Long@hp.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Douglas Hatch <doug.hatch@hp.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Scott J Norton <scott.norton@hp.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Will Deacon <will.deacon@arm.com>
Link: http://lkml.kernel.org/r/1436459543-29126-2-git-send-email-Waiman.Long@hp.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-08-03 10:57:10 +02:00
..
lglock.c sched/stop_machine: Fix deadlock between multiple stop_two_cpus() 2015-06-19 10:03:12 +02:00
lockdep.c Merge branch 'sched-hrtimers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-06-24 15:09:40 -07:00
lockdep_internals.h lockdep: Increase static allocations 2014-04-18 14:20:50 +02:00
lockdep_proc.c lockdep: Fix a race between /proc/lock_stat and module unload 2015-06-07 15:46:30 +02:00
lockdep_states.h
locktorture.c locktorture: Change longdelay_us to longdelay_ms 2015-05-27 12:59:57 -07:00
Makefile rtmutex: Delete scriptable tester 2015-07-20 11:45:45 +02:00
mcs_spinlock.h locking/qspinlock: Introduce a simple generic 4-byte queued spinlock 2015-05-08 12:36:25 +02:00
mutex-debug.c mutex: Always clear owner field upon mutex_unlock() 2015-01-09 11:20:39 +01:00
mutex-debug.h
mutex.c locking/mutex: Further simplify mutex_spin_on_owner() 2015-04-09 08:10:23 +02:00
mutex.h locking/mutexes: Use MUTEX_SPIN_ON_OWNER when appropriate 2014-08-13 10:32:02 +02:00
osq_lock.c locking: Remove ACCESS_ONCE() usage 2015-02-24 08:44:16 +01:00
percpu-rwsem.c locking: Move the percpu-rwsem code to kernel/locking/ 2013-11-06 09:24:22 +01:00
qrwlock.c locking/qrwlock: Reduce reader/writer to reader lock transfer latency 2015-08-03 10:57:10 +02:00
qspinlock.c locking/pvqspinlock: Implement simple paravirt support for the qspinlock 2015-05-08 12:37:05 +02:00
qspinlock_paravirt.h locking/pvqspinlock: Order pv_unhash() after cmpxchg() on unlock slowpath 2015-08-03 10:57:09 +02:00
rtmutex-debug.c rtmutex: Cleanup deadlock detector debug logic 2014-06-21 22:05:30 +02:00
rtmutex-debug.h rtmutex: Cleanup deadlock detector debug logic 2014-06-21 22:05:30 +02:00
rtmutex.c rtmutex: Delete scriptable tester 2015-07-20 11:45:45 +02:00
rtmutex.h rtmutex: Cleanup deadlock detector debug logic 2014-06-21 22:05:30 +02:00
rtmutex_common.h rtmutex: Delete scriptable tester 2015-07-20 11:45:45 +02:00
rwsem-spinlock.c locking/rwsem: Document barrier need when waking tasks 2015-02-18 16:57:10 +01:00
rwsem-xadd.c locking/rwsem: Reduce spinlock contention in wakeup after up_read()/up_write() 2015-05-08 12:27:59 +02:00
rwsem.c locking/rwsem: Set lock ownership ASAP 2015-02-18 16:57:13 +01:00
rwsem.h locking/rwsem: Set lock ownership ASAP 2015-02-18 16:57:13 +01:00
semaphore.c locking/semaphore: Resolve some shadow warnings 2014-09-04 07:17:24 +02:00
spinlock.c spinlock: Add spin_lock_bh_nested() 2015-01-03 14:32:57 -05:00
spinlock_debug.c locking: Move the spinlock code to kernel/locking/ 2013-11-06 07:55:21 +01:00