linux/kernel/rcu
Joel Fernandes (Google) bd56e0a4a2 rcu/tree: nocb: Avoid raising softirq for offloaded ready-to-execute CBs
Testing showed that rcu_pending() can return 1 when offloaded callbacks
are ready to execute.  This invokes RCU core processing, for example,
by raising RCU_SOFTIRQ, eventually resulting in a call to rcu_core().
However, rcu_core() explicitly avoids in any way manipulating offloaded
callbacks, which are instead handled by the rcuog and rcuoc kthreads,
which work independently of rcu_core().

One exception to this independence is that rcu_core() invokes
do_nocb_deferred_wakeup(), however, rcu_pending() also checks
rcu_nocb_need_deferred_wakeup() in order to correctly handle this case,
invoking rcu_core() when needed.

This commit therefore avoids needlessly invoking RCU core processing
by checking rcu_segcblist_ready_cbs() only on non-offloaded CPUs.
This reduces overhead, for example, by reducing softirq activity.

This change passed 30 minute tests of TREE01 through TREE09 each.

On TREE08, there is at most 150us from the time that rcu_pending() chose
not to invoke RCU core processing to the time when the ready callbacks
were invoked by the rcuoc kthread.  This provides further evidence that
there is no need to invoke rcu_core() for offloaded callbacks that are
ready to invoke.

Cc: Neeraj Upadhyay <neeraju@codeaurora.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Reviewed-by: Neeraj Upadhyay <neeraju@codeaurora.org>
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
2020-11-19 19:37:17 -08:00
..
Kconfig rcu: Clarify nocb kthreads naming in RCU_NOCB_CPU config 2020-11-19 19:37:16 -08:00
Kconfig.debug Merge branch 'strictgp.2020.08.24a' into HEAD 2020-09-03 09:47:42 -07:00
Makefile rcuperf: Change rcuperf to rcuscale 2020-08-24 18:39:24 -07:00
rcu.h treewide: Make all debug_obj_descriptors const 2020-09-24 21:56:25 +02:00
rcu_segcblist.c rcu/segcblist: Prevent useless GP start if no CBs to accelerate 2020-09-03 09:39:59 -07:00
rcu_segcblist.h rcu: Implement rcu_segcblist_is_offloaded() config dependent 2020-11-19 19:37:16 -08:00
rcuscale.c rcuperf: Change rcuperf to rcuscale 2020-08-24 18:39:24 -07:00
rcutorture.c rcutorture: Allow pointer leaks to test diagnostic code 2020-08-24 18:45:36 -07:00
refscale.c refperf: Avoid null pointer dereference when buf fails to allocate 2020-08-24 18:45:35 -07:00
srcutiny.c rcu: Use CONFIG_PREEMPTION where appropriate 2019-12-09 12:37:51 -08:00
srcutree.c srcu: Remove KCSAN stubs 2020-08-24 18:36:03 -07:00
sync.c rcu/sync: Simplify the state machine 2019-05-28 09:05:23 -07:00
tasks.h rcu-tasks: Enclose task-list scan in rcu_read_lock() 2020-09-16 16:32:38 -07:00
tiny.c rcu: Rename *_kfree_callback/*_kfree_rcu_offset/kfree_call_* 2020-06-29 11:59:25 -07:00
tree.c rcu/tree: nocb: Avoid raising softirq for offloaded ready-to-execute CBs 2020-11-19 19:37:17 -08:00
tree.h rcu: Fix single-CPU check in rcu_blocking_is_gp() 2020-11-19 19:37:16 -08:00
tree_exp.h rcu: Initialize at declaration time in rcu_exp_handler() 2020-08-24 18:36:03 -07:00
tree_plugin.h Merge branch 'strictgp.2020.08.24a' into HEAD 2020-09-03 09:47:42 -07:00
tree_stall.h rcu: Panic after fixed number of stalls 2020-11-19 19:37:16 -08:00
update.c Merge tag 'core-rcu-2020-10-12' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2020-10-18 14:34:50 -07:00