Joel Fernandes (Google) bd56e0a4a2 rcu/tree: nocb: Avoid raising softirq for offloaded ready-to-execute CBs ... Testing showed that rcu_pending() can return 1 when offloaded callbacks are ready to execute. This invokes RCU core processing, for example, by raising RCU_SOFTIRQ, eventually resulting in a call to rcu_core(). However, rcu_core() explicitly avoids in any way manipulating offloaded callbacks, which are instead handled by the rcuog and rcuoc kthreads, which work independently of rcu_core(). One exception to this independence is that rcu_core() invokes do_nocb_deferred_wakeup(), however, rcu_pending() also checks rcu_nocb_need_deferred_wakeup() in order to correctly handle this case, invoking rcu_core() when needed. This commit therefore avoids needlessly invoking RCU core processing by checking rcu_segcblist_ready_cbs() only on non-offloaded CPUs. This reduces overhead, for example, by reducing softirq activity. This change passed 30 minute tests of TREE01 through TREE09 each. On TREE08, there is at most 150us from the time that rcu_pending() chose not to invoke RCU core processing to the time when the ready callbacks were invoked by the rcuoc kthread. This provides further evidence that there is no need to invoke rcu_core() for offloaded callbacks that are ready to invoke. Cc: Neeraj Upadhyay <neeraju@codeaurora.org> Reviewed-by: Frederic Weisbecker <frederic@kernel.org> Reviewed-by: Neeraj Upadhyay <neeraju@codeaurora.org> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org> Signed-off-by: Paul E. McKenney <paulmck@kernel.org>		2020-11-19 19:37:17 -08:00
..
bpf	Fixes for 5.10-rc1 from the networking tree:	2020-10-23 12:05:49 -07:00
cgroup	kernel/: fix repeated words in comments	2020-10-16 11:11:19 -07:00
configs
debug	kdb: Fix pager search for multi-line strings	2020-10-01 14:44:08 +01:00
dma	dma-mapping: move more functions to dma-map-ops.h	2020-10-20 10:41:07 +02:00
entry	arch-cleanup-2020-10-22	2020-10-23 10:06:38 -07:00
events	task_work: cleanup notification modes	2020-10-17 15:05:30 -06:00
gcov	gcov: add support for GCC 10.1	2020-09-11 09:33:54 -07:00
irq	task_work: cleanup notification modes	2020-10-17 15:05:30 -06:00
kcsan	kernel/: fix repeated words in comments	2020-10-16 11:11:19 -07:00
livepatch	kernel/: fix repeated words in comments	2020-10-16 11:11:19 -07:00
locking	Merge tag 'core-rcu-2020-10-12' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip	2020-10-18 14:34:50 -07:00
power	kernel/: fix repeated words in comments	2020-10-16 11:11:19 -07:00
printk	Urgent printk fix for 5.10	2020-10-16 12:52:37 -07:00
rcu	rcu/tree: nocb: Avoid raising softirq for offloaded ready-to-execute CBs	2020-11-19 19:37:17 -08:00
sched	treewide: Convert macro and uses of __section(foo) to __section("foo")	2020-10-25 14:51:49 -07:00
time	random32: add noise from network and scheduling activity	2020-10-24 20:21:57 +02:00
trace	treewide: Convert macro and uses of __section(foo) to __section("foo")	2020-10-25 14:51:49 -07:00
.gitignore
acct.c	kernel: acct.c: fix some kernel-doc nits	2020-10-16 11:11:19 -07:00
async.c
audit.c	audit: Remove redundant null check	2020-08-26 09:10:39 -04:00
audit.h	audit: change unnecessary globals into statics	2020-08-17 20:26:58 -04:00
audit_fsnotify.c
audit_tree.c
audit_watch.c
auditfilter.c	treewide: Use fallthrough pseudo-keyword	2020-08-23 17:36:59 -05:00
auditsc.c
backtracetest.c
bounds.c
capability.c	LSM: Signal to SafeSetID when setting group IDs	2020-10-13 09:17:34 -07:00
compat.c	treewide: Use fallthrough pseudo-keyword	2020-08-23 17:36:59 -05:00
configs.c
context_tracking.c
cpu.c
cpu_pm.c	notifier: Fix broken error handling pattern	2020-09-01 09:58:03 +02:00
crash_core.c
crash_dump.c
cred.c
delayacct.c
dma.c
elfcore.c
exec_domain.c
exit.c	pid: move pidfd_get_pid() to pid.c	2020-10-18 09:27:10 -07:00
extable.c
fail_function.c
fork.c	kernel/: fix repeated words in comments	2020-10-16 11:11:19 -07:00
freezer.c
futex.c	futex: Adjust absolute futex timeouts with per time namespace offset	2020-10-20 17:02:57 +02:00
gen_kheaders.sh
groups.c	LSM: Signal to SafeSetID when setting group IDs	2020-10-13 09:17:34 -07:00
hung_task.c
iomem.c
irq_work.c
jump_label.c	kernel/: fix repeated words in comments	2020-10-16 11:11:19 -07:00
kallsyms.c	treewide: Convert macro and uses of __section(foo) to __section("foo")	2020-10-25 14:51:49 -07:00
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kcov.c
kexec.c	LSM: Introduce kernel_post_load_data() hook	2020-10-05 13:37:03 +02:00
kexec_core.c	kernel/: fix repeated words in comments	2020-10-16 11:11:19 -07:00
kexec_elf.c
kexec_file.c	kernel/resource: move and rename IORESOURCE_MEM_DRIVER_MANAGED	2020-10-16 11:11:18 -07:00
kexec_internal.h
kheaders.c
kmod.c
kprobes.c	Updates for tracing and bootconfig:	2020-10-15 15:51:28 -07:00
ksysfs.c
kthread.c	kernel/: fix repeated words in comments	2020-10-16 11:11:19 -07:00
latencytop.c
Makefile	Kbuild updates for v5.10	2020-10-22 13:13:57 -07:00
module-internal.h
module.c	Modules updates for v5.10	2020-10-22 13:08:57 -07:00
module_signature.c
module_signing.c
notifier.c	notifier: Fix broken error handling pattern	2020-09-01 09:58:03 +02:00
nsproxy.c
padata.c	padata: fix possible padata_works_lock deadlock	2020-09-04 17:51:55 +10:00
panic.c	panic: dump registers on panic_on_warn	2020-10-16 11:11:22 -07:00
params.c
pid.c	pid: move pidfd_get_pid() to pid.c	2020-10-18 09:27:10 -07:00
pid_namespace.c	kernel/: fix repeated words in comments	2020-10-16 11:11:19 -07:00
profile.c
ptrace.c
range.c	kernel.h: split out min()/max() et al. helpers	2020-10-16 11:11:19 -07:00
reboot.c
regset.c
relay.c	kernel/relay.c: drop unneeded initialization	2020-10-16 11:11:22 -07:00
resource.c	kernel/resource: make iomem_resource implicit in release_mem_region_adjustable()	2020-10-16 11:11:18 -07:00
rseq.c
scftorture.c	scftorture: Add cond_resched() to test loop	2020-08-24 18:38:38 -07:00
scs.c
seccomp.c	seccomp: Make duplicate listener detection non-racy	2020-10-08 13:17:47 -07:00
signal.c	treewide: Use fallthrough pseudo-keyword	2020-08-23 17:36:59 -05:00
smp.c	Merge tag 'core-rcu-2020-10-12' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip	2020-10-18 14:34:50 -07:00
smpboot.c
smpboot.h
softirq.c	softirq: Add debug check to __raise_softirq_irqoff()	2020-09-16 15:18:56 +02:00
stackleak.c	stackleak: let stack_erasing_sysctl take a kernel pointer buffer	2020-09-19 13:13:39 -07:00
stacktrace.c	stacktrace: Remove reliable argument from arch_stack_walk() callback	2020-09-18 14:24:16 +01:00
static_call.c	static_call: Fix return type of static_call_init	2020-10-02 21:18:25 +02:00
stop_machine.c
sys.c	kernel/sys.c: fix prototype of prctl_get_tid_address()	2020-10-25 11:44:16 -07:00
sys_ni.c	mm/madvise: introduce process_madvise() syscall: an external memory hinting API	2020-10-18 09:27:10 -07:00
sysctl-test.c
sysctl.c	rcu: Panic after fixed number of stalls	2020-11-19 19:37:16 -08:00
task_work.c	task_work: cleanup notification modes	2020-10-17 15:05:30 -06:00
taskstats.c	taskstats: move specifying netlink policy back to ops	2020-10-02 19:11:12 -07:00
test_kprobes.c
torture.c
tracepoint.c	tracepoint: Fix out of sync data passing by static caller	2020-10-02 21:18:25 +02:00
tsacct.c
ucount.c
uid16.c
uid16.h
umh.c	usermodehelper: reset umask to default before executing user process	2020-10-06 10:31:52 -07:00
up.c
user-return-notifier.c
user.c
user_namespace.c	kernel/: fix repeated words in comments	2020-10-16 11:11:19 -07:00
usermode_driver.c
utsname.c
utsname_sysctl.c
watch_queue.c	watch_queue: Limit the number of watches a user can hold	2020-08-17 09:39:18 -07:00
watchdog.c
watchdog_hld.c
workqueue.c	workqueue: fix a kernel-doc warning	2020-10-16 07:28:20 +02:00
workqueue_internal.h