linux/kernel
Paul E. McKenney da123016ca rcu-tasks: Fix computation of CPU-to-list shift counts
The ->percpu_enqueue_shift field is used to map from the running CPU
number to the index of the corresponding callback list.  This mapping
can change at runtime in response to varying callback load, resulting
in varying levels of contention on the callback-list locks.

Unfortunately, the initial value of this field is correct only if the
system happens to have a power-of-two number of CPUs, otherwise the
callbacks from the high-numbered CPUs can be placed into the callback list
indexed by 1 (rather than 0), and those index-1 callbacks will be ignored.
This can result in soft lockups and hangs.

This commit therefore corrects this mapping, adding one to this shift
count as needed for systems having odd numbers of CPUs.

Fixes: 7a30871b6a ("rcu-tasks: Introduce ->percpu_enqueue_shift for dynamic queue selection")
Reported-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Reported-by: Martin Lau <kafai@fb.com>
Cc: Neeraj Upadhyay <neeraj.iitr10@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
2022-01-26 13:04:05 -08:00
..
bpf bpf: Fix ringbuf memory type confusion when passing to helpers 2022-01-19 01:21:46 +01:00
cgroup Merge branch 'for-5.17' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup 2022-01-11 09:14:37 -08:00
configs configs: introduce debug.config for CI-like setup 2022-01-20 08:52:55 +02:00
debug
dma hyperv-next for 5.17 2022-01-16 15:53:00 +02:00
entry
events Peter Zijlstra says: 2022-01-12 16:26:58 -08:00
futex Merge branch 'signal-for-v5.17' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2022-01-17 05:49:30 +02:00
gcov
irq proc: remove PDE_DATA() completely 2022-01-22 08:33:37 +02:00
kcsan KCSAN updates for v5.17 2022-01-11 09:51:26 -08:00
livepatch Livepatching changes for 5.17 2022-01-16 10:08:13 +02:00
locking locking/rwlocks: introduce write_lock_nested 2022-01-22 08:33:37 +02:00
power PM: hibernate: Allow ACPI hardware signature to be honoured 2021-12-08 16:06:10 +01:00
printk printk: fix build warning when CONFIG_PRINTK=n 2022-01-22 08:33:36 +02:00
rcu rcu-tasks: Fix computation of CPU-to-list shift counts 2022-01-26 13:04:05 -08:00
sched Merge branch 'signal-for-v5.17' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2022-01-17 05:49:30 +02:00
time bitmap patches for 5.17-rc1 2022-01-23 06:20:44 +02:00
trace ftrace: Fix s390 breakage from sorting mcount tables 2022-01-23 08:07:02 +02:00
.gitignore
acct.c
async.c
audit.c audit/stable-5.17 PR 20220110 2022-01-11 13:08:21 -08:00
audit.h
audit_fsnotify.c
audit_tree.c audit: use struct_size() helper in kmalloc() 2021-12-14 17:39:42 -05:00
audit_watch.c
auditfilter.c audit/stable-5.17 PR 20220110 2022-01-11 13:08:21 -08:00
auditsc.c
backtracetest.c
bounds.c
capability.c
cfi.c
compat.c
configs.c
context_tracking.c
cpu.c
cpu_pm.c
crash_core.c kernel/crash_core: suppress unknown crashkernel parameter warning 2021-12-25 12:20:55 -08:00
crash_dump.c
cred.c
delayacct.c delayacct: track delays from memory compact 2022-01-20 08:52:55 +02:00
dma.c
exec_domain.c
exit.c exit: Fix the exit_code for wait_task_zombie 2022-01-08 12:43:57 -06:00
extable.c
fail_function.c
fork.c Merge branch 'signal-for-v5.17' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2022-01-17 05:49:30 +02:00
freezer.c
gen_kheaders.sh
groups.c
hung_task.c hung_task: move hung_task sysctl interface to hung_task.c 2022-01-22 08:33:34 +02:00
iomem.c
irq_work.c
jump_label.c
kallsyms.c Livepatching changes for 5.17 2022-01-16 10:08:13 +02:00
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kcov.c
kexec.c
kexec_core.c exit: Move oops specific logic from do_exit into make_task_dead 2021-12-13 12:04:45 -06:00
kexec_elf.c
kexec_file.c
kexec_internal.h
kheaders.c
kmod.c
kprobes.c kprobe: move sysctl_kprobes_optimization to kprobes.c 2022-01-22 08:33:36 +02:00
ksysfs.c
kthread.c Merge branch 'akpm' (patches from Andrew) 2022-01-20 10:41:01 +02:00
latencytop.c
Makefile module: add in-kernel support for decompressing 2022-01-11 18:45:02 -08:00
module-internal.h module: add in-kernel support for decompressing 2022-01-11 18:45:02 -08:00
module.c Merge branch 'modules-next' of git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux 2022-01-17 07:32:51 +02:00
module_decompress.c kernel: Fix spelling mistake "compresser" -> "compressor" 2022-01-13 07:17:47 -08:00
module_signature.c
module_signing.c
notifier.c notifier: Return an error when a callback has already been registered 2021-12-29 10:37:33 +01:00
nsproxy.c
padata.c
panic.c panic: remove oops_id 2022-01-20 08:52:55 +02:00
params.c kobject: remove kset from struct kset_uevent_ops callbacks 2021-12-28 11:26:18 +01:00
pid.c
pid_namespace.c
profile.c exit: Remove profile_handoff_task 2022-01-08 12:43:57 -06:00
ptrace.c ptrace: Remove second setting of PT_SEIZED in ptrace_attach 2022-01-08 12:43:57 -06:00
range.c
reboot.c
regset.c
relay.c
resource.c proc: remove PDE_DATA() completely 2022-01-22 08:33:37 +02:00
resource_kunit.c
rseq.c
scftorture.c scftorture: Always log error message 2021-12-07 16:36:17 -08:00
scs.c
seccomp.c
signal.c Merge branch 'signal-for-v5.17' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2022-01-17 05:49:30 +02:00
smp.c
smpboot.c
smpboot.h
softirq.c
stackleak.c stackleak: move stack_erasing sysctl to stackleak.c 2022-01-22 08:33:35 +02:00
stacktrace.c
static_call.c
stop_machine.c
sys.c Merge branch 'akpm' (patches from Andrew) 2022-01-20 10:41:01 +02:00
sys_ni.c mm/mempolicy: wire up syscall set_mempolicy_home_node 2022-01-15 16:30:30 +02:00
sysctl-test.c
sysctl.c sysctl: returns -EINVAL when a negative value is passed to proc_doulongvec_minmax 2022-01-22 08:33:37 +02:00
task_work.c
taskstats.c
torture.c locktorture,rcutorture,torture: Always log error message 2021-12-07 16:36:17 -08:00
tracepoint.c
tsacct.c taskstats: Cleanup the use of task->exit_code 2022-01-08 12:43:57 -06:00
ucount.c ucounts: Fix rlimit max values check 2021-12-09 15:37:18 -06:00
uid16.c
uid16.h
umh.c
up.c
user-return-notifier.c
user.c
user_namespace.c
usermode_driver.c
utsname.c
utsname_sysctl.c
watch_queue.c
watchdog.c watchdog: move watchdog sysctl interface to watchdog.c 2022-01-22 08:33:34 +02:00
watchdog_hld.c
workqueue.c Merge branch 'workqueue/for-5.16-fixes' into workqueue/for-5.17 2022-01-10 07:54:04 -10:00
workqueue_internal.h