Íñigo Huguet 09a99ab16c sfc: set affinity hints in local NUMA node only ... Affinity hints were being set to CPUs in local NUMA node first, and then in other CPUs. This was creating 2 unintended issues: 1. Channels created to be assigned each to a different physical core were assigned to hyperthreading siblings because of being in same NUMA node. Since the patch previous to this one, this did not longer happen with default rss_cpus modparam because less channels are created. 2. XDP channels could be assigned to CPUs in different NUMA nodes, decreasing performance too much (to less than half in some of my tests). This patch sets the affinity hints spreading the channels only in local NUMA node's CPUs. A fallback for the case that no CPU in local NUMA node is online has been added too. Example of CPUs being assigned in a non optimal way before this and the previous patch (note: in this system, xdp-8 to xdp-15 are created because num_possible_cpus == 64, but num_present_cpus == 32 so they're never used): $ lscpu | grep -i numa NUMA node(s): 2 NUMA node0 CPU(s): 0-7,16-23 NUMA node1 CPU(s): 8-15,24-31 $ grep -H . /proc/irq/*/0000:07:00.0*/../smp_affinity_list /proc/irq/141/0000:07:00.0-0/../smp_affinity_list:0 /proc/irq/142/0000:07:00.0-1/../smp_affinity_list:1 /proc/irq/143/0000:07:00.0-2/../smp_affinity_list:2 /proc/irq/144/0000:07:00.0-3/../smp_affinity_list:3 /proc/irq/145/0000:07:00.0-4/../smp_affinity_list:4 /proc/irq/146/0000:07:00.0-5/../smp_affinity_list:5 /proc/irq/147/0000:07:00.0-6/../smp_affinity_list:6 /proc/irq/148/0000:07:00.0-7/../smp_affinity_list:7 /proc/irq/149/0000:07:00.0-8/../smp_affinity_list:16 /proc/irq/150/0000:07:00.0-9/../smp_affinity_list:17 /proc/irq/151/0000:07:00.0-10/../smp_affinity_list:18 /proc/irq/152/0000:07:00.0-11/../smp_affinity_list:19 /proc/irq/153/0000:07:00.0-12/../smp_affinity_list:20 /proc/irq/154/0000:07:00.0-13/../smp_affinity_list:21 /proc/irq/155/0000:07:00.0-14/../smp_affinity_list:22 /proc/irq/156/0000:07:00.0-15/../smp_affinity_list:23 /proc/irq/157/0000:07:00.0-xdp-0/../smp_affinity_list:8 /proc/irq/158/0000:07:00.0-xdp-1/../smp_affinity_list:9 /proc/irq/159/0000:07:00.0-xdp-2/../smp_affinity_list:10 /proc/irq/160/0000:07:00.0-xdp-3/../smp_affinity_list:11 /proc/irq/161/0000:07:00.0-xdp-4/../smp_affinity_list:12 /proc/irq/162/0000:07:00.0-xdp-5/../smp_affinity_list:13 /proc/irq/163/0000:07:00.0-xdp-6/../smp_affinity_list:14 /proc/irq/164/0000:07:00.0-xdp-7/../smp_affinity_list:15 /proc/irq/165/0000:07:00.0-xdp-8/../smp_affinity_list:24 /proc/irq/166/0000:07:00.0-xdp-9/../smp_affinity_list:25 /proc/irq/167/0000:07:00.0-xdp-10/../smp_affinity_list:26 /proc/irq/168/0000:07:00.0-xdp-11/../smp_affinity_list:27 /proc/irq/169/0000:07:00.0-xdp-12/../smp_affinity_list:28 /proc/irq/170/0000:07:00.0-xdp-13/../smp_affinity_list:29 /proc/irq/171/0000:07:00.0-xdp-14/../smp_affinity_list:30 /proc/irq/172/0000:07:00.0-xdp-15/../smp_affinity_list:31 CPUs assignments after this and previous patch, so normal channels created only one per core in NUMA node and affinities set only to local NUMA node: $ grep -H . /proc/irq/*/0000:07:00.0*/../smp_affinity_list /proc/irq/116/0000:07:00.0-0/../smp_affinity_list:0 /proc/irq/117/0000:07:00.0-1/../smp_affinity_list:1 /proc/irq/118/0000:07:00.0-2/../smp_affinity_list:2 /proc/irq/119/0000:07:00.0-3/../smp_affinity_list:3 /proc/irq/120/0000:07:00.0-4/../smp_affinity_list:4 /proc/irq/121/0000:07:00.0-5/../smp_affinity_list:5 /proc/irq/122/0000:07:00.0-6/../smp_affinity_list:6 /proc/irq/123/0000:07:00.0-7/../smp_affinity_list:7 /proc/irq/124/0000:07:00.0-xdp-0/../smp_affinity_list:16 /proc/irq/125/0000:07:00.0-xdp-1/../smp_affinity_list:17 /proc/irq/126/0000:07:00.0-xdp-2/../smp_affinity_list:18 /proc/irq/127/0000:07:00.0-xdp-3/../smp_affinity_list:19 /proc/irq/128/0000:07:00.0-xdp-4/../smp_affinity_list:20 /proc/irq/129/0000:07:00.0-xdp-5/../smp_affinity_list:21 /proc/irq/130/0000:07:00.0-xdp-6/../smp_affinity_list:22 /proc/irq/131/0000:07:00.0-xdp-7/../smp_affinity_list:23 /proc/irq/132/0000:07:00.0-xdp-8/../smp_affinity_list:0 /proc/irq/133/0000:07:00.0-xdp-9/../smp_affinity_list:1 /proc/irq/134/0000:07:00.0-xdp-10/../smp_affinity_list:2 /proc/irq/135/0000:07:00.0-xdp-11/../smp_affinity_list:3 /proc/irq/136/0000:07:00.0-xdp-12/../smp_affinity_list:4 /proc/irq/137/0000:07:00.0-xdp-13/../smp_affinity_list:5 /proc/irq/138/0000:07:00.0-xdp-14/../smp_affinity_list:6 /proc/irq/139/0000:07:00.0-xdp-15/../smp_affinity_list:7 Signed-off-by: Íñigo Huguet <ihuguet@redhat.com> Acked-by: Martin Habets <habetsm.xilinx@gmail.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>		2022-03-01 17:12:38 -08:00
..
falcon	Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net	2022-01-05 14:36:10 -08:00
bitfield.h
ef10.c	sfc: The size of the RX recycle ring should be more flexible	2022-02-01 20:34:59 -08:00
ef10_regs.h
ef10_sriov.c	ethernet: constify references to netdev->dev_addr in drivers	2021-10-14 09:22:11 -07:00
ef10_sriov.h	ethernet: constify references to netdev->dev_addr in drivers	2021-10-14 09:22:11 -07:00
ef100.c
ef100.h
ef100_ethtool.c	ethtool: extend ringparam setting/getting API with rx_buf_len	2021-11-22 12:31:49 +00:00
ef100_ethtool.h
ef100_netdev.c
ef100_netdev.h
ef100_nic.c	sfc: The size of the RX recycle ring should be more flexible	2022-02-01 20:34:59 -08:00
ef100_nic.h
ef100_regs.h
ef100_rx.c
ef100_rx.h
ef100_tx.c
ef100_tx.h
efx.c	net: Don't include filter.h from net/sock.h	2021-12-29 08:48:14 -08:00
efx.h
efx_channels.c	sfc: set affinity hints in local NUMA node only	2022-03-01 17:12:38 -08:00
efx_channels.h
efx_common.c	net: Don't include filter.h from net/sock.h	2021-12-29 08:48:14 -08:00
efx_common.h
enum.h
ethtool.c	ethtool: extend ringparam setting/getting API with rx_buf_len	2021-11-22 12:31:49 +00:00
ethtool_common.c	sfc: Fix reading non-legacy supported link modes	2021-10-18 14:28:02 +01:00
ethtool_common.h
farch.c
farch_regs.h
filter.h
io.h
Kconfig
Makefile
mcdi.c
mcdi.h
mcdi_filters.c
mcdi_filters.h
mcdi_functions.c
mcdi_functions.h
mcdi_mon.c
mcdi_pcol.h
mcdi_port.c
mcdi_port.h
mcdi_port_common.c	ethernet/sfc: remove redundant rc variable	2022-01-04 12:41:41 +00:00
mcdi_port_common.h
mtd.c
net_driver.h	sfc: The size of the RX recycle ring should be more flexible	2022-02-01 20:34:59 -08:00
nic.c
nic.h
nic_common.h	sfc: The size of the RX recycle ring should be more flexible	2022-02-01 20:34:59 -08:00
ptp.c	net_tstamp: add new flag HWTSTAMP_FLAG_BONDED_PHC_INDEX	2021-12-14 12:28:24 +00:00
ptp.h
rx.c	bpf: Let bpf_warn_invalid_xdp_action() report more info	2021-12-13 22:28:27 +01:00
rx_common.c	sfc: The size of the RX recycle ring should be more flexible	2022-02-01 20:34:59 -08:00
rx_common.h	sfc: The size of the RX recycle ring should be more flexible	2022-02-01 20:34:59 -08:00
selftest.c
selftest.h
siena.c	sfc: The size of the RX recycle ring should be more flexible	2022-02-01 20:34:59 -08:00
siena_sriov.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net	2021-10-22 11:41:16 +01:00
siena_sriov.h	ethernet: constify references to netdev->dev_addr in drivers	2021-10-14 09:22:11 -07:00
sriov.c
sriov.h
tx.c	sfc: last resort fallback for lack of xdp tx queues	2021-09-09 11:17:37 +01:00
tx.h
tx_common.c
tx_common.h
tx_tso.c
vfdi.h
workarounds.h