mirror of
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
synced 2025-08-05 16:54:27 +00:00
b75eba76d3
Applications with many concurrent connections, high variance in receive queue length and tight memory bounds cannot allocate worst-case buffer size to drain sockets. Knowing the size of receive queue length, applications can optimize how they allocate buffers to read from the socket. The number of bytes pending on the socket is directly available through ioctl(FIONREAD/SIOCINQ) and can be approximated using getsockopt(MEMINFO) (rmem_alloc includes skb overheads in addition to application data). But, both of these options add an extra syscall per recvmsg. Moreover, ioctl(FIONREAD/SIOCINQ) takes the socket lock. Add the TCP_INQ socket option to TCP. When this socket option is set, recvmsg() relays the number of bytes available on the socket for reading to the application via the TCP_CM_INQ control message. Calculate the number of bytes after releasing the socket lock to include the processed backlog, if any. To avoid an extra branch in the hot path of recvmsg() for this new control message, move all cmsg processing inside an existing branch for processing receive timestamps. Since the socket lock is not held when calculating the size of receive queue, TCP_INQ is a hint. For example, it can overestimate the queue size by one byte, if FIN is received. With this method, applications can start reading from the socket using a small buffer, and then use larger buffers based on the remaining data when needed. V3 change-log: As suggested by David Miller, added loads with barrier to check whether we have multiple threads calling recvmsg in parallel. When that happens we lock the socket to calculate inq. V4 change-log: Removed inline from a static function. Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com> Signed-off-by: Yuchung Cheng <ycheng@google.com> Signed-off-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Eric Dumazet <edumazet@google.com> Reviewed-by: Neal Cardwell <ncardwell@google.com> Suggested-by: David Miller <davem@davemloft.net> Signed-off-by: David S. Miller <davem@davemloft.net> |
||
|---|---|---|
| .. | ||
| netfilter | ||
| af_inet.c | ||
| ah4.c | ||
| arp.c | ||
| cipso_ipv4.c | ||
| datagram.c | ||
| devinet.c | ||
| esp4.c | ||
| esp4_offload.c | ||
| fib_frontend.c | ||
| fib_lookup.h | ||
| fib_notifier.c | ||
| fib_rules.c | ||
| fib_semantics.c | ||
| fib_trie.c | ||
| fou.c | ||
| gre_demux.c | ||
| gre_offload.c | ||
| icmp.c | ||
| igmp.c | ||
| inet_connection_sock.c | ||
| inet_diag.c | ||
| inet_fragment.c | ||
| inet_hashtables.c | ||
| inet_timewait_sock.c | ||
| inetpeer.c | ||
| ip_forward.c | ||
| ip_fragment.c | ||
| ip_gre.c | ||
| ip_input.c | ||
| ip_options.c | ||
| ip_output.c | ||
| ip_sockglue.c | ||
| ip_tunnel.c | ||
| ip_tunnel_core.c | ||
| ip_vti.c | ||
| ipcomp.c | ||
| ipconfig.c | ||
| ipip.c | ||
| ipmr.c | ||
| ipmr_base.c | ||
| Kconfig | ||
| Makefile | ||
| metrics.c | ||
| netfilter.c | ||
| ping.c | ||
| proc.c | ||
| protocol.c | ||
| raw.c | ||
| raw_diag.c | ||
| route.c | ||
| syncookies.c | ||
| sysctl_net_ipv4.c | ||
| tcp.c | ||
| tcp_bbr.c | ||
| tcp_bic.c | ||
| tcp_cdg.c | ||
| tcp_cong.c | ||
| tcp_cubic.c | ||
| tcp_dctcp.c | ||
| tcp_diag.c | ||
| tcp_fastopen.c | ||
| tcp_highspeed.c | ||
| tcp_htcp.c | ||
| tcp_hybla.c | ||
| tcp_illinois.c | ||
| tcp_input.c | ||
| tcp_ipv4.c | ||
| tcp_lp.c | ||
| tcp_metrics.c | ||
| tcp_minisocks.c | ||
| tcp_nv.c | ||
| tcp_offload.c | ||
| tcp_output.c | ||
| tcp_rate.c | ||
| tcp_recovery.c | ||
| tcp_scalable.c | ||
| tcp_timer.c | ||
| tcp_ulp.c | ||
| tcp_vegas.c | ||
| tcp_vegas.h | ||
| tcp_veno.c | ||
| tcp_westwood.c | ||
| tcp_yeah.c | ||
| tunnel4.c | ||
| udp.c | ||
| udp_diag.c | ||
| udp_impl.h | ||
| udp_offload.c | ||
| udp_tunnel.c | ||
| udplite.c | ||
| xfrm4_input.c | ||
| xfrm4_mode_beet.c | ||
| xfrm4_mode_transport.c | ||
| xfrm4_mode_tunnel.c | ||
| xfrm4_output.c | ||
| xfrm4_policy.c | ||
| xfrm4_protocol.c | ||
| xfrm4_state.c | ||
| xfrm4_tunnel.c | ||