Bug 199963

Summary:	UDP rx_queue incorrect calculation in /proc/net/udp
Product:	Networking	Reporter:	trevor.francis
Component:	IPV4	Assignee:	Stephen Hemminger (stephen)
Status:	NEW ---
Severity:	normal	CC:	dsmythies
Priority:	P1
Hardware:	All
OS:	Linux
Kernel Version:	Kernels >= 4.15	Subsystem:
Regression:	No	Bisected commit-id:
Attachments:	Graph of UDP rx queue lengths during iperf test.

Description trevor.francis 2018-06-07 13:21:23 UTC

since upgrading to any kernel >= 4.15 the rx_queue in /proc/net/udp is now reporting a queue, regardless of system load and regardless of what applications are running on it. The tx_queue is always 0, but rx_queue has seemingly random spikes of udp queueing. This is observed across hundreds of servers with either varying or no workload.

netstat -nl|grep ^udp
udp 4352 0 0.0.0.0:68 0.0.0.0:*

sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
14645: 3500007F:0035 00000000:0000 07 00000000:0000C900 00:00000000 00000000 101 0 3367 2 ffff8da177fdcc00 0

Comment 1 Doug Smythies 2018-06-07 18:54:44 UTC

I bisected the kernel and got:

0d4a6608f68c7532dcbfec2ea1150c9761767d03 is the first bad commit
commit 0d4a6608f68c7532dcbfec2ea1150c9761767d03
Author: Paolo Abeni <pabeni@redhat.com>
Date:   Tue Sep 19 12:11:43 2017 +0200

    udp: do rmem bulk free even if the rx sk queue is empty

However, I also found this:
https://www.mail-archive.com/netdev@vger.kernel.org/msg239743.html
and see that Paolo is already on it.

Comment 2 Doug Smythies 2018-06-08 23:12:37 UTC

I am not on the netdev e-mail list, but anyhow saw a patch posted there. I tried it on top of kernel 4.17, and it seems to work. I do not know how to create some real RX queue, so didn't (couldn't) test that.

Reference:
https://marc.info/?l=linux-netdev&m=152845055527577&w=2

Comment 3 Doug Smythies 2018-06-09 19:25:57 UTC

Created attachment 276437 [details]
Graph of UDP rx queue lengths during iperf test.

Using iperf, I was able to creates non-zero reported UDP Rx queues. The graph compares a 5 minute test with Kernel 4.4, 4.17, and 4.17 + the proposed patch. The results are as expected.