Bug 205061

Summary: iwlwifi: ax200 : SYSASSERT 103C when UDP UL is more than 120Mbps
Product: Drivers Reporter: Ivan Klokov (iklokov)
Component: network-wirelessAssignee: DO NOT USE - assign "network-wireless-intel" component instead (linuxwifi)
Status: CLOSED CODE_FIX    
Severity: normal CC: a3at.mail, greearb, kagurazakakotori, linuxwifi, luca, robert.carter
Priority: P1    
Hardware: Intel   
OS: Linux   
Kernel Version: Linux 5.2.11-050211-generic #201908290731 SMP Thu Aug 29 07:33:42 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux19 x86_64 x86_64 x86_64 GNU/Linux Subsystem:
Regression: No Bisected commit-id:
Attachments: kernel dmesg log
dmesg after crash. UDP upstream, 4 parallel client streams

Description Ivan Klokov 2019-10-01 09:30:21 UTC
Created attachment 285273 [details]
kernel dmesg log

We are testing ax200ngw using iperf. When iperf generates 100Mbps UDP uplink, then all looks good. However when UDP UL traffic is 120Mbps, firwmare crashes. Here is an iperf command: 
$ iperf -i 0.5 -u -y c -w 1M -c 192.168.1.21 -t 60 -b 120M -e -p 10001

Firmware version:
$ ethtool -i wlo1 | grep firmware
firmware-version: 48.4fa0041f.0

Kernel log is attached.
Note that the issue can be reproduced with both 11ac and 11ax AP.

Best regards,
Ivan
Comment 1 robert.carter 2019-10-09 18:23:23 UTC
Created attachment 285435 [details]
dmesg after crash. UDP upstream, 4  parallel client streams

I get a slightly different assert code, 0xEDC:

  Loaded firmware version: 48.4fa0041f.0
  0x00000EDC | ADVANCED_SYSASSERT
Comment 2 Luca Coelho 2019-11-20 19:41:49 UTC
Today I found a bug in the driver that causes problems during high TX throughput (or rather, high number of frames that would cause mac80211 to use AMSDU).  Can you please try this patch and see if it helps with the issue you are experiencing?

https://patchwork.kernel.org/patch/11253471/
Comment 3 robert.carter 2019-11-21 20:23:14 UTC
Initial upstream and downstream throughput testing suggests this patch resolves both the driver crash and poor throughput performance. I tested the patch in the 5.2.x kernel.
Comment 4 Luca Coelho 2019-11-23 08:04:32 UTC
Great! Thanks for testing! I'm closing this bug, please reopen it if you experience the same problem again (or open a new one if appropriate).
Comment 5 Luca Coelho 2019-11-25 07:49:26 UTC
*** Bug 205109 has been marked as a duplicate of this bug. ***
Comment 6 robert.carter 2019-11-25 22:23:05 UTC
Further testing shows that while the patch fixes the firmware crash, UDP downlink (AP to AX200 STA) under high loads is stalling. At UDP rates of >~400 Mbps, the driver appears stops receiving, as shown by RX packet counters in "iw dev wlp3s0 station dump". Was there more changes dropped during the merge crash? I'll open a new bug for this issue.
Comment 7 Luca Coelho 2019-12-16 08:04:44 UTC
Robert, thanks for the input.  We have other bugs reported on high throughput issues, can you check the open bugs and see if any relates to the one you're seeing? It will be easier to follow on a different issues, since it's not the same as the one solved in this report.  Thanks!