Bug 217034
Summary: | rtw88: LM842 / rtw8822cu: nmcli device wifi rescan -> rtw_8822cu 1-1:1.2: timed out to flush queue 3 | ||
---|---|---|---|
Product: | Networking | Reporter: | Andreas Henriksson (andreas) |
Component: | Wireless | Assignee: | networking_wireless (networking_wireless) |
Status: | NEW --- | ||
Severity: | normal | CC: | pbrobinson, petter, pkshih |
Priority: | P1 | ||
Hardware: | All | ||
OS: | Linux | ||
Kernel Version: | 6.1.7 | Subsystem: | |
Regression: | No | Bisected commit-id: |
Description
Andreas Henriksson
2023-02-13 15:59:31 UTC
Upgraded to firmware 9.9.14 from https://github.com/pkshih/linux-firmware/raw/main/rtw88/rtw8822c_fw.bin (see https://lore.kernel.org/linux-wireless/95bf72c328de6450d8e0136317ea849830e4f67e.camel@realtek.com/ ) and now get this behaviour instead: ``` $ ping -n -c 3 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=55 time=25.2 ms 64 bytes from 8.8.8.8: icmp_seq=2 ttl=55 time=15.6 ms 64 bytes from 8.8.8.8: icmp_seq=3 ttl=55 time=16.4 ms --- 8.8.8.8 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2006ms rtt min/avg/max/mdev = 15.593/19.072/25.235/4.369 ms $ nmcli c show --active NAME UUID TYPE DEVICE ubnt5g 8b2fcc88-3c8d-4917-897b-5379edfa556f wifi wlx34c9f08deb6e $ sudo nmcli dev wifi rescan [ 521.790095] rtw_8822cu 1-1:1.2: timed out to flush queue 3 [ 527.756301] rtw_8822cu 1-1:1.2: failed to get tx report from firmware $ ping -n -c 3 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. From 192.168.1.112 icmp_seq=1 Destination Host Unreachable From 192.168.1.112 icmp_seq=2 Destination Host Unreachable From 192.168.1.112 icmp_seq=3 Destination Host Unreachable --- 8.8.8.8 ping statistics --- 3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2047ms pipe 3 [ 558.186433] rtw_8822cu 1-1:1.2: failed to get tx report from firmware $ nmcli c show --active NAME UUID TYPE DEVICE ubnt5g 8b2fcc88-3c8d-4917-897b-5379edfa556f wifi wlx34c9f08deb6e ``` Conclusion: updating from firmware 9.9.10 to 9.9.14 made the problem worse! Since interference could affect TX, please try to disable EDCCA via debugfs: $ echo 0 > /sys/kernel/debug/ieee80211/phy0/rtw88/edcca_enable I can see same issue as Andreas. When using 9.9.15 it not only give me the error about tx report, the interfaces stops working apparently (see crash below after some time): I have tried to disable EDCCA and it doesn't seem to matter. The "timed out to flush queue 3" is not present if I run with Sascha's WIP commit (https://lkml.kernel.org/linux-wireless/20230404072508.578056-2-s.hauer@pengutronix.de/T/#u) but the consequence is the same.. # cat /sys/kernel/debug/ieee80211/phy0/rtw88/edcca_enable EDCCA disabled: EDCCA mode 0 # nmcli device wifi rescan # [ 752.958662] rtw_8822cu 1-1:1.2: failed to get tx report from firmware # ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. ^C --- 8.8.8.8 ping statistics --- 2 packets transmitted, 0 received, 100% packet loss, time 1081ms # [ 1012.899698] wlx34c9f08deb60: deauthenticated from d8:07:b6:83:ae:57 (Reason: 34=DISASSOC_LOW_ACK) [ 1013.295268] rtw_8822cu 1-1:1.2: timed out to flush queue 1 [ 1013.515490] rtw_8822cu 1-1:1.2: timed out to flush queue 2 [ 1013.846358] rtw_8822cu 1-1:1.2: timed out to flush queue 1 [ 1014.075720] rtw_8822cu 1-1:1.2: timed out to flush queue 2 [ 1015.012322] rtw_8822cu 1-1:1.2: failed to get tx report from firmware [ 1017.526872] rtw_8822cu 1-1:1.2: error beacon valid [ 1017.534740] rtw_8822cu 1-1:1.2: Download probe request to firmware failed [ 1017.542307] rtw_8822cu 1-1:1.2: Update probe request failed [ 1017.609953] rtw_8822cu 1-1:1.2: HW scan failed with status: -16 [ 1018.953131] rtw_8822cu 1-1:1.2: error beacon valid [ 1018.959368] rtw_8822cu 1-1:1.2: failed to download rsvd page [ 1018.973283] rtw_8822cu 1-1:1.2: failed to download firmware [ 1018.998986] rtw_8822cu 1-1:1.2: leave idle state failed [ 1019.054864] rtw_8822cu 1-1:1.2: failed to leave ips state [ 1019.060422] rtw_8822cu 1-1:1.2: failed to leave idle state [ 1033.143599] ------------[ cut here ]------------ [ 1033.148854] WARNING: CPU: 0 PID: 288 at net/wireless/core.c:1463 cfg80211_netdev_notifier_call+0x3f0/0x754 [ 1033.159033] Modules linked in: nf_tables libcrc32c nfnetlink rtw88_8822cu rtw88_8822c btusb btrtl rtw88_usb btbcm btintel btmtk rtw88_core imx_sdma ip_tables x_tables [ 1033.174622] CPU: 0 PID: 288 Comm: NetworkManager Not tainted 6.1.7-00082-ga92091912f99 #1 [ 1033.182832] Hardware name: Freescale i.MX6 SoloX (Device Tree) [ 1033.188688] unwind_backtrace from show_stack+0x10/0x14 [ 1033.193964] show_stack from dump_stack_lvl+0x58/0x70 [ 1033.199052] dump_stack_lvl from __warn+0xb4/0x24c [ 1033.203877] __warn from warn_slowpath_fmt+0x74/0xbc [ 1033.208870] warn_slowpath_fmt from cfg80211_netdev_notifier_call+0x3f0/0x754 [ 1033.216045] cfg80211_netdev_notifier_call from raw_notifier_call_chain+0x34/0x68 [ 1033.223573] raw_notifier_call_chain from __dev_notify_flags+0x64/0xe8 [ 1033.230137] __dev_notify_flags from dev_change_flags+0x3c/0x44 [ 1033.236085] dev_change_flags from do_setlink+0x28c/0xe90 [ 1033.241515] do_setlink from rtnl_newlink+0x53c/0x91c [ 1033.246594] rtnl_newlink from rtnetlink_rcv_msg+0x170/0x540 [ 1033.252279] rtnetlink_rcv_msg from netlink_rcv_skb+0xbc/0x120 [ 1033.258148] netlink_rcv_skb from netlink_unicast+0x1ac/0x2e8 [ 1033.263929] netlink_unicast from netlink_sendmsg+0x1c4/0x43c [ 1033.269709] netlink_sendmsg from ____sys_sendmsg+0x184/0x254 [ 1033.275492] ____sys_sendmsg from ___sys_sendmsg+0x6c/0xa4 [ 1033.281011] ___sys_sendmsg from sys_sendmsg+0x50/0x98 [ 1033.286181] sys_sendmsg from ret_fast_syscall+0x0/0x1c [ 1033.291437] Exception stack(0xf0fd9fa8 to 0xf0fd9ff0) [ 1033.296509] 9fa0: 00000000 beae6ea8 0000000d beae6ea8 00000000 00000000 [ 1033.304706] 9fc0: 00000000 beae6ea8 b5e2f4d0 00000128 00000000 0067a000 beae7000 b4901848 [ 1033.312899] 9fe0: 00000128 beae6e78 b69202c9 b6921676 [ 1033.318676] irq event stamp: 1038571 [ 1033.322440] hardirqs last enabled at (1038595): [<c019674c>] __up_console_sem+0x50/0x60 [ 1033.330580] hardirqs last disabled at (1038602): [<c0196738>] __up_console_sem+0x3c/0x60 [ 1033.338798] softirqs last enabled at (1038592): [<c01017b4>] __do_softirq+0x2a4/0x5fc [ 1033.346886] softirqs last disabled at (1038579): [<c012ec04>] __irq_exit_rcu+0x138/0x17c [ 1033.355103] ---[ end trace 0000000000000000 ]--- [ 1033.739995] rtw_8822cu 1-1:1.2: error beacon valid [ 1033.745988] rtw_8822cu 1-1:1.2: failed to download rsvd page [ 1033.763797] rtw_8822cu 1-1:1.2: failed to download firmware Sascha has a patch similar to this. Could you give it a try: [1] https://lore.kernel.org/linux-wireless/20230404072508.578056-1-s.hauer@pengutronix.de/T/#m54b7c8c604b91cfce470fcec8fc7d4c20f3056c9 (In reply to Ping-Ke Shih from comment #4) > Sascha has a patch similar to this. Could you give it a try: > > > [1] > https://lore.kernel.org/linux-wireless/20230404072508.578056-1-s. > hauer@pengutronix.de/T/#m54b7c8c604b91cfce470fcec8fc7d4c20f3056c9 Yes, like mentioned in my latest comment I have tried using patch in [2], but still get the same problem (besides that the flooding of "timed out to flush queue 2" don't appear anymore). Any ideas what might cause this? In below thread [1] you can see another problem that also among other things causes the "failed to get tx report from firmware" to appear. [1] https://lkml.kernel.org/linux-wireless/188007.70407276-sendEmail@petterwork [2] https://lore.kernel.org/linux-wireless/20230404072508.578056-1-s.hauer@pengutronix.de/T/#m54b7c8c604b91cfce470fcec8fc7d4c20f3056c9 Ping-Ke Shih, or someone else. Any new about this? Still have problems with this. More or less each time I run rescan the driver stop working. ~# ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=119 time=14.4 ms 64 bytes from 8.8.8.8: icmp_seq=2 ttl=119 time=15.1 ms ^C --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 14.377/14.746/15.116/0.369 ms ~# sudo nmcli dev wifi rescan [ 5359.184667] rtw_8822cu 1-1:1.2: failed to get tx report from firmware ~# ping 8.8.8.8 <-- stops working PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. ^C --- 8.8.8.8 ping statistics --- 11 packets transmitted, 0 received, 100% packet loss, time 10407ms ~# ip addr ... wlx34c9f08deb60 valid_lft 7159sec preferred_lft 7159sec inet6 fe80::3bc3:2caf:608a:2e32/64 scope link noprefixroute valid_lft forever preferred_lft forever ... [ 5389.664909] rtw_8822cu 1-1:1.2: failed to get tx report from firmware (In reply to Petter Mabäcker from comment #6) > Ping-Ke Shih, or someone else. Any new about this? Still have problems with > this. More or less each time I run rescan the driver stop working. > > ~# ping 8.8.8.8 > PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. > 64 bytes from 8.8.8.8: icmp_seq=1 ttl=119 time=14.4 ms > 64 bytes from 8.8.8.8: icmp_seq=2 ttl=119 time=15.1 ms > ^C > --- 8.8.8.8 ping statistics --- > 2 packets transmitted, 2 received, 0% packet loss, time 1001ms > rtt min/avg/max/mdev = 14.377/14.746/15.116/0.369 ms > > ~# sudo nmcli dev wifi rescan > [ 5359.184667] rtw_8822cu 1-1:1.2: failed to get tx report from firmware > > ~# ping 8.8.8.8 <-- stops working > PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. > > ^C > --- 8.8.8.8 ping statistics --- > 11 packets transmitted, 0 received, 100% packet loss, time 10407ms > > ~# ip addr > ... > wlx34c9f08deb60 > valid_lft 7159sec preferred_lft 7159sec > inet6 fe80::3bc3:2caf:608a:2e32/64 scope link noprefixroute > valid_lft forever preferred_lft forever > ... > [ 5389.664909] rtw_8822cu 1-1:1.2: failed to get tx report from firmware Just a note. From what I can tell, this seems to be similar to the situation in [1], where it seems to work fine on a more powerful machine such as i.MX8/RPi4 but I get problems on slower single core machines such as i.MX6 SoloX etc. Hope this can help in troubleshooting the issue. (In reply to Petter Mabäcker from comment #7) > (In reply to Petter Mabäcker from comment #6) > > Ping-Ke Shih, or someone else. Any new about this? Still have problems with > > this. More or less each time I run rescan the driver stop working. > > > > ~# ping 8.8.8.8 > > PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. > > 64 bytes from 8.8.8.8: icmp_seq=1 ttl=119 time=14.4 ms > > 64 bytes from 8.8.8.8: icmp_seq=2 ttl=119 time=15.1 ms > > ^C > > --- 8.8.8.8 ping statistics --- > > 2 packets transmitted, 2 received, 0% packet loss, time 1001ms > > rtt min/avg/max/mdev = 14.377/14.746/15.116/0.369 ms > > > > ~# sudo nmcli dev wifi rescan > > [ 5359.184667] rtw_8822cu 1-1:1.2: failed to get tx report from firmware > > > > ~# ping 8.8.8.8 <-- stops working > > PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. > > > > ^C > > --- 8.8.8.8 ping statistics --- > > 11 packets transmitted, 0 received, 100% packet loss, time 10407ms > > > > ~# ip addr > > ... > > wlx34c9f08deb60 > > valid_lft 7159sec preferred_lft 7159sec > > inet6 fe80::3bc3:2caf:608a:2e32/64 scope link noprefixroute > > valid_lft forever preferred_lft forever > > ... > > [ 5389.664909] rtw_8822cu 1-1:1.2: failed to get tx report from firmware > > Just a note. From what I can tell, this seems to be similar to the situation > in [1], where it seems to work fine on a more powerful machine such as > i.MX8/RPi4 but I get problems on slower single core machines such as i.MX6 > SoloX etc. Hope this can help in troubleshooting the issue. I forgot the reference :) [1] = https://lore.kernel.org/linux-wireless/20230526055551.1823094-1-petter@technux.se/T/#t (In reply to Petter Mabäcker from comment #8) > (In reply to Petter Mabäcker from comment #7) > > (In reply to Petter Mabäcker from comment #6) > > > Ping-Ke Shih, or someone else. Any new about this? Still have problems > with > > > this. More or less each time I run rescan the driver stop working. > > > > > > ~# ping 8.8.8.8 > > > PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. > > > 64 bytes from 8.8.8.8: icmp_seq=1 ttl=119 time=14.4 ms > > > 64 bytes from 8.8.8.8: icmp_seq=2 ttl=119 time=15.1 ms > > > ^C > > > --- 8.8.8.8 ping statistics --- > > > 2 packets transmitted, 2 received, 0% packet loss, time 1001ms > > > rtt min/avg/max/mdev = 14.377/14.746/15.116/0.369 ms > > > > > > ~# sudo nmcli dev wifi rescan > > > [ 5359.184667] rtw_8822cu 1-1:1.2: failed to get tx report from firmware > > > > > > ~# ping 8.8.8.8 <-- stops working > > > PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. > > > > > > ^C > > > --- 8.8.8.8 ping statistics --- > > > 11 packets transmitted, 0 received, 100% packet loss, time 10407ms > > > > > > ~# ip addr > > > ... > > > wlx34c9f08deb60 > > > valid_lft 7159sec preferred_lft 7159sec > > > inet6 fe80::3bc3:2caf:608a:2e32/64 scope link noprefixroute > > > valid_lft forever preferred_lft forever > > > ... > > > [ 5389.664909] rtw_8822cu 1-1:1.2: failed to get tx report from firmware > > > > Just a note. From what I can tell, this seems to be similar to the > situation > > in [1], where it seems to work fine on a more powerful machine such as > > i.MX8/RPi4 but I get problems on slower single core machines such as i.MX6 > > SoloX etc. Hope this can help in troubleshooting the issue. > > I forgot the reference :) [1] = > https://lore.kernel.org/linux-wireless/20230526055551.1823094-1- > petter@technux.se/T/#t Some additional findings.. When using the patch in [1] the problems occurs more seldom. But I have found out that disabling HW offload scan and use sw instead solves the problem on my i.MX6 SoloX board at least. I thought doing it in sw would be worse from a performance perspective. But it appears that the issue is related to HW offload scan, when running of slower systems/usb(?). Any theories around this? For the time being I have a way forward by avoid using HW offload scan, even if that is a bit sad, since I want to make use of it if possible.. [1] https://lore.kernel.org/linux-wireless/20230530130917.2716182-1-petter@technux.se/T/#t |