Bug 216039 - mt7921e network has random long timeouts
Summary: mt7921e network has random long timeouts
Status: NEW
Alias: None
Product: Networking
Classification: Unclassified
Component: Wireless (show other bugs)
Hardware: All Linux
: P1 high
Assignee: networking_wireless@kernel-bugs.osdl.org
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-05-27 17:41 UTC by Andrzej
Modified: 2022-11-19 21:01 UTC (History)
1 user (show)

See Also:
Kernel Version: 5.18.0, 6.0.0-rc7, 6.0.9
Subsystem:
Regression: Yes
Bisected commit-id:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Description Andrzej 2022-05-27 17:41:24 UTC 
I am currently using kernel 5.15.12 so this problem does not appear.
When i am opening random site on kernel 5.18 it sometimes loads normally but most of the time it has timeouts(or loads within ~10s). Dmesg after timeout shows nothing.




System:    Host: PiAj Kernel: 5.15.12 x86_64 bits: 64 compiler: gcc v: 10.2.1 Desktop: Xfce 4.16.0 
           Distro: Debian GNU/Linux 11 (bullseye) 
Machine:   Type: Laptop System: LENOVO product: 82K2 v: IdeaPad Gaming 3 15ACH6 serial: <superuser required> 
           Mobo: LENOVO model: LNVNB161216 v: No DPK serial: <superuser required> UEFI: LENOVO v: H3CN30WW(V2.00) 
           date: 06/28/2021 
Battery:   ID-1: BAT1 charge: 23.2 Wh condition: 44.2/45.0 Wh (98%) 
           model: 0x43 0x65 0x6C 0x78 0x70 0x65 0x 0x4C 0x32 0x30 0x43 0x33 0x50 0x4500 status: Charging 
CPU:       Info: 6-Core model: AMD Ryzen 5 5600H with Radeon Graphics bits: 64 type: MT MCP arch: Zen 3 rev: 0 
           L2 cache: 3 MiB 
           flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm bogomips: 79050 
           Speed: 1051 MHz min/max: 1200/3300 MHz boost: enabled Core speeds (MHz): 1: 1051 2: 1190 3: 3157 
           4: 2129 5: 1641 6: 1197 7: 1828 8: 1197 9: 1498 10: 1197 11: 2777 12: 2190 
Graphics:  Device-1: NVIDIA GA107 vendor: Lenovo driver: nvidia v: 515.43.04 bus ID: 01:00.0 
           Device-2: Advanced Micro Devices [AMD/ATI] Cezanne vendor: Lenovo driver: amdgpu v: kernel 
           bus ID: 05:00.0 
           Device-3: IMC Networks Integrated Camera type: USB driver: uvcvideo bus ID: 1-3:3 
           Display: x11 server: X.Org 1.20.11 driver: loaded: amdgpu,ati,nvidia 
           unloaded: fbdev,modesetting,nouveau,vesa resolution: 1920x1080~60Hz 
           OpenGL: renderer: AMD RENOIR (DRM 3.42.0 5.15.12 LLVM 11.0.1) v: 4.6 Mesa 20.3.5 direct render: Yes 
Audio:     Device-1: Advanced Micro Devices [AMD/ATI] vendor: Lenovo driver: snd_hda_intel v: kernel 
           bus ID: 05:00.1 
           Device-2: Advanced Micro Devices [AMD] Raven/Raven2/FireFlight/Renoir Audio Processor vendor: Lenovo 
           driver: N/A bus ID: 05:00.5 
           Device-3: Advanced Micro Devices [AMD] Family 17h HD Audio vendor: Lenovo driver: snd_hda_intel 
           v: kernel bus ID: 05:00.6 
           Sound Server: ALSA v: k5.15.12 
Network:   Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet vendor: Lenovo driver: r8169 v: kernel 
           port: 2000 bus ID: 02:00.0 
           IF: enp2s0 state: down mac: 7c:8a:e1:a2:2a:d7 
           Device-2: MEDIATEK vendor: Lenovo driver: mt7921e v: kernel port: 2000 bus ID: 03:00.0 
           IF: wlo1 state: up mac: b4:b5:b6:78:e6:d5 
Bluetooth: Device-1: Foxconn / Hon Hai Wireless_Device type: USB driver: btusb v: 0.8 bus ID: 1-4:4 
           Report: ID: hci0 state: down address: 00:00:00:00:00:00 
Drives:    Local Storage: total: 476.94 GiB used: 109.96 GiB (23.1%) 
           ID-1: /dev/nvme0n1 vendor: Micron model: MTFDHBA512QFD size: 476.94 GiB 
Partition: ID-1: / size: 466.95 GiB used: 109.96 GiB (23.5%) fs: ext4 dev: /dev/nvme0n1p2 
           ID-2: /boot/efi size: 511 MiB used: 3.4 MiB (0.7%) fs: vfat dev: /dev/nvme0n1p1 
Swap:      ID-1: swap-1 type: partition size: 976 MiB used: 0 KiB (0.0%) dev: /dev/nvme0n1p3 
Sensors:   System Temperatures: cpu: 47.0 C mobo: N/A gpu: amdgpu temp: 43.0 C 
           Fan Speeds (RPM): N/A 
Info:      Processes: 257 Uptime: 16m Memory: 13.58 GiB used: 1.91 GiB (14.1%) Init: systemd runlevel: 5 
           Compilers: gcc: 10.2.1 Packages: 1878 Shell: Bash v: 5.1.4 inxi: 3.3.01 


sudo dmesg | grep mt7921
[   10.543878] mt7921e 0000:03:00.0: enabling device (0000 -> 0002)
[   10.561160] mt7921e 0000:03:00.0: ASIC revision: 79610010
[   10.653680] mt7921e 0000:03:00.0: HW/SW Version: 0x8a108a10, Build Time: 20210122002654a
[   10.662793] mt7921e 0000:03:00.0: WM Firmware Version: ____000000, Build Time: 20210122002730
[   12.783923] mt7921e 0000:03:00.0 wlo1: renamed from wlan0


Network connection log
[  815.602656] wlo1: authenticate with 34:2e:b6:7f:15:df
[  815.602669] wlo1: bad VHT capabilities, disabling VHT
[  815.602671] wlo1: Invalid HE elem, Disable HE
[  815.615546] wlo1: send auth to 34:2e:b6:7f:15:df (try 1/3)
[  815.618327] wlo1: authenticated
[  815.622028] wlo1: associate with 34:2e:b6:7f:15:df (try 1/3)
[  815.627299] wlo1: RX AssocResp from 34:2e:b6:7f:15:df (capab=0x431 status=0 aid=35)
[  815.649901] wlo1: associated
[  815.739689] IPv6: ADDRCONF(NETDEV_CHANGE): wlo1: link becomes ready
Comment 1 Andrzej 2022-05-28 17:39:45 UTC 
Unfortunetally in kernel 5.15.43 on random time (5min - 8h) when i get timeout and disconnect wireless network and connect again, it almost freeze all the system (cannot reboot etc.)

May 28 19:24:52 PiAj kernel: [  708.065780] mt7921e 0000:03:00.0: Message 00000021 (seq 11) timeout
May 28 19:24:52 PiAj kernel: [  708.149229] mt7921e 0000:03:00.0: HW/SW Version: 0x8a108a10, Build Time: 20210122002654a
May 28 19:24:52 PiAj kernel: [  708.149229] 
May 28 19:24:52 PiAj kernel: [  708.157199] mt7921e 0000:03:00.0: WM Firmware Version: ____000000, Build Time: 20210122002730
May 28 19:24:52 PiAj kernel: [  708.189539] mt7921e 0000:03:00.0: Firmware init done
May 28 19:24:55 PiAj kernel: [  711.394127] mt7921e 0000:03:00.0: Message 00000021 (seq 2) timeout
May 28 19:24:55 PiAj kernel: [  711.477342] mt7921e 0000:03:00.0: HW/SW Version: 0x8a108a10, Build Time: 20210122002654a
May 28 19:24:55 PiAj kernel: [  711.477342] 
May 28 19:24:55 PiAj kernel: [  711.485916] mt7921e 0000:03:00.0: WM Firmware Version: ____000000, Build Time: 20210122002730
May 28 19:24:56 PiAj kernel: [  711.518442] mt7921e 0000:03:00.0: Firmware init done
May 28 19:24:59 PiAj kernel: [  714.722726] mt7921e 0000:03:00.0: Message 00000021 (seq 8) timeout
May 28 19:24:59 PiAj kernel: [  714.806120] mt7921e 0000:03:00.0: HW/SW Version: 0x8a108a10, Build Time: 20210122002654a
May 28 19:24:59 PiAj kernel: [  714.806120] 
May 28 19:24:59 PiAj kernel: [  714.814262] mt7921e 0000:03:00.0: WM Firmware Version: ____000000, Build Time: 20210122002730
May 28 19:24:59 PiAj kernel: [  714.846052] mt7921e 0000:03:00.0: Firmware init done
May 28 19:25:00 PiAj systemd-timesyncd[623]: Timed out waiting for reply from 170.187.158.81:123 (2.debian.pool.ntp.org).
May 28 19:25:02 PiAj kernel: [  718.051148] mt7921e 0000:03:00.0: Message 00000021 (seq 14) timeout
May 28 19:25:02 PiAj kernel: [  718.139140] mt7921e 0000:03:00.0: HW/SW Version: 0x8a108a10, Build Time: 20210122002654a
May 28 19:25:02 PiAj kernel: [  718.139140] 
May 28 19:25:02 PiAj kernel: [  718.147269] mt7921e 0000:03:00.0: WM Firmware Version: ____000000, Build Time: 20210122002730
May 28 19:25:02 PiAj kernel: [  718.180225] mt7921e 0000:03:00.0: Firmware init done
May 28 19:25:05 PiAj kernel: [  721.379382] mt7921e 0000:03:00.0: Message 00000021 (seq 5) timeout
May 28 19:25:05 PiAj kernel: [  721.379409] mt7921e 0000:03:00.0: chip reset failed
May 28 19:25:08 PiAj kernel: [  724.455435] mt7921e 0000:03:00.0: Message 40000001 (seq 6) timeout
May 28 19:25:10 PiAj systemd-timesyncd[623]: Timed out waiting for reply from 17.253.54.251:123 (2.debian.pool.ntp.org).
May 28 19:25:12 PiAj kernel: [  727.523958] mt7921e 0000:03:00.0: Message 00000027 (seq 7) timeout
May 28 19:25:12 PiAj kernel: [  727.530163] mt7921e 0000:03:00.0: chip reset
May 28 19:25:15 PiAj kernel: [  730.596497] mt7921e 0000:03:00.0: Message 40000003 (seq 8) timeout
May 28 19:25:18 PiAj kernel: [  733.668607] mt7921e 0000:03:00.0: Message 40000003 (seq 9) timeout
May 28 19:25:18 PiAj kernel: [  733.668744] wlo1: Driver requested disconnection from AP 00:00:00:00:00:00
May 28 19:25:21 PiAj kernel: [  736.996934] mt7921e 0000:03:00.0: Message 80000010 (seq 10) timeout
May 28 19:25:21 PiAj kernel: [  736.996944] mt7921e 0000:03:00.0: Failed to get patch semaphore
May 28 19:25:21 PiAj kernel: [  737.081112] mt7921e 0000:03:00.0: HW/SW Version: 0x8a108a10, Build Time: 20210122002654a
May 28 19:25:21 PiAj kernel: [  737.081112] 
May 28 19:25:21 PiAj kernel: [  737.089043] mt7921e 0000:03:00.0: WM Firmware Version: ____000000, Build Time: 20210122002730
May 28 19:25:21 PiAj kernel: [  737.121942] mt7921e 0000:03:00.0: Firmware init done
May 28 19:25:24 PiAj kernel: [  740.325331] mt7921e 0000:03:00.0: Message 00000021 (seq 1) timeout
May 28 19:25:24 PiAj kernel: [  740.409653] mt7921e 0000:03:00.0: HW/SW Version: 0x8a108a10, Build Time: 20210122002654a
May 28 19:25:24 PiAj kernel: [  740.409653] 
May 28 19:25:24 PiAj kernel: [  740.417643] mt7921e 0000:03:00.0: WM Firmware Version: ____000000, Build Time: 20210122002730
May 28 19:25:24 PiAj kernel: [  740.450093] mt7921e 0000:03:00.0: Firmware init done
May 28 19:25:28 PiAj kernel: [  743.653668] mt7921e 0000:03:00.0: Message 00000021 (seq 7) timeout
May 28 19:25:28 PiAj kernel: [  743.736770] mt7921e 0000:03:00.0: HW/SW Version: 0x8a108a10, Build Time: 20210122002654a
May 28 19:25:28 PiAj kernel: [  743.736770] 
May 28 19:25:28 PiAj kernel: [  743.745055] mt7921e 0000:03:00.0: WM Firmware Version: ____000000, Build Time: 20210122002730
May 28 19:25:28 PiAj kernel: [  743.778296] mt7921e 0000:03:00.0: Firmware init done
May 28 19:25:31 PiAj kernel: [  746.981754] mt7921e 0000:03:00.0: Message 00000021 (seq 13) timeout
May 28 19:25:31 PiAj kernel: [  747.065898] mt7921e 0000:03:00.0: HW/SW Version: 0x8a108a10, Build Time: 20210122002654a
May 28 19:25:31 PiAj kernel: [  747.065898] 
May 28 19:25:31 PiAj kernel: [  747.073847] mt7921e 0000:03:00.0: WM Firmware Version: ____000000, Build Time: 20210122002730
May 28 19:25:31 PiAj kernel: [  747.106079] mt7921e 0000:03:00.0: Firmware init done
May 28 19:25:34 PiAj kernel: [  750.310169] mt7921e 0000:03:00.0: Message 00000021 (seq 4) timeout
May 28 19:25:34 PiAj kernel: [  750.398328] mt7921e 0000:03:00.0: HW/SW Version: 0x8a108a10, Build Time: 20210122002654a
Comment 2 kaienkira 2022-06-28 01:22:00 UTC 
same issue for Thinkbook 14p Gen2, mt7921e driver crashed randomly in 5.18.5 ~ 5.18.7

Jun 27 17:42:39 ChkTB14PDev kernel: ------------[ cut here ]------------
Jun 27 17:42:39 ChkTB14PDev kernel: WARNING: CPU: 15 PID: 847 at kernel/kthread.c:659 kthread_park+0x7f/0x90
Jun 27 17:42:39 ChkTB14PDev kernel: Modules linked in: cmac ccm vfat fat snd_sof_amd_renoir snd_sof_amd_acp snd_sof_pci snd_hda_codec_realtek snd_sof snd_hda_codec_generic snd_sof_utils snd_hda_codec_hdmi ledtrig_audio snd_hda_>
Jun 27 17:42:39 ChkTB14PDev kernel:  cryptd rapl platform_profile cfg80211 tpm_crb typec acpi_als mc i2c_piix4 k10temp mac_hid roles wmi industrialio_triggered_buffer tpm_tis i2c_hid_acpi rfkill video tpm_tis_core kfifo_buf tpm>
Jun 27 17:42:39 ChkTB14PDev kernel: CPU: 15 PID: 847 Comm: kworker/u32:0 Tainted: G           OE     5.18.7-zen1-1-zen #1 c159fb3cbb1adc41030345f347e6d84e4be47f08
Jun 27 17:42:39 ChkTB14PDev kernel: Hardware name: LENOVO 20YN/LNVNB161216, BIOS GWCN33WW 06/28/2021
Jun 27 17:42:39 ChkTB14PDev kernel: Workqueue: mt76 mt7921_mac_reset_work [mt7921_common]
Jun 27 17:42:39 ChkTB14PDev kernel: RIP: 0010:kthread_park+0x7f/0x90
Jun 27 17:42:39 ChkTB14PDev kernel: Code: 89 df e8 b4 c8 01 00 48 85 c0 74 27 31 c0 5b 5d c3 cc 0f 0b 48 8b ab d0 06 00 00 a8 04 74 af 0f 0b b8 da ff ff ff 5b 5d c3 cc <0f> 0b b8 f0 ff ff ff eb db 0f 0b eb d5 0f 1f 40 00 f3 0f >
Jun 27 17:42:39 ChkTB14PDev kernel: RSP: 0018:ffffc23e818ffdf0 EFLAGS: 00010202
Jun 27 17:42:39 ChkTB14PDev kernel: RAX: 0000000000000004 RBX: ffffa0344905c100 RCX: 0000000000000000
Jun 27 17:42:39 ChkTB14PDev kernel: RDX: 0000000000000001 RSI: 0000000000004187 RDI: ffffa0344905c100
Jun 27 17:42:39 ChkTB14PDev kernel: RBP: ffffa03443829180 R08: ffffa034493224a0 R09: ffffc23e818ffda8
Jun 27 17:42:39 ChkTB14PDev kernel: R10: ffffffff8a25aa20 R11: 0000000000000002 R12: ffffa034493208e0
Jun 27 17:42:39 ChkTB14PDev kernel: R13: ffffa034493220e0 R14: ffffa03449328610 R15: ffffa03449322430
Jun 27 17:42:39 ChkTB14PDev kernel: FS:  0000000000000000(0000) GS:ffffa036e01c0000(0000) knlGS:0000000000000000
Jun 27 17:42:39 ChkTB14PDev kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 27 17:42:39 ChkTB14PDev kernel: CR2: 000055e29d7cc018 CR3: 000000025b610000 CR4: 0000000000750ee0
Jun 27 17:42:39 ChkTB14PDev kernel: PKRU: 55555554
Jun 27 17:42:39 ChkTB14PDev kernel: Call Trace:
Jun 27 17:42:39 ChkTB14PDev kernel:  <TASK>
Jun 27 17:42:39 ChkTB14PDev kernel:  mt7921e_mac_reset+0xa2/0x3d0 [mt7921e 0ad6f82269d7c9bd66ea71fdd1185cee3e632072]
Jun 27 17:42:39 ChkTB14PDev kernel:  mt7921_mac_reset_work+0xa3/0x14e [mt7921_common d2359554f4393f31dfa933224edde23e4c4af2ea]
Jun 27 17:42:39 ChkTB14PDev kernel:  process_one_work+0x255/0x410
Jun 27 17:42:39 ChkTB14PDev kernel:  worker_thread+0x55/0x4d0
Jun 27 17:42:39 ChkTB14PDev kernel:  ? process_one_work+0x410/0x410
Jun 27 17:42:39 ChkTB14PDev kernel:  kthread+0x13f/0x160
Jun 27 17:42:39 ChkTB14PDev kernel:  ? kthread_complete_and_exit+0x20/0x20
Jun 27 17:42:39 ChkTB14PDev kernel:  ret_from_fork+0x22/0x30
Jun 27 17:42:39 ChkTB14PDev kernel:  </TASK>
Jun 27 17:42:39 ChkTB14PDev kernel: ---[ end trace 0000000000000000 ]---
Comment 3 Andrzej 2022-10-03 10:32:42 UTC 
Long timeouts still exist in kernel 6.0.0-rc7
Comment 4 Andrzej 2022-11-19 20:57:42 UTC 
Is there any way to provide any test data ? Still exists in stable kernel 6.0.9
Comment 5 Andrzej 2022-11-19 21:01:53 UTC 
73 packets transmitted, 40 received, 45.2055% packet loss, time 136124ms
Note You need to log in before you can comment on or make changes to this bug.