Bug 218206

Summary: iwlwifi: rfkill locking up kernel 6.5.12, 6.6.2
Product: Drivers Reporter: Linus Lotz (register+kernelbugzilla)
Component: network-wireless-intelAssignee: Default virtual assignee for network-wireless-intel (drivers_network-wireless-intel)
Status: CLOSED CODE_FIX    
Severity: normal CC: bagasdotme, darrell, entaahlaah, johannes, kernel, kvalo, mpagano, regressions
Priority: P3    
Hardware: All   
OS: Linux   
Kernel Version: Subsystem:
Regression: No Bisected commit-id:
Attachments: attempted fix

Description Linus Lotz 2023-11-29 09:44:48 UTC 
Rfkilling the iwlwifi can lock up my machine, rfkill will not respond and other networking related things (e.g. ip a or ip r)
I see this on 6.5.12 and 6.6.2 (fedora kernel). On 6.5.11 it does not lockup AFAICT, but also shows kernel oops.
WiFi Hardware: AC 8265

Kernel oops for 6.5.11:
Nov 29 10:12:11 hql024707 rfkill[6712]: block set for id 1
Nov 29 10:12:11 hql024707 NetworkManager[1138]: <info>  [1701249131.5557] manager: rfkill: Wi-Fi now disabled by radio killswitch
Nov 29 10:12:11 hql024707 kernel: ------------[ cut here ]------------
Nov 29 10:12:11 hql024707 kernel: Timeout waiting for hardware access (CSR_GP_CNTRL 0x00000008)
Nov 29 10:12:11 hql024707 kernel: WARNING: CPU: 7 PID: 5222 at drivers/net/wireless/intel/iwlwifi/pcie/trans.c:2190 __iwl_trans_pcie>
Nov 29 10:12:11 hql024707 kernel: Modules linked in: uinput rfcomm snd_seq_dummy snd_hrtimer des_generic libdes md4 nf_conntrack_net>
Nov 29 10:12:11 hql024707 kernel:  snd_hda_intel intel_tcc_cooling x86_pkg_temp_thermal snd_usbmidi_lib intel_powerclamp snd_intel_d>
Nov 29 10:12:11 hql024707 kernel:  intel_hid int3400_thermal int3403_thermal acpi_pad acpi_thermal_rel sparse_keymap int340x_thermal>
Nov 29 10:12:11 hql024707 kernel: CPU: 7 PID: 5222 Comm: kworker/7:0 Not tainted 6.5.11-300.fc39.x86_64 #1
Nov 29 10:12:11 hql024707 kernel: Hardware name: Dell Inc. Latitude 7490/0KP0FT, BIOS 1.33.0 07/06/2023
Nov 29 10:12:11 hql024707 kernel: Workqueue: events cfg80211_rfkill_block_work [cfg80211]
Nov 29 10:12:11 hql024707 kernel: RIP: 0010:__iwl_trans_pcie_grab_nic_access+0x14a/0x150 [iwlwifi]
Nov 29 10:12:11 hql024707 kernel: Code: 00 00 00 48 89 df e8 f5 93 fe ff 4c 89 f7 e8 5d c6 d9 d6 eb 8c 89 c6 48 c7 c7 38 14 4d c1 c6>
Nov 29 10:12:11 hql024707 kernel: RSP: 0018:ffffc014514ebb90 EFLAGS: 00010282
Nov 29 10:12:11 hql024707 kernel: RAX: 0000000000000000 RBX: ffff9d4c20ee0028 RCX: 0000000000000027
Nov 29 10:12:11 hql024707 kernel: RDX: ffff9d4f6e5e1548 RSI: 0000000000000001 RDI: ffff9d4f6e5e1540
Nov 29 10:12:11 hql024707 kernel: RBP: 0000000000000008 R08: 0000000000000000 R09: ffffc014514eba20
Nov 29 10:12:11 hql024707 kernel: R10: 0000000000000003 R11: ffffffff99345d68 R12: 0000000000000001
Nov 29 10:12:11 hql024707 kernel: R13: 0000000000000000 R14: ffff9d4c20ee35cc R15: 0000000000000011
Nov 29 10:12:11 hql024707 kernel: FS:  0000000000000000(0000) GS:ffff9d4f6e5c0000(0000) knlGS:0000000000000000
Nov 29 10:12:11 hql024707 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 29 10:12:11 hql024707 kernel: CR2: 00007ffb507f74c0 CR3: 00000002fc222001 CR4: 00000000003706e0
Nov 29 10:12:11 hql024707 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov 29 10:12:11 hql024707 kernel: Timeout waiting for hardware access (CSR_GP_CNTRL 0x00000008)
Nov 29 10:12:11 hql024707 kernel: WARNING: CPU: 7 PID: 5222 at drivers/net/wireless/intel/iwlwifi/pcie/trans.c:2190 __iwl_trans_pcie>
Nov 29 10:12:11 hql024707 kernel: Modules linked in: uinput rfcomm snd_seq_dummy snd_hrtimer des_generic libdes md4 nf_conntrack_net>
Nov 29 10:12:11 hql024707 kernel:  snd_hda_intel intel_tcc_cooling x86_pkg_temp_thermal snd_usbmidi_lib intel_powerclamp snd_intel_d>
Nov 29 10:12:11 hql024707 kernel:  intel_hid int3400_thermal int3403_thermal acpi_pad acpi_thermal_rel sparse_keymap int340x_thermal>
Nov 29 10:12:11 hql024707 kernel: CPU: 7 PID: 5222 Comm: kworker/7:0 Not tainted 6.5.11-300.fc39.x86_64 #1
Nov 29 10:12:11 hql024707 kernel: Hardware name: Dell Inc. Latitude 7490/0KP0FT, BIOS 1.33.0 07/06/2023
Nov 29 10:12:11 hql024707 kernel: Workqueue: events cfg80211_rfkill_block_work [cfg80211]
Nov 29 10:12:11 hql024707 kernel: RIP: 0010:__iwl_trans_pcie_grab_nic_access+0x14a/0x150 [iwlwifi]
Nov 29 10:12:11 hql024707 kernel: Code: 00 00 00 48 89 df e8 f5 93 fe ff 4c 89 f7 e8 5d c6 d9 d6 eb 8c 89 c6 48 c7 c7 38 14 4d c1 c6>
Nov 29 10:12:11 hql024707 kernel: RSP: 0018:ffffc014514ebb90 EFLAGS: 00010282
Nov 29 10:12:11 hql024707 kernel: RAX: 0000000000000000 RBX: ffff9d4c20ee0028 RCX: 0000000000000027
Nov 29 10:12:11 hql024707 kernel: RDX: ffff9d4f6e5e1548 RSI: 0000000000000001 RDI: ffff9d4f6e5e1540
Nov 29 10:12:11 hql024707 kernel: RBP: 0000000000000008 R08: 0000000000000000 R09: ffffc014514eba20
Nov 29 10:12:11 hql024707 kernel: R10: 0000000000000003 R11: ffffffff99345d68 R12: 0000000000000001
Nov 29 10:12:11 hql024707 kernel: R13: 0000000000000000 R14: ffff9d4c20ee35cc R15: 0000000000000011
Nov 29 10:12:11 hql024707 kernel: FS:  0000000000000000(0000) GS:ffff9d4f6e5c0000(0000) knlGS:0000000000000000
Nov 29 10:12:11 hql024707 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 29 10:12:11 hql024707 kernel: CR2: 00007ffb507f74c0 CR3: 00000002fc222001 CR4: 00000000003706e0
Nov 29 10:12:11 hql024707 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov 29 10:12:11 hql024707 kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Nov 29 10:12:11 hql024707 kernel: Call Trace:
Nov 29 10:12:11 hql024707 kernel:  <TASK>
Nov 29 10:12:11 hql024707 kernel:  ? __iwl_trans_pcie_grab_nic_access+0x14a/0x150 [iwlwifi]
Nov 29 10:12:11 hql024707 kernel:  ? __warn+0x81/0x130
Nov 29 10:12:11 hql024707 kernel:  ? __iwl_trans_pcie_grab_nic_access+0x14a/0x150 [iwlwifi]
Nov 29 10:12:11 hql024707 kernel:  ? report_bug+0x171/0x1a0
Nov 29 10:12:11 hql024707 kernel:  ? prb_read_valid+0x1b/0x30
Nov 29 10:12:11 hql024707 kernel:  ? handle_bug+0x3c/0x80
Nov 29 10:12:11 hql024707 kernel:  ? exc_invalid_op+0x17/0x70
Nov 29 10:12:11 hql024707 kernel:  ? asm_exc_invalid_op+0x1a/0x20
Nov 29 10:12:11 hql024707 kernel:  ? __iwl_trans_pcie_grab_nic_access+0x14a/0x150 [iwlwifi]
Nov 29 10:12:11 hql024707 kernel:  iwl_trans_pcie_grab_nic_access+0x1a/0x40 [iwlwifi]
Nov 29 10:12:11 hql024707 kernel:  iwl_write_prph_delay+0x27/0x90 [iwlwifi]
Nov 29 10:12:11 hql024707 kernel:  iwl_fw_dbg_stop_restart_recording+0x128/0x290 [iwlwifi]
Nov 29 10:12:11 hql024707 kernel:  ? __cancel_work_timer+0xff/0x190
Nov 29 10:12:11 hql024707 kernel:  iwl_mvm_stop_device+0x3a/0x80 [iwlmvm]
Nov 29 10:12:11 hql024707 kernel:  __iwl_mvm_mac_stop+0x59/0x120 [iwlmvm]
Nov 29 10:12:11 hql024707 kernel:  iwl_mvm_mac_stop+0x7f/0xd0 [iwlmvm]
Nov 29 10:12:11 hql024707 kernel:  drv_stop+0x34/0x100 [mac80211]
Nov 29 10:12:11 hql024707 kernel:  ieee80211_do_stop+0x5d0/0x890 [mac80211]
Nov 29 10:12:11 hql024707 kernel:  ieee80211_stop+0x4d/0x180 [mac80211]
Nov 29 10:12:11 hql024707 kernel:  __dev_close_many+0x9b/0x110
Nov 29 10:12:11 hql024707 kernel:  dev_close_many+0x8f/0x150
Nov 29 10:12:11 hql024707 kernel:  dev_close+0x83/0xb0
Nov 29 10:12:11 hql024707 kernel:  cfg80211_shutdown_all_interfaces+0x4d/0xf0 [cfg80211]
Nov 29 10:12:11 hql024707 kernel:  cfg80211_rfkill_block_work+0x1e/0x30 [cfg80211]
Nov 29 10:12:11 hql024707 kernel:  process_one_work+0x1de/0x3f0
Nov 29 10:12:11 hql024707 kernel:  worker_thread+0x51/0x390
Nov 29 10:12:11 hql024707 kernel:  ? __pfx_worker_thread+0x10/0x10
Nov 29 10:12:11 hql024707 kernel:  kthread+0xe5/0x120
Nov 29 10:12:11 hql024707 kernel:  ? __pfx_kthread+0x10/0x10
Nov 29 10:12:11 hql024707 kernel:  ret_from_fork+0x31/0x50
Nov 29 10:12:11 hql024707 kernel:  ? __pfx_kthread+0x10/0x10
Nov 29 10:12:11 hql024707 kernel:  ret_from_fork_asm+0x1b/0x30
Nov 29 10:12:11 hql024707 kernel:  </TASK>
Nov 29 10:12:11 hql024707 kernel: ---[ end trace 0000000000000000 ]---
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: iwlwifi transaction failed, dumping registers
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: iwlwifi device config registers:
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 00000000: 24fd8086 00100406 02800078 00000000 ec000004 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 00000020: 00000000 00000000 00000000 00508086 00000000 000000c8 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 00000040: 00020010 10008ec0 00190c10 0045e811 10110142 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 00000060: 00000000 00080812 00000405 00000000 00010001 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 00000080: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 000000a0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 000000c0: 00000000 00000000 c823d001 0d000000 00814005 fee007b8 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 000000e0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 00000100: 14010001 00000000 00000000 00462031 00002000 00002000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 00000120: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 00000140: 14c10003 ffb58619 a0510bff 15410018 10031003 0001001e 00481e1f 405>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: iwlwifi device memory mapped registers:
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 00000000: 00080000 00000000 00000000 00000000 00000000 00000000 00000010 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 00000020: 00000011 00000008 00000230 d55555d5 d55555d5 d55555d5 80008040 001>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: iwlwifi device AER capability structure:
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 00000000: 14010001 00000000 00000000 00462031 00002000 00002000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: 00000020: 00000000 00000000 00000000
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: iwlwifi parent port (0000:00:1c.2) config registers:
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000000: 9d128086 00100407 060400f1 00810000 00000000 00000000 00020200 200>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000020: ec00ec00 0001fff1 00000000 00000000 00000000 00000040 00000000 001>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000040: 01428010 00008001 00100000 03724813 70110042 0034b200 01480000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000060: 00000000 00000837 00000400 0000000e 00010003 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000080: 00019005 fee00298 00000000 00000000 0000a00d 081c1028 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 000000a0: c8030001 00000000 00000000 00000000 00000000 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 000000c0: 00000000 00000000 00000000 00000000 07001001 00001842 899e0008 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 000000e0: 00630300 00000000 00100016 00000000 00000150 4c000000 08410fb3 030>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000100: 14010001 00000000 00010000 00060011 00000000 00002000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000120: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000140: 2001000d 0000000f 00000000 00000000 00000000 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000160: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000180: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 000001a0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 000001c0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 000001e0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000200: 2201001e 00b0281f 4058280f
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:02:00.0: iwlwifi root port (0000:00:1c.2) AER cap structure:
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000000: 14010001 00000000 00010000 00060011 00000000 00002000 00000000 000>
Nov 29 10:12:11 hql024707 kernel: iwlwifi 0000:00:1c.2: 00000020: 00000000 00000000 00000000 00000000 00000000 00000000

Kernel 6.5.12:
Nov 28 13:51:50 hql024707 kernel: iwlwifi 0000:02:00.0: RF_KILL bit toggled to disable radio.
Nov 28 13:51:50 hql024707 kernel: iwlwifi 0000:02:00.0: reporting RF_KILL (radio disabled)
Nov 28 13:51:50 hql024707 iwd[1160]: rfkill id 1 not found in a RFKILL_OP_CHANGE event
Nov 28 13:51:50 hql024707 kernel: wlan0: deauthenticating from 4c:71:0d:50:14:0d by local choice (Reason: 3=DEAUTH_LEAVING)
Nov 28 13:51:50 hql024707 systemd[1]: Starting systemd-rfkill.service - Load/Save RF Kill Switch Status...
Nov 28 13:51:50 hql024707 systemd[1]: Started systemd-rfkill.service - Load/Save RF Kill Switch Status.
Nov 28 13:51:50 hql024707 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='u>
Nov 28 13:51:52 hql024707 kernel: iwlwifi 0000:02:00.0: fail to flush all tx fifo queues Q 5
Nov 28 13:51:52 hql024707 kernel: iwlwifi 0000:02:00.0: Queue 5 is active on fifo 3 and stuck for 10000 ms. SW [6, 7] HW [7, 7] FH T>
Nov 28 13:51:54 hql024707 kernel: iwlwifi 0000:02:00.0: fail to flush all tx fifo queues Q 5
Nov 28 13:51:54 hql024707 kernel: iwlwifi 0000:02:00.0: Queue 5 is active on fifo 3 and stuck for 10000 ms. SW [6, 7] HW [7, 7] FH T>
Nov 28 13:51:54 hql024707 kernel: ------------[ cut here ]------------
Nov 28 13:51:54 hql024707 kernel: WARNING: CPU: 5 PID: 31284 at net/mac80211/sta_info.c:1442 __sta_info_destroy_part2+0x235/0x300 [m>
Nov 28 13:51:54 hql024707 kernel: Modules linked in: snd_usb_audio snd_usbmidi_lib snd_ump snd_rawmidi r8153_ecm cdc_ether usbnet r8>
Nov 28 13:51:54 hql024707 kernel:  snd_soc_sst_dsp snd_soc_acpi_intel_match snd_soc_acpi snd_hda_codec_hdmi snd_soc_core mac80211 sn>
Nov 28 13:51:54 hql024707 kernel:  processor_thermal_rfim processor_thermal_mbox processor_thermal_rapl intel_pch_thermal intel_xhci>
Nov 28 13:51:54 hql024707 kernel: CPU: 5 PID: 31284 Comm: kworker/5:0 Not tainted 6.5.12-300.fc39.x86_64 #1
Nov 28 13:51:54 hql024707 kernel: Hardware name: Dell Inc. Latitude 7490/0KP0FT, BIOS 1.33.0 07/06/2023
Nov 28 13:51:54 hql024707 kernel: Workqueue: events cfg80211_rfkill_block_work [cfg80211]
Nov 28 13:51:54 hql024707 kernel: RIP: 0010:__sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 28 13:51:54 hql024707 kernel: Code: bb d4 00 00 00 00 0f 84 68 ff ff ff 45 31 c0 b9 01 00 00 00 48 89 da 4c 89 f6 4c 89 e7 e8 83>
Nov 28 13:51:54 hql024707 kernel: RSP: 0018:ffffb7cf18e4f958 EFLAGS: 00010282
Nov 28 13:51:54 hql024707 kernel: RAX: 00000000ffffff92 RBX: ffff8ab318bfc000 RCX: 0000000000000027
Nov 28 13:51:54 hql024707 kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8ab29fb9a098
Nov 28 13:51:54 hql024707 kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: ffffb7cf18e4f4b8
Nov 28 13:51:54 hql024707 kernel: R10: 0000000000000003 R11: ffffffff9f345d88 R12: ffff8ab29fb98900
Nov 28 13:51:54 hql024707 kernel: R13: ffff8ab318bfcab8 R14: ffff8ab2a2e149c0 R15: ffff8ab2a2e149c0
Nov 28 13:51:54 hql024707 kernel: FS:  0000000000000000(0000) GS:ffff8ab5ee540000(0000) knlGS:0000000000000000
Nov 28 13:51:54 hql024707 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 28 13:51:54 hql024707 kernel: CR2: 000014bad53a7000 CR3: 0000000359222004 CR4: 00000000003706e0
Nov 28 13:51:54 hql024707 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov 28 13:51:54 hql024707 kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Nov 28 13:51:54 hql024707 kernel: Call Trace:
Nov 28 13:51:54 hql024707 kernel:  <TASK>
Nov 28 13:51:54 hql024707 kernel:  ? __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 28 13:51:54 hql024707 kernel:  ? __warn+0x81/0x130
Nov 28 13:51:54 hql024707 kernel:  ? __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 28 13:51:54 hql024707 kernel:  ? report_bug+0x171/0x1a0
Nov 28 13:51:54 hql024707 kernel:  ? handle_bug+0x3c/0x80
Nov 28 13:51:54 hql024707 kernel:  ? exc_invalid_op+0x17/0x70
Nov 28 13:51:54 hql024707 kernel:  ? asm_exc_invalid_op+0x1a/0x20
Nov 28 13:51:54 hql024707 kernel:  ? __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 28 13:51:54 hql024707 kernel:  ? __sta_info_destroy_part2+0x22d/0x300 [mac80211]
Nov 28 13:51:54 hql024707 kernel:  __sta_info_flush+0x169/0x1f0 [mac80211]
Nov 28 13:51:54 hql024707 kernel:  ieee80211_set_disassoc+0x16c/0x620 [mac80211]
Nov 28 13:51:54 hql024707 kernel:  ieee80211_mgd_deauth+0x13f/0x3d0 [mac80211]
Nov 28 13:51:54 hql024707 kernel:  cfg80211_mlme_deauth+0xba/0x1e0 [cfg80211]
Nov 28 13:51:54 hql024707 kernel:  cfg80211_mlme_down+0x65/0x90 [cfg80211]
Nov 28 13:51:54 hql024707 kernel:  cfg80211_disconnect+0x180/0x1f0 [cfg80211]
Nov 28 13:51:54 hql024707 kernel:  cfg80211_netdev_notifier_call+0x141/0x4d0 [cfg80211]
Nov 28 13:51:54 hql024707 kernel:  ? iwl_mvm_send_cmd+0x1f/0x60 [iwlmvm]
Nov 28 13:51:54 hql024707 kernel:  ? iwl_mvm_mc_iface_iterator+0xcd/0xf0 [iwlmvm]
Nov 28 13:51:54 hql024707 kernel:  ? __iterate_interfaces+0x9d/0x110 [mac80211]
Nov 28 13:51:54 hql024707 kernel:  ? __pfx_iwl_mvm_mc_iface_iterator+0x10/0x10 [iwlmvm]
Nov 28 13:51:54 hql024707 kernel:  ? iwl_mvm_send_cmd_pdu+0x62/0xb0 [iwlmvm]
Nov 28 13:51:54 hql024707 kernel:  ? rtnl_is_locked+0x15/0x20
Nov 28 13:51:54 hql024707 kernel:  ? inetdev_event+0x3d/0x6f0
Nov 28 13:51:54 hql024707 kernel:  ? iwl_mvm_recalc_multicast+0x5d/0xa0 [iwlmvm]
Nov 28 13:51:54 hql024707 kernel:  notifier_call_chain+0x5a/0xd0
Nov 28 13:51:54 hql024707 kernel:  __dev_close_many+0x5f/0x110
Nov 28 13:51:54 hql024707 kernel:  dev_close_many+0x8f/0x150
Nov 28 13:51:54 hql024707 kernel:  dev_close+0x83/0xb0
Nov 28 13:51:54 hql024707 kernel:  cfg80211_shutdown_all_interfaces+0x4d/0xf0 [cfg80211]
Nov 28 13:51:54 hql024707 kernel:  cfg80211_rfkill_block_work+0x1e/0x30 [cfg80211]
Nov 28 13:51:54 hql024707 kernel:  process_one_work+0x1de/0x3f0
Nov 28 13:51:54 hql024707 kernel:  worker_thread+0x51/0x390
Nov 28 13:51:54 hql024707 kernel:  ? __pfx_worker_thread+0x10/0x10
Nov 28 13:51:54 hql024707 kernel:  kthread+0xe5/0x120
Nov 28 13:51:54 hql024707 kernel:  ? __pfx_kthread+0x10/0x10
Nov 28 13:51:54 hql024707 kernel:  ret_from_fork+0x31/0x50
Nov 28 13:51:54 hql024707 kernel:  ? __pfx_kthread+0x10/0x10
Nov 28 13:51:54 hql024707 kernel:  ret_from_fork_asm+0x1b/0x30
Nov 28 13:51:54 hql024707 kernel:  </TASK>
Nov 28 13:51:54 hql024707 kernel: ---[ end trace 0000000000000000 ]---

and 6.6.2:
Nov 29 09:52:43 hql024707 kernel: iwlwifi 0000:02:00.0: fail to flush all tx fifo queues Q 5
Nov 29 09:52:43 hql024707 kernel: iwlwifi 0000:02:00.0: Queue 5 is active on fifo 3 and stuck for 10000 ms. SW [5, 6] HW [6, 6] FH T>
Nov 29 09:52:45 hql024707 kernel: iwlwifi 0000:02:00.0: fail to flush all tx fifo queues Q 5
Nov 29 09:52:45 hql024707 kernel: iwlwifi 0000:02:00.0: Queue 5 is active on fifo 3 and stuck for 10000 ms. SW [5, 6] HW [6, 6] FH T>
Nov 29 09:52:45 hql024707 kernel: ------------[ cut here ]------------
Nov 29 09:52:45 hql024707 kernel: WARNING: CPU: 2 PID: 1143 at net/mac80211/sta_info.c:1442 __sta_info_destroy_part2+0x235/0x300 [ma>
Nov 29 09:52:45 hql024707 kernel: Modules linked in: uinput rfcomm snd_seq_dummy snd_hrtimer des_generic libdes md4 nf_conntrack_net>
Nov 29 09:52:45 hql024707 kernel:  snd_compress ac97_bus iwlmvm ee1004 snd_pcm_dmaengine snd_hda_codec_generic kvm mei_wdt dell_rbtn>
Nov 29 09:52:45 hql024707 kernel:  int3400_thermal acpi_thermal_rel intel_hid int3403_thermal acpi_pad int340x_thermal_zone sparse_k>
Nov 29 09:52:45 hql024707 kernel: CPU: 2 PID: 1143 Comm: NetworkManager Not tainted 6.6.2-201.fc39.x86_64 #1
Nov 29 09:52:45 hql024707 kernel: Hardware name: Dell Inc. Latitude 7490/0KP0FT, BIOS 1.33.0 07/06/2023
Nov 29 09:52:45 hql024707 kernel: RIP: 0010:__sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 29 09:52:45 hql024707 kernel: Code: bb d4 00 00 00 00 0f 84 68 ff ff ff 45 31 c0 b9 01 00 00 00 48 89 da 4c 89 f6 4c 89 e7 e8 83>
Nov 29 09:52:45 hql024707 kernel: RSP: 0018:ffffc90002663808 EFLAGS: 00010282
Nov 29 09:52:45 hql024707 kernel: RAX: 00000000ffffff92 RBX: ffff88812954c000 RCX: 0000000000000027
Nov 29 09:52:45 hql024707 kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff888125cfa098
Nov 29 09:52:45 hql024707 kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: ffffc90002663360
Nov 29 09:52:45 hql024707 kernel: R10: 0000000000000003 R11: ffffffffaa346088 R12: ffff888125cf8900
Nov 29 09:52:45 hql024707 kernel: R13: ffff88812954cab8 R14: ffff8881226549c0 R15: ffff8881226549c0
Nov 29 09:52:45 hql024707 kernel: FS:  00007fb50e081540(0000) GS:ffff88846e480000(0000) knlGS:0000000000000000
Nov 29 09:52:45 hql024707 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 29 09:52:45 hql024707 kernel: CR2: 000055a6c513e700 CR3: 000000012573a003 CR4: 00000000003706e0
Nov 29 09:52:45 hql024707 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov 29 09:52:45 hql024707 kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Nov 29 09:52:45 hql024707 kernel: Call Trace:
Nov 29 09:52:45 hql024707 kernel:  <TASK>
Nov 29 09:52:45 hql024707 kernel:  ? __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 29 09:52:45 hql024707 kernel:  ? __warn+0x81/0x130
Nov 29 09:52:45 hql024707 kernel:  ? __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 29 09:52:45 hql024707 kernel:  ? report_bug+0x171/0x1a0
Nov 29 09:52:45 hql024707 kernel:  ? handle_bug+0x3c/0x80
Nov 29 09:52:45 hql024707 kernel:  ? exc_invalid_op+0x17/0x70
Nov 29 09:52:45 hql024707 kernel:  ? asm_exc_invalid_op+0x1a/0x20
Nov 29 09:52:45 hql024707 kernel:  ? __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 29 09:52:45 hql024707 kernel:  ? __sta_info_destroy_part2+0x22d/0x300 [mac80211]
Nov 29 09:52:45 hql024707 kernel:  __sta_info_flush+0x169/0x1f0 [mac80211]
Nov 29 09:52:45 hql024707 kernel:  ieee80211_set_disassoc+0x16c/0x620 [mac80211]
Nov 29 09:52:45 hql024707 kernel:  ieee80211_mgd_deauth+0x13f/0x3d0 [mac80211]
Nov 29 09:52:45 hql024707 kernel:  cfg80211_mlme_deauth+0xba/0x1e0 [cfg80211]
Nov 29 09:52:45 hql024707 kernel:  cfg80211_mlme_down+0x65/0x90 [cfg80211]
Nov 29 09:52:45 hql024707 kernel:  cfg80211_disconnect+0x180/0x1f0 [cfg80211]
Nov 29 09:52:45 hql024707 kernel:  cfg80211_netdev_notifier_call+0x141/0x4d0 [cfg80211]
Nov 29 09:52:45 hql024707 kernel:  ? iwl_mvm_send_cmd+0x1f/0x60 [iwlmvm]
Nov 29 09:52:45 hql024707 kernel:  ? iwl_mvm_mc_iface_iterator+0xcd/0xf0 [iwlmvm]
Nov 29 09:52:45 hql024707 kernel:  ? __iterate_interfaces+0x9d/0x110 [mac80211]
Nov 29 09:52:45 hql024707 kernel:  ? __pfx_iwl_mvm_mc_iface_iterator+0x10/0x10 [iwlmvm]
Nov 29 09:52:45 hql024707 kernel:  ? iwl_mvm_send_cmd_pdu+0x62/0xb0 [iwlmvm]
Nov 29 09:52:45 hql024707 kernel:  ? rtnl_is_locked+0x15/0x20
Nov 29 09:52:45 hql024707 kernel:  ? inetdev_event+0x3d/0x6f0
Nov 29 09:52:45 hql024707 kernel:  ? iwl_mvm_recalc_multicast+0x5d/0xa0 [iwlmvm]
Nov 29 09:52:45 hql024707 kernel:  notifier_call_chain+0x5a/0xd0
Nov 29 09:52:45 hql024707 kernel:  __dev_close_many+0x5f/0x110
Nov 29 09:52:45 hql024707 kernel:  dev_close_many+0x8f/0x150
Nov 29 09:52:45 hql024707 kernel:  dev_close+0x83/0xb0
Nov 29 09:52:45 hql024707 kernel:  cfg80211_shutdown_all_interfaces+0x4d/0xf0 [cfg80211]
Nov 29 09:52:45 hql024707 kernel:  cfg80211_rfkill_set_block+0x2a/0x40 [cfg80211]
Nov 29 09:52:45 hql024707 kernel:  rfkill_set_block+0x91/0x160 [rfkill]
Nov 29 09:52:45 hql024707 kernel:  rfkill_fop_write+0x161/0x200 [rfkill]
Nov 29 09:52:45 hql024707 kernel:  vfs_write+0xef/0x420
Nov 29 09:52:45 hql024707 kernel:  ? __fget_light+0x99/0x100
Nov 29 09:52:45 hql024707 kernel:  ksys_write+0xbb/0xf0
Nov 29 09:52:45 hql024707 kernel:  do_syscall_64+0x5d/0x90
Nov 29 09:52:45 hql024707 kernel:  ? syscall_exit_to_user_mode+0x2b/0x40
Nov 29 09:52:45 hql024707 kernel:  ? do_syscall_64+0x6c/0x90
Nov 29 09:52:45 hql024707 kernel:  ? exit_to_user_mode_prepare+0x142/0x1f0
Nov 29 09:52:45 hql024707 kernel:  ? syscall_exit_to_user_mode+0x2b/0x40
Nov 29 09:52:45 hql024707 kernel:  ? do_syscall_64+0x6c/0x90
Nov 29 09:52:45 hql024707 kernel:  ? do_syscall_64+0x6c/0x90
Nov 29 09:52:45 hql024707 kernel:  ? syscall_exit_to_user_mode+0x2b/0x40
Nov 29 09:52:45 hql024707 kernel:  ? do_syscall_64+0x6c/0x90
Nov 29 09:52:45 hql024707 kernel:  ? do_syscall_64+0x6c/0x90
Nov 29 09:52:45 hql024707 kernel:  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
Nov 29 09:52:45 hql024707 kernel: RIP: 0033:0x7fb50f1adc6d
Nov 29 09:52:45 hql024707 kernel: Code: e5 48 83 ec 20 48 89 55 e8 48 89 75 f0 89 7d f8 e8 b8 1d f8 ff 48 8b 55 e8 48 8b 75 f0 41 89>
Nov 29 09:52:45 hql024707 kernel: RSP: 002b:00007ffe89561910 EFLAGS: 00000293 ORIG_RAX: 0000000000000001
Nov 29 09:52:45 hql024707 kernel: RAX: ffffffffffffffda RBX: 0000000000000029 RCX: 00007fb50f1adc6d
Nov 29 09:52:45 hql024707 kernel: RDX: 0000000000000008 RSI: 00007ffe89561958 RDI: 0000000000000029
Nov 29 09:52:45 hql024707 kernel: RBP: 00007ffe89561930 R08: 0000000000000000 R09: 0000000000000007
Nov 29 09:52:45 hql024707 kernel: R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000000
Nov 29 09:52:45 hql024707 kernel: R13: 000056426be6f070 R14: 0000000000000000 R15: 000056426be034c0
Nov 29 09:52:45 hql024707 kernel:  </TASK>
Nov 29 09:52:45 hql024707 kernel: ---[ end trace 0000000000000000 ]---
Comment 1 Bagas Sanjaya 2023-11-29 23:29:04 UTC 
(In reply to Linus Lotz from comment #0)
> Rfkilling the iwlwifi can lock up my machine, rfkill will not respond and
> other networking related things (e.g. ip a or ip r)
> I see this on 6.5.12 and 6.6.2 (fedora kernel). On 6.5.11 it does not lockup
> AFAICT, but also shows kernel oops.
> WiFi Hardware: AC 8265
> 

Can you check latest mainline (currently v6.7-rc3)?
Comment 2 Darrell Enns 2023-11-30 08:09:14 UTC 
I'm seeing the same issue. I tested on 6.6.2, 6.6.3, and mainline 6.7-rc3. Error log from 6.7-rc3 is below. Wi-Fi interface is 8265 as well.

iwlwifi 0000:02:00.0: fail to flush all tx fifo queues Q 5
iwlwifi 0000:02:00.0: Queue 5 is active on fifo 3 and stuck for 10000 ms. SW [8, 9] HW [9, 9] FH TRB=0x080305008
iwlwifi 0000:02:00.0: fail to flush all tx fifo queues Q 5
iwlwifi 0000:02:00.0: Queue 5 is active on fifo 3 and stuck for 10000 ms. SW [8, 9] HW [9, 9] FH TRB=0x080305008
------------[ cut here ]------------
WARNING: CPU: 0 PID: 649 at net/mac80211/sta_info.c:1460 __sta_info_destroy_part2+0x153/0x180 [mac80211]
Modules linked in: ccm rfcomm snd_seq_dummy snd_hrtimer snd_seq snd_seq_device xt_MASQUERADE xt_conntrack xt_CHECKSUM ipt_REJECT xt_tcpudp nft_compat nf_nat_tftp nf_conntrack_tftp bridge stp llc>
 processor_thermal_wt_hint videobuf2_memops mac80211 btbcm videobuf2_v4l2 snd_hda_codec processor_thermal_rfim btmtk processor_thermal_rapl rapl intel_rapl_msr hid_multitouch libarc4 snd_hda_cor>
 fuse loop ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 dm_crypt cbc encrypted_keys trusted asn1_encoder tee dm_mod crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyva>
CPU: 0 PID: 649 Comm: NetworkManager Not tainted 6.7.0-rc3-1-mainline #1 b8c026e389a4dc995d566501cbd0917d68610b9f
Hardware name: Dell Inc. Latitude 7490/0KP0FT, BIOS 1.15.1 06/23/2020
RIP: 0010:__sta_info_destroy_part2+0x153/0x180 [mac80211]
Code: bb d4 00 00 00 00 0f 84 66 ff ff ff 45 31 c0 b9 01 00 00 00 48 89 da 4c 89 ee 4c 89 e7 e8 a5 85 ff ff 85 c0 0f 84 48 ff ff ff <0f> 0b e9 41 ff ff ff 41 0f b6 d6 be 03 00 00 00 48 89 df e8 >
RSP: 0018:ffffc90000fb3818 EFLAGS: 00010282
RAX: 00000000ffffff92 RBX: ffff88812597e000 RCX: 0000000000000027
RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88810f241ff8
RBP: 0000000000000000 R08: 0000000000000000 R09: ffffc90000fb3370
R10: 0000000000000003 R11: ffffffff8beca728 R12: ffff88810f240900
R13: ffff8881080c89c0 R14: 0000000000000000 R15: ffff88810f240f18
FS:  00007fdb50c7c200(0000) GS:ffff888226400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1b0f1ff2d0 CR3: 000000010023a004 CR4: 00000000003706f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 ? __sta_info_destroy_part2+0x153/0x180 [mac80211 12acca448a201f2f84624f1173efdd3e4dced1a8]
 ? __warn+0x81/0x130
 ? __sta_info_destroy_part2+0x153/0x180 [mac80211 12acca448a201f2f84624f1173efdd3e4dced1a8]
 ? report_bug+0x171/0x1a0
 ? handle_bug+0x3c/0x80
 ? exc_invalid_op+0x17/0x70
 ? asm_exc_invalid_op+0x1a/0x20
 ? __sta_info_destroy_part2+0x153/0x180 [mac80211 12acca448a201f2f84624f1173efdd3e4dced1a8]
 ? __sta_info_destroy_part2+0x14b/0x180 [mac80211 12acca448a201f2f84624f1173efdd3e4dced1a8]
 __sta_info_flush+0x15d/0x1e0 [mac80211 12acca448a201f2f84624f1173efdd3e4dced1a8]
 ieee80211_set_disassoc+0x13b/0x600 [mac80211 12acca448a201f2f84624f1173efdd3e4dced1a8]
 ieee80211_mgd_deauth+0x13f/0x3f0 [mac80211 12acca448a201f2f84624f1173efdd3e4dced1a8]
 cfg80211_mlme_deauth+0xb7/0x1d0 [cfg80211 28f00249cf15d1e73a3d187f2041a7f5e689b19c]
 cfg80211_mlme_down+0x65/0x90 [cfg80211 28f00249cf15d1e73a3d187f2041a7f5e689b19c]
 cfg80211_disconnect+0x177/0x1e0 [cfg80211 28f00249cf15d1e73a3d187f2041a7f5e689b19c]
 cfg80211_netdev_notifier_call+0x121/0x4a0 [cfg80211 28f00249cf15d1e73a3d187f2041a7f5e689b19c]
 ? psi_task_switch+0x122/0x230
 ? iwl_mvm_send_cmd+0x1f/0x60 [iwlmvm fb2f7d6420fb3b8490507cc18e4d4bfa493c3bef]
 ? iwl_mvm_mc_iface_iterator+0xcd/0xf0 [iwlmvm fb2f7d6420fb3b8490507cc18e4d4bfa493c3bef]
 ? __iterate_interfaces+0x9d/0x110 [mac80211 12acca448a201f2f84624f1173efdd3e4dced1a8]
 ? __pfx_iwl_mvm_mc_iface_iterator+0x10/0x10 [iwlmvm fb2f7d6420fb3b8490507cc18e4d4bfa493c3bef]
 ? iwl_mvm_send_cmd_pdu+0x62/0xb0 [iwlmvm fb2f7d6420fb3b8490507cc18e4d4bfa493c3bef]
 ? rtnl_is_locked+0x15/0x20
 ? inetdev_event+0x3d/0x6f0
 notifier_call_chain+0x5a/0xd0
 __dev_close_many+0x5f/0x110
 dev_close_many+0x8f/0x150
 dev_close+0x83/0xb0
 cfg80211_shutdown_all_interfaces+0x4d/0xf0 [cfg80211 28f00249cf15d1e73a3d187f2041a7f5e689b19c]
 cfg80211_rfkill_set_block+0x2a/0x40 [cfg80211 28f00249cf15d1e73a3d187f2041a7f5e689b19c]
 rfkill_set_block+0x91/0x160 [rfkill 5da52e881c8a94c4bd7c85bdc4b5bc876cbc9bf6]
 rfkill_fop_write+0x161/0x200 [rfkill 5da52e881c8a94c4bd7c85bdc4b5bc876cbc9bf6]
 vfs_write+0xef/0x400
 ? __fget_light+0x85/0x100
 ksys_write+0xbb/0xf0
 do_syscall_64+0x61/0xe0
 ? do_sys_openat2+0x97/0xe0
 ? syscall_exit_to_user_mode+0x2b/0x40
 ? do_syscall_64+0x70/0xe0
 ? do_syscall_64+0x70/0xe0
 ? syscall_exit_to_user_mode+0x2b/0x40
 ? do_syscall_64+0x70/0xe0
 ? do_syscall_64+0x70/0xe0
 ? do_syscall_64+0x70/0xe0
 ? do_syscall_64+0x70/0xe0
 ? do_syscall_64+0x70/0xe0
 entry_SYSCALL_64_after_hwframe+0x6e/0x76
RIP: 0033:0x7fdb51b7006f
Code: 89 54 24 18 48 89 74 24 10 89 7c 24 08 e8 19 4d f8 ff 48 8b 54 24 18 48 8b 74 24 10 41 89 c0 8b 7c 24 08 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 31 44 89 c7 48 89 44 24 08 e8 6c 4d f8 >
RSP: 002b:00007ffe6b7be030 EFLAGS: 00000293 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000016 RCX: 00007fdb51b7006f
RDX: 0000000000000008 RSI: 00007ffe6b7be078 RDI: 0000000000000016
RBP: 00007ffe6b7be0e0 R08: 0000000000000000 R09: 0000000000000007
R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000000
R13: 0000000000000000 R14: 0000559c42229b60 R15: 0000559c42270170
 </TASK>
---[ end trace 0000000000000000 ]---
Comment 3 The Linux kernel's regression tracker (Thorsten Leemhuis) 2023-12-01 08:07:24 UTC 
FWIW, this smells like some mainline commit broke things and was backported to stable kernels. A bisection between 6.5.11 and 6.5.12 would really help to confirm this theory and should be relative quick to manage. Something like that will likely be needed unless the intel wifi developers show up with an idea of what might be cause this.
Comment 4 Mike Pagano 2023-12-01 23:15:28 UTC 
One of our user's appears to be having similar issues

All attachments are on this bug along with the result of a git bisect.

https://bugs.gentoo.org/918128

I can reattach everything here, if preferable.
Comment 5 The Linux kernel's regression tracker (Thorsten Leemhuis) 2023-12-02 05:23:50 UTC 
(In reply to Mike Pagano from comment #4)
> I can reattach everything here, if preferable.

At least summing things up here with a link or two to attachments there likely  would likely be the best.

But FWIW, I briefly looked and there is one think that might be good to know: is reverting commit 37fb29bd1f90 (which seems to be the culprit) on 6.7-rc fixing the issue?
Comment 6 Mike Pagano 2023-12-02 13:52:57 UTC 
Original Description:

Upon upgrading to 6.6.2, the rfkill switch on one of my laptops quit working correctly.  The switch on the laptop that quit working is an actual hardblock rfkill switch.  Upon trying to disconnect the Wifi, everything freezes up and the only solution is to forcibly power off the machine.  It just hangs; can't check logs, can't issue an unblock via rfkill, can't do anything at all related the network, etc.

Bisect result between 6.6.1 and 6.6.2

-----------------------------------------------------------------

f1f2e068bbe7783eff75ab85ea8566084b138aed is the first bad commit
commit f1f2e068bbe7783eff75ab85ea8566084b138aed
Author: Johannes Berg <johannes.berg@intel.com>
Date:   Tue Oct 17 12:16:43 2023 +0300

    wifi: iwlwifi: pcie: synchronize IRQs before NAPI
    
    [ Upstream commit 37fb29bd1f90f16d1abc95c0e9f0ff8eec9829ad ]
    
    When we want to synchronize the NAPI, which was added in
    commit 5af2bb3168db ("wifi: iwlwifi: call napi_synchronize()
    before freeing rx/tx queues"), we also need to make sure we
    can't actually reschedule the NAPI. Yes, this happens while
    interrupts are disabled, but interrupts may still be running
    or pending. Also call iwl_pcie_synchronize_irqs() to ensure
    we won't reschedule the NAPI.
    
    Fixes: 4cf2f5904d97 ("iwlwifi: queue: avoid memory leak in reset flow")
    Signed-off-by: Johannes Berg <johannes.berg@intel.com>
    Signed-off-by: Gregory Greenman <gregory.greenman@intel.com>
    Link: https://lore.kernel.org/r/20231017115047.a0f4104b479a.Id5c50a944f709092aa6256e32d8c63b2b8d8d3ac@changeid
    Signed-off-by: Johannes Berg <johannes.berg@intel.com>
    Signed-off-by: Sasha Levin <sashal@kernel.org>

 drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c | 1 +
 drivers/net/wireless/intel/iwlwifi/pcie/trans.c      | 1 +
 2 files changed, 2 insertions(+)
Comment 7 Mike Pagano 2023-12-02 13:54:21 UTC 
Kernel log


Nov 24 07:28:50 Gentoo-Xmonad kernel: ------------[ cut here ]------------
Nov 24 07:28:50 Gentoo-Xmonad kernel: WARNING: CPU: 1 PID: 41 at net/mac80211/sta_info.c:1442 __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel: Modules linked in: ccm ipt_REJECT nf_reject_ipv4 xt_LOG nf_log_syslog xt_limit xt_addrtype xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip6table_filter ip6_tables iptable_filter ip_tables iwlmvm i915 mac80211 libarc4 i2c_algo_bit drm_buddy ttm snd_hda_codec_idt snd_hda_codec_generic drm_display_helper dell_laptop snd_hda_intel dell_wmi iwlwifi iTCO_wdt snd_intel_dspcfg ledtrig_audio intel_pmc_bxt cec joydev coretemp dell_smbios snd_intel_sdw_acpi gpio_ich iTCO_vendor_support wmi_bmof sparse_keymap dell_wmi_descriptor dcdbas sha512_ssse3 cfg80211 ums_realtek snd_hda_codec serio_raw pcspkr acpi_cpufreq i2c_i801 drm_kms_helper snd_hda_core i2c_smbus snd_hwdep lpc_ich rfkill snd_pcm drm snd_timer snd video soundcore wmi dm_crypt trusted asn1_encoder uas usb_storage
Nov 24 07:28:50 Gentoo-Xmonad kernel: CPU: 1 PID: 41 Comm: kworker/1:1 Not tainted 6.6.2-gentoo #1
Nov 24 07:28:50 Gentoo-Xmonad kernel: Hardware name: Dell Inc. Inspiron 1545                   /0G848F, BIOS A14 12/07/2009
Nov 24 07:28:50 Gentoo-Xmonad kernel: Workqueue: events cfg80211_rfkill_block_work [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel: RIP: 0010:__sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel: Code: bb d4 00 00 00 00 0f 84 68 ff ff ff 45 31 c0 b9 01 00 00 00 48 89 da 4c 89 f6 4c 89 e7 e8 23 88 ff ff 85 c0 0f 84 4a ff ff ff <0f> 0b e9 43 ff ff ff 41 0f b6 d5 be 03 00 00 00 48 89 df e8 83 b6
Nov 24 07:28:50 Gentoo-Xmonad kernel: RSP: 0018:ffffa99fc01679a0 EFLAGS: 00010282
Nov 24 07:28:50 Gentoo-Xmonad kernel: RAX: 00000000ffffff92 RBX: ffff9d2347e12000 RCX: 0000000000000027
Nov 24 07:28:50 Gentoo-Xmonad kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9d23482d2098
Nov 24 07:28:50 Gentoo-Xmonad kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: ffffffff9885a6e0
Nov 24 07:28:50 Gentoo-Xmonad kernel: R10: ffffa99fc0167538 R11: ffffffff9893a728 R12: ffff9d23482d0900
Nov 24 07:28:50 Gentoo-Xmonad kernel: R13: ffff9d2347e12ab8 R14: ffff9d234a5a49c0 R15: ffff9d234a5a49c0
Nov 24 07:28:50 Gentoo-Xmonad kernel: FS:  0000000000000000(0000) GS:ffff9d2457d00000(0000) knlGS:0000000000000000
Nov 24 07:28:50 Gentoo-Xmonad kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 24 07:28:50 Gentoo-Xmonad kernel: CR2: 000055c1dfaee140 CR3: 000000003f832000 CR4: 00000000000406a0
Nov 24 07:28:50 Gentoo-Xmonad kernel: Call Trace:
Nov 24 07:28:50 Gentoo-Xmonad kernel:  <TASK>
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __warn+0x7d/0x130
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? report_bug+0x16d/0x1a0
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? handle_bug+0x3c/0x80
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? exc_invalid_op+0x13/0x60
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? asm_exc_invalid_op+0x16/0x20
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  __sta_info_flush+0x12e/0x1a0 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ieee80211_set_disassoc+0x157/0x5f0 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ieee80211_mgd_deauth+0x13b/0x3d0 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? enqueue_entity+0xb4/0x330
Nov 24 07:28:50 Gentoo-Xmonad kernel:  cfg80211_mlme_deauth+0xa7/0x1a0 [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  cfg80211_mlme_down+0x61/0x80 [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  cfg80211_disconnect+0x184/0x1f0 [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  cfg80211_netdev_notifier_call+0x12a/0x4a0 [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __wake_up_common_lock+0x7f/0xa0
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? select_task_rq_fair+0x5d1/0x1590
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? ep_poll_callback+0x230/0x280
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __wake_up_common+0x74/0x140
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? sched_clock_cpu+0xee/0x190
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __smp_call_single_queue+0xa9/0x110
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? ttwu_queue_wakelist+0xe6/0x100
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? try_to_wake_up+0x178/0x4f0
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? rtnl_is_locked+0x11/0x20
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? inetdev_event+0x39/0x6f0
Nov 24 07:28:50 Gentoo-Xmonad kernel:  notifier_call_chain+0x58/0xc0
Nov 24 07:28:50 Gentoo-Xmonad kernel:  __dev_close_many+0x4f/0xf0
Nov 24 07:28:50 Gentoo-Xmonad kernel:  dev_close_many+0x6b/0x100
Nov 24 07:28:50 Gentoo-Xmonad kernel:  dev_close+0x39/0x60
Nov 24 07:28:50 Gentoo-Xmonad kernel:  cfg80211_shutdown_all_interfaces+0x49/0xf0 [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  cfg80211_rfkill_block_work+0x1a/0x20 [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  process_one_work+0x165/0x330
Nov 24 07:28:50 Gentoo-Xmonad kernel:  worker_thread+0x2f1/0x410
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __pfx_worker_thread+0x10/0x10
Nov 24 07:28:50 Gentoo-Xmonad kernel:  kthread+0xe4/0x110
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __pfx_kthread+0x10/0x10
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ret_from_fork+0x30/0x50
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __pfx_kthread+0x10/0x10
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ret_from_fork_asm+0x1b/0x30
Nov 24 07:28:50 Gentoo-Xmonad kernel:  </TASK>
Nov 24 07:28:50 Gentoo-Xmonad kernel: ---[ end trace 0000000000000000 ]---
Nov 24 07:28:56 Gentoo-Xmonad kernel: iwlwifi 0000:0c:00.0: Queue 5 is active on fifo 3 and stuck for 10000 ms. SW [6, 7] HW [6, 7] FH TRB=0x080305006
Comment 8 Mike Pagano 2023-12-02 13:54:21 UTC 
Kernel log


Nov 24 07:28:50 Gentoo-Xmonad kernel: ------------[ cut here ]------------
Nov 24 07:28:50 Gentoo-Xmonad kernel: WARNING: CPU: 1 PID: 41 at net/mac80211/sta_info.c:1442 __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel: Modules linked in: ccm ipt_REJECT nf_reject_ipv4 xt_LOG nf_log_syslog xt_limit xt_addrtype xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip6table_filter ip6_tables iptable_filter ip_tables iwlmvm i915 mac80211 libarc4 i2c_algo_bit drm_buddy ttm snd_hda_codec_idt snd_hda_codec_generic drm_display_helper dell_laptop snd_hda_intel dell_wmi iwlwifi iTCO_wdt snd_intel_dspcfg ledtrig_audio intel_pmc_bxt cec joydev coretemp dell_smbios snd_intel_sdw_acpi gpio_ich iTCO_vendor_support wmi_bmof sparse_keymap dell_wmi_descriptor dcdbas sha512_ssse3 cfg80211 ums_realtek snd_hda_codec serio_raw pcspkr acpi_cpufreq i2c_i801 drm_kms_helper snd_hda_core i2c_smbus snd_hwdep lpc_ich rfkill snd_pcm drm snd_timer snd video soundcore wmi dm_crypt trusted asn1_encoder uas usb_storage
Nov 24 07:28:50 Gentoo-Xmonad kernel: CPU: 1 PID: 41 Comm: kworker/1:1 Not tainted 6.6.2-gentoo #1
Nov 24 07:28:50 Gentoo-Xmonad kernel: Hardware name: Dell Inc. Inspiron 1545                   /0G848F, BIOS A14 12/07/2009
Nov 24 07:28:50 Gentoo-Xmonad kernel: Workqueue: events cfg80211_rfkill_block_work [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel: RIP: 0010:__sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel: Code: bb d4 00 00 00 00 0f 84 68 ff ff ff 45 31 c0 b9 01 00 00 00 48 89 da 4c 89 f6 4c 89 e7 e8 23 88 ff ff 85 c0 0f 84 4a ff ff ff <0f> 0b e9 43 ff ff ff 41 0f b6 d5 be 03 00 00 00 48 89 df e8 83 b6
Nov 24 07:28:50 Gentoo-Xmonad kernel: RSP: 0018:ffffa99fc01679a0 EFLAGS: 00010282
Nov 24 07:28:50 Gentoo-Xmonad kernel: RAX: 00000000ffffff92 RBX: ffff9d2347e12000 RCX: 0000000000000027
Nov 24 07:28:50 Gentoo-Xmonad kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9d23482d2098
Nov 24 07:28:50 Gentoo-Xmonad kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: ffffffff9885a6e0
Nov 24 07:28:50 Gentoo-Xmonad kernel: R10: ffffa99fc0167538 R11: ffffffff9893a728 R12: ffff9d23482d0900
Nov 24 07:28:50 Gentoo-Xmonad kernel: R13: ffff9d2347e12ab8 R14: ffff9d234a5a49c0 R15: ffff9d234a5a49c0
Nov 24 07:28:50 Gentoo-Xmonad kernel: FS:  0000000000000000(0000) GS:ffff9d2457d00000(0000) knlGS:0000000000000000
Nov 24 07:28:50 Gentoo-Xmonad kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 24 07:28:50 Gentoo-Xmonad kernel: CR2: 000055c1dfaee140 CR3: 000000003f832000 CR4: 00000000000406a0
Nov 24 07:28:50 Gentoo-Xmonad kernel: Call Trace:
Nov 24 07:28:50 Gentoo-Xmonad kernel:  <TASK>
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __warn+0x7d/0x130
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? report_bug+0x16d/0x1a0
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? handle_bug+0x3c/0x80
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? exc_invalid_op+0x13/0x60
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? asm_exc_invalid_op+0x16/0x20
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __sta_info_destroy_part2+0x235/0x300 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  __sta_info_flush+0x12e/0x1a0 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ieee80211_set_disassoc+0x157/0x5f0 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ieee80211_mgd_deauth+0x13b/0x3d0 [mac80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? enqueue_entity+0xb4/0x330
Nov 24 07:28:50 Gentoo-Xmonad kernel:  cfg80211_mlme_deauth+0xa7/0x1a0 [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  cfg80211_mlme_down+0x61/0x80 [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  cfg80211_disconnect+0x184/0x1f0 [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  cfg80211_netdev_notifier_call+0x12a/0x4a0 [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __wake_up_common_lock+0x7f/0xa0
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? select_task_rq_fair+0x5d1/0x1590
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? ep_poll_callback+0x230/0x280
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __wake_up_common+0x74/0x140
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? sched_clock_cpu+0xee/0x190
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __smp_call_single_queue+0xa9/0x110
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? ttwu_queue_wakelist+0xe6/0x100
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? try_to_wake_up+0x178/0x4f0
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? rtnl_is_locked+0x11/0x20
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? inetdev_event+0x39/0x6f0
Nov 24 07:28:50 Gentoo-Xmonad kernel:  notifier_call_chain+0x58/0xc0
Nov 24 07:28:50 Gentoo-Xmonad kernel:  __dev_close_many+0x4f/0xf0
Nov 24 07:28:50 Gentoo-Xmonad kernel:  dev_close_many+0x6b/0x100
Nov 24 07:28:50 Gentoo-Xmonad kernel:  dev_close+0x39/0x60
Nov 24 07:28:50 Gentoo-Xmonad kernel:  cfg80211_shutdown_all_interfaces+0x49/0xf0 [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  cfg80211_rfkill_block_work+0x1a/0x20 [cfg80211]
Nov 24 07:28:50 Gentoo-Xmonad kernel:  process_one_work+0x165/0x330
Nov 24 07:28:50 Gentoo-Xmonad kernel:  worker_thread+0x2f1/0x410
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __pfx_worker_thread+0x10/0x10
Nov 24 07:28:50 Gentoo-Xmonad kernel:  kthread+0xe4/0x110
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __pfx_kthread+0x10/0x10
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ret_from_fork+0x30/0x50
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ? __pfx_kthread+0x10/0x10
Nov 24 07:28:50 Gentoo-Xmonad kernel:  ret_from_fork_asm+0x1b/0x30
Nov 24 07:28:50 Gentoo-Xmonad kernel:  </TASK>
Nov 24 07:28:50 Gentoo-Xmonad kernel: ---[ end trace 0000000000000000 ]---
Nov 24 07:28:56 Gentoo-Xmonad kernel: iwlwifi 0000:0c:00.0: Queue 5 is active on fifo 3 and stuck for 10000 ms. SW [6, 7] HW [6, 7] FH TRB=0x080305006
Comment 9 Mike Pagano 2023-12-02 18:29:25 UTC 
Our user reports reverting that commit from the bisect resulted in success.

"Well,... THAT WORKED!!!

I can enable/disable the rfkill switch like I normally would.  The iw event output is back to normal.

...  "
Comment 10 Johannes Berg 2023-12-04 10:47:15 UTC 
OK, thanks for that. I'm a bit confused, the warning shouldn't deadlock anything?

The warning should be fixed by https://p.sipsolutions.net/a17c13bdfb41671a.txt (but I'm not sure where the patch is stuck now.)

But I don't know why it's also dead-locking, which seems unrelated.
Comment 11 Johannes Berg 2023-12-06 20:35:49 UTC 
Created attachment 305551 [details]
attempted fix

Can you try the attached patch? Seems this can only happen on old devices.
Comment 12 Mike Pagano 2023-12-07 17:31:57 UTC 
(In reply to Johannes Berg from comment #11)
> Created attachment 305551 [details]
> attempted fix
> 
> Can you try the attached patch? Seems this can only happen on old devices.

User has reported success with this patch. Thank-you for your work.
Comment 13 The Linux kernel's regression tracker (Thorsten Leemhuis) 2023-12-15 10:10:52 UTC 
(In reply to Mike Pagano from comment #12)
> (In reply to Johannes Berg from comment #11)
> > 
> > Can you try the attached patch? Seems this can only happen on old devices.
> User has reported success with this patch. Thank-you for your work.

Johannes, what's the plan forward wrt to merging your fix? It of course would be great if more people would have tested it (any volunteers?), but it seems we are a bit out of luck here.
Comment 14 Kalle Valo 2023-12-18 10:28:50 UTC 
(In reply to The Linux kernel's regression tracker (Thorsten Leemhuis) from comment #13)
> (In reply to Mike Pagano from comment #12)
> > (In reply to Johannes Berg from comment #11)
> > > 
> > > Can you try the attached patch? Seems this can only happen on old
> devices.
> > User has reported success with this patch. Thank-you for your work.
> 
> Johannes, what's the plan forward wrt to merging your fix? It of course
> would be great if more people would have tested it (any volunteers?), but it
> seems we are a bit out of luck here.

It looks like that the patch is under review:

https://patchwork.kernel.org/project/linux-wireless/patch/20231215111335.59aab00baed7.Iadfe154d6248e7f9dfd69522e5429dbbd72925d7@changeid/

I'm guessing it will go to wireless tree this week.
Comment 15 The Linux kernel's regression tracker (Thorsten Leemhuis) 2023-12-18 10:35:26 UTC 
(In reply to Kalle Valo from comment #14)
> It looks like that the patch is under review:

Yeah, thx for mentioning it, I noticed that myself yesterday; was a kind of race condition, that submission afaics happened three minutes after my comment (unless by brain screwed up when converting timezones) :-D