Bug 205719

Summary: [v5.4 regression, bisected] Wireless-AC 9560 no longer initializes
Product: Drivers Reporter: Anders Kaseorg (andersk)
Component: network-wirelessAssignee: DO NOT USE - assign "network-wireless-intel" component instead (linuxwifi)
Status: CLOSED CODE_FIX    
Severity: normal CC: leander, leho, luca
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: v5.4-rc4 Subsystem:
Regression: Yes Bisected commit-id:
Attachments: full dmesg log
Patch to fix the power gating issue.

Description Anders Kaseorg 2019-11-30 07:03:26 UTC 
Created attachment 286125 [details]
full dmesg log

Kernel v5.4-rc4 and later breaks Wi-Fi on my ThinkPad X1 Yoga Gen 4 with Core i7-10710U (Comet Lake) and Intel Wireless-AC 9560, running Fedora 32 rawhide.  I get the below errors in `dmesg` and no devices in `iw list`.  According to `git bisect`:

aa0cc7dde17bb6b8cc533bbcfe3f53d70e0dd269 is the first bad commit
commit aa0cc7dde17bb6b8cc533bbcfe3f53d70e0dd269
Author: Luca Coelho <luciano.coelho@intel.com>
Date:   Tue Oct 8 13:21:02 2019 +0300

    iwlwifi: pcie: change qu with jf devices to use qu configuration
    
    There were a bunch of devices with qu and jf that were loading the
    configuration with pu and jf, which is wrong.  Fix them all
    accordingly.  Additionally, remove 0x1010 and 0x1210 subsytem IDs from
    the list, since they are obviously wrong, and 0x0044 and 0x0244, which
    were duplicate.
    
    Cc: stable@vger.kernel.org # 5.1+
    Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

 drivers/net/wireless/intel/iwlwifi/pcie/drv.c | 274 +++++++++++++-------------
 1 file changed, 137 insertions(+), 137 deletions(-)

iwlwifi messages from `dmesg` running this commit (full `dmesg` log attached):

[   17.069310] Intel(R) Wireless WiFi driver for Linux
[   17.069311] Copyright(c) 2003- 2015 Intel Corporation
[   17.069424] iwlwifi 0000:00:14.3: enabling device (0000 -> 0002)
[   17.142522] iwlwifi 0000:00:14.3: Direct firmware load for iwlwifi-Qu-b0-jf-b0-50.ucode failed with error -2
[   17.142535] iwlwifi 0000:00:14.3: Direct firmware load for iwlwifi-Qu-b0-jf-b0-49.ucode failed with error -2
[   17.144139] iwlwifi 0000:00:14.3: TLV_FW_FSEQ_VERSION: FSEQ Version: 43.2.23.17
[   17.144326] iwlwifi 0000:00:14.3: loaded firmware version 48.4fa0041f.0 op_mode iwlmvm
[   17.515317] iwlwifi 0000:00:14.3: Detected Intel(R) Wireless-AC 9560 160MHz, REV=0x354
[   18.536212] iwlwifi 0000:00:14.3: Collecting data: trigger 15 fired.
[   18.537296] iwlwifi 0000:00:14.3: Start IWL Error Log Dump:
[   18.537297] iwlwifi 0000:00:14.3: Status: 0x00000000, count: -472949507
[   18.537298] iwlwifi 0000:00:14.3: Loaded firmware version: 48.4fa0041f.0
[   18.537300] iwlwifi 0000:00:14.3: 0x5E27CEB7 | ADVANCED_SYSASSERT          
[   18.537301] iwlwifi 0000:00:14.3: 0x99BD7889 | trm_hw_status0
[   18.537302] iwlwifi 0000:00:14.3: 0x4D0F41F3 | trm_hw_status1
[   18.537303] iwlwifi 0000:00:14.3: 0x8BA52979 | branchlink2
[   18.537304] iwlwifi 0000:00:14.3: 0xFD6E1430 | interruptlink1
[   18.537305] iwlwifi 0000:00:14.3: 0x149920E5 | interruptlink2
[   18.537306] iwlwifi 0000:00:14.3: 0xE519C8AB | data1
[   18.537307] iwlwifi 0000:00:14.3: 0x5DE91B89 | data2
[   18.537308] iwlwifi 0000:00:14.3: 0x8B0E1ECB | data3
[   18.537309] iwlwifi 0000:00:14.3: 0x5BC792CF | beacon time
[   18.537311] iwlwifi 0000:00:14.3: 0xF89BA0B4 | tsf low
[   18.537312] iwlwifi 0000:00:14.3: 0xE71AA131 | tsf hi
[   18.537313] iwlwifi 0000:00:14.3: 0xFC632697 | time gp1
[   18.537314] iwlwifi 0000:00:14.3: 0x570E0D24 | time gp2
[   18.537315] iwlwifi 0000:00:14.3: 0x8AF860E6 | uCode revision type
[   18.537317] iwlwifi 0000:00:14.3: 0xE44DBB4C | uCode version major
[   18.537318] iwlwifi 0000:00:14.3: 0xD80B71DB | uCode version minor
[   18.537319] iwlwifi 0000:00:14.3: 0x56C5E9D2 | hw version
[   18.537320] iwlwifi 0000:00:14.3: 0xE760D8C7 | board version
[   18.537321] iwlwifi 0000:00:14.3: 0x8939119F | hcmd
[   18.537322] iwlwifi 0000:00:14.3: 0xF66B47CE | isr0
[   18.537323] iwlwifi 0000:00:14.3: 0xB3D9D7D2 | isr1
[   18.537324] iwlwifi 0000:00:14.3: 0xBE2B5CB5 | isr2
[   18.537325] iwlwifi 0000:00:14.3: 0xB1CBE4BB | isr3
[   18.537325] iwlwifi 0000:00:14.3: 0xB4077859 | isr4
[   18.537326] iwlwifi 0000:00:14.3: 0x31DFFB1C | last cmd Id
[   18.537327] iwlwifi 0000:00:14.3: 0x767B4078 | wait_event
[   18.537329] iwlwifi 0000:00:14.3: 0x61478397 | l2p_control
[   18.537330] iwlwifi 0000:00:14.3: 0x7FFACEC3 | l2p_duration
[   18.537331] iwlwifi 0000:00:14.3: 0x1FEE6A21 | l2p_mhvalid
[   18.537332] iwlwifi 0000:00:14.3: 0x561176D1 | l2p_addr_match
[   18.537333] iwlwifi 0000:00:14.3: 0x3EC598E9 | lmpm_pmg_sel
[   18.537333] iwlwifi 0000:00:14.3: 0xDE45086D | timestamp
[   18.537334] iwlwifi 0000:00:14.3: 0x6D0A75E4 | flow_handler
[   18.537725] iwlwifi 0000:00:14.3: Start IWL Error Log Dump:
[   18.537726] iwlwifi 0000:00:14.3: Status: 0x00000000, count: 7
[   18.537727] iwlwifi 0000:00:14.3: 0x201013F1 | ADVANCED_SYSASSERT
[   18.537728] iwlwifi 0000:00:14.3: 0x00000000 | umac branchlink1
[   18.537729] iwlwifi 0000:00:14.3: 0xC008CF5C | umac branchlink2
[   18.537730] iwlwifi 0000:00:14.3: 0x00000000 | umac interruptlink1
[   18.537731] iwlwifi 0000:00:14.3: 0x00000000 | umac interruptlink2
[   18.537732] iwlwifi 0000:00:14.3: 0x00000003 | umac data1
[   18.537733] iwlwifi 0000:00:14.3: 0x20000302 | umac data2
[   18.537734] iwlwifi 0000:00:14.3: 0x01300202 | umac data3
[   18.537736] iwlwifi 0000:00:14.3: 0x00000030 | umac major
[   18.537737] iwlwifi 0000:00:14.3: 0x4FA0041F | umac minor
[   18.537738] iwlwifi 0000:00:14.3: 0x00005C90 | frame pointer
[   18.537739] iwlwifi 0000:00:14.3: 0xC0887F58 | stack pointer
[   18.537740] iwlwifi 0000:00:14.3: 0x00000000 | last host cmd
[   18.537741] iwlwifi 0000:00:14.3: 0x00000000 | isr status reg
[   18.537778] iwlwifi 0000:00:14.3: Fseq Registers:
[   18.537800] iwlwifi 0000:00:14.3: 0x00000003 | FSEQ_ERROR_CODE
[   18.537822] iwlwifi 0000:00:14.3: 0x00000000 | FSEQ_TOP_INIT_VERSION
[   18.537843] iwlwifi 0000:00:14.3: 0xBF3CB876 | FSEQ_CNVIO_INIT_VERSION
[   18.537865] iwlwifi 0000:00:14.3: 0x0000A384 | FSEQ_OTP_VERSION
[   18.537887] iwlwifi 0000:00:14.3: 0xB0C1F9DF | FSEQ_TOP_CONTENT_VERSION
[   18.537908] iwlwifi 0000:00:14.3: 0xCCCB76D7 | FSEQ_ALIVE_TOKEN
[   18.537930] iwlwifi 0000:00:14.3: 0x5EBBD77F | FSEQ_CNVI_ID
[   18.537951] iwlwifi 0000:00:14.3: 0xC01AA94A | FSEQ_CNVR_ID
[   18.537973] iwlwifi 0000:00:14.3: 0x20000302 | CNVI_AUX_MISC_CHIP
[   18.537997] iwlwifi 0000:00:14.3: 0x01300202 | CNVR_AUX_MISC_CHIP
[   18.538021] iwlwifi 0000:00:14.3: 0x0000485B | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[   18.538074] iwlwifi 0000:00:14.3: 0xA5A5A5A2 | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[   18.538271] iwlwifi 0000:00:14.3: SecBoot CPU1 Status: 0x5c01, CPU2 Status: 0x3
[   18.538272] iwlwifi 0000:00:14.3: Failed to start RT ucode: -110
[   18.538275] iwlwifi 0000:00:14.3: Firmware not running - cannot dump error
[   18.550654] iwlwifi 0000:00:14.3: Failed to run INIT ucode: -110
Comment 1 Anders Kaseorg 2019-11-30 23:11:43 UTC 
Oddly, it works fine in kernel 5.3.12-300.fc31.x86_64, even though aa0cc7dde17bb6b8cc533bbcfe3f53d70e0dd269 was backported as v5.3.8~76.  Was this problem already noticed and fixed in Fedora 31 but not rawhide or upstream?

More specific hardware information:

$ sudo lspci -vvnn -d ::0280
[sudo] password for anders: 
00:14.3 Network controller [0280]: Intel Corporation Device [8086:02f0]
	Subsystem: Intel Corporation Device [8086:0030]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at ea238000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [c8] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0
			ExtTag- RBE-
		DevCtl:	CorrErr- NonFatalErr- FatalErr- UnsupReq-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr+ NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
		DevCap2: Completion Timeout: Range B, TimeoutDis+, LTR+, OBFF Via WAKE#
			 AtomicOpsCap: 32bit- 64bit- 128bitCAS-
		DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis-, LTR+, OBFF Disabled
			 AtomicOpsCtl: ReqEn-
	Capabilities: [80] MSI-X: Enable+ Count=16 Masked-
		Vector table: BAR=0 offset=00002000
		PBA: BAR=0 offset=00003000
	Capabilities: [100 v1] Latency Tolerance Reporting
		Max snoop latency: 0ns
		Max no snoop latency: 0ns
	Capabilities: [164 v1] Vendor Specific Information: ID=0010 Rev=0 Len=014 <?>
	Kernel driver in use: iwlwifi
	Kernel modules: iwlwifi
Comment 2 Anders Kaseorg 2019-12-01 06:18:14 UTC 
I may have figured out the problem.  These two commits, which were merged independently of each other, actually conflict with each other semantically.

commit 809805a820c6445f7a701ded24fdc6bbc841d1e4
Author:     Luca Coelho <luciano.coelho@intel.com>
AuthorDate: Thu Aug 2 17:08:40 2018 +0300
Commit:     Luca Coelho <luciano.coelho@intel.com>
CommitDate: Fri Sep 6 15:42:13 2019 +0300

    iwlwifi: pcie: move some cfg mangling from trans_pcie_alloc to probe
    
    There were a couple of special handling to find the correct cfg inside
    iwl_trans_pcie_alloc().  Move them to iwl_pci_probe() so they're
    together with the rest of the decisions.
    
    Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

commit 968dcfb4905245dc64d65312c0d17692fa087b99
Author:     Luca Coelho <luciano.coelho@intel.com>
AuthorDate: Thu Aug 29 11:13:46 2019 +0300
Commit:     Kalle Valo <kvalo@codeaurora.org>
CommitDate: Tue Sep 3 16:50:59 2019 +0300

    iwlwifi: assign directly to iwl_trans->cfg in QuZ detection
    
    We were erroneously assigning the new configuration to a local
    variable cfg, but that was not being assigned to anything, so the
    change was getting lost.  Assign directly to iwl_trans->cfg instead.
    
    Fixes: 5a8c31aa6357 ("iwlwifi: pcie: fix recognition of QuZ devices")
    Cc: stable@vger.kernel.org # 5.2
    Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
    Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

Both commits attempt to fix the buggy dead assignments to the local variable ‘cfg’ introduced by 5a8c31aa63578cb0ff390a57537f1cb4b312a1ed, but they fix it in incompatible ways.  The former commit moves the ‘iwl_trans->cfg = cfg’ assignment later, while the latter commit changes the ‘cfg’ assignments to ‘iwl_trans->cfg’ assignments.  With both commits applied, we now have the opposite bug of dead assignments to ‘iwl_trans->cfg’.
Comment 3 Anders Kaseorg 2019-12-01 06:49:35 UTC 
Well, crap.  Reverting 968dcfb4905245dc64d65312c0d17692fa087b99 didn’t actually help me.  (It might still help someone, though, as the present state of that code is clearly wrong.)

This time I got

[   15.755673] Intel(R) Wireless WiFi driver for Linux
[   15.755673] Copyright(c) 2003- 2015 Intel Corporation
[   15.755802] iwlwifi 0000:00:14.3: enabling device (0000 -> 0002)
[   15.759704] idma64 idma64.1: Found Intel integrated DMA 64-bit
[   15.766218] iwlwifi 0000:00:14.3: Direct firmware load for iwlwifi-QuZ-a0-jf-b0-50.ucode failed with error -2
[   15.766417] iwlwifi 0000:00:14.3: Direct firmware load for iwlwifi-QuZ-a0-jf-b0-49.ucode failed with error -2
[   15.768024] iwlwifi 0000:00:14.3: TLV_FW_FSEQ_VERSION: FSEQ Version: 43.2.23.17
[   15.768769] iwlwifi 0000:00:14.3: loaded firmware version 48.4fa0041f.0 op_mode iwlmvm
[   16.194368] iwlwifi 0000:00:14.3: Detected Intel(R) Wireless-AC 9560 160MHz, REV=0x354
[   17.254707] iwlwifi 0000:00:14.3: Collecting data: trigger 15 fired.
[   17.505525] iwlwifi 0000:00:14.3: Start IWL Error Log Dump:
[   17.505527] iwlwifi 0000:00:14.3: Status: 0x00000000, count: 6
[   17.505528] iwlwifi 0000:00:14.3: Loaded firmware version: 48.4fa0041f.0
[   17.505530] iwlwifi 0000:00:14.3: 0x00000084 | NMI_INTERRUPT_UNKNOWN       
[   17.505531] iwlwifi 0000:00:14.3: 0x000022F0 | trm_hw_status0
[   17.505532] iwlwifi 0000:00:14.3: 0x00000000 | trm_hw_status1
[   17.505532] iwlwifi 0000:00:14.3: 0x004C2214 | branchlink2
[   17.505533] iwlwifi 0000:00:14.3: 0x004B0CE8 | interruptlink1
[   17.505534] iwlwifi 0000:00:14.3: 0x004B0CE8 | interruptlink2
[   17.505535] iwlwifi 0000:00:14.3: 0x00019786 | data1
[   17.505536] iwlwifi 0000:00:14.3: 0xFF000000 | data2
[   17.505536] iwlwifi 0000:00:14.3: 0xE0000008 | data3
[   17.505537] iwlwifi 0000:00:14.3: 0x00000000 | beacon time
[   17.505538] iwlwifi 0000:00:14.3: 0x000F9480 | tsf low
[   17.505539] iwlwifi 0000:00:14.3: 0x00000000 | tsf hi
[   17.505540] iwlwifi 0000:00:14.3: 0x00000000 | time gp1
[   17.505540] iwlwifi 0000:00:14.3: 0x000FF3A0 | time gp2
[   17.505541] iwlwifi 0000:00:14.3: 0x00000001 | uCode revision type
[   17.505542] iwlwifi 0000:00:14.3: 0x00000030 | uCode version major
[   17.505543] iwlwifi 0000:00:14.3: 0x4FA0041F | uCode version minor
[   17.505544] iwlwifi 0000:00:14.3: 0x00000351 | hw version
[   17.505545] iwlwifi 0000:00:14.3: 0x18089004 | board version
[   17.505545] iwlwifi 0000:00:14.3: 0x8003F502 | hcmd
[   17.505546] iwlwifi 0000:00:14.3: 0x00020000 | isr0
[   17.505547] iwlwifi 0000:00:14.3: 0x00000000 | isr1
[   17.505548] iwlwifi 0000:00:14.3: 0x00F00002 | isr2
[   17.505549] iwlwifi 0000:00:14.3: 0x00C0000C | isr3
[   17.505549] iwlwifi 0000:00:14.3: 0x00000000 | isr4
[   17.505550] iwlwifi 0000:00:14.3: 0x00000000 | last cmd Id
[   17.505551] iwlwifi 0000:00:14.3: 0x00019786 | wait_event
[   17.505552] iwlwifi 0000:00:14.3: 0x00000000 | l2p_control
[   17.505552] iwlwifi 0000:00:14.3: 0x00000000 | l2p_duration
[   17.505553] iwlwifi 0000:00:14.3: 0x00000000 | l2p_mhvalid
[   17.505554] iwlwifi 0000:00:14.3: 0x00000000 | l2p_addr_match
[   17.505555] iwlwifi 0000:00:14.3: 0x0000000B | lmpm_pmg_sel
[   17.505556] iwlwifi 0000:00:14.3: 0x00000000 | timestamp
[   17.505556] iwlwifi 0000:00:14.3: 0x00AC0000 | flow_handler
[   17.505589] iwlwifi 0000:00:14.3: Start IWL Error Log Dump:
[   17.505590] iwlwifi 0000:00:14.3: Status: 0x00000000, count: 7
[   17.505591] iwlwifi 0000:00:14.3: 0x20000066 | NMI_INTERRUPT_HOST
[   17.505592] iwlwifi 0000:00:14.3: 0x00000000 | umac branchlink1
[   17.505592] iwlwifi 0000:00:14.3: 0xC008CF5C | umac branchlink2
[   17.505593] iwlwifi 0000:00:14.3: 0x8048A78A | umac interruptlink1
[   17.505594] iwlwifi 0000:00:14.3: 0x8048A78A | umac interruptlink2
[   17.505595] iwlwifi 0000:00:14.3: 0x01000000 | umac data1
[   17.505596] iwlwifi 0000:00:14.3: 0x8048A78A | umac data2
[   17.505597] iwlwifi 0000:00:14.3: 0x00000000 | umac data3
[   17.505597] iwlwifi 0000:00:14.3: 0x00000030 | umac major
[   17.505598] iwlwifi 0000:00:14.3: 0x4FA0041F | umac minor
[   17.505599] iwlwifi 0000:00:14.3: 0x000FF399 | frame pointer
[   17.505600] iwlwifi 0000:00:14.3: 0xC0886284 | stack pointer
[   17.505601] iwlwifi 0000:00:14.3: 0x0000FFFF | last host cmd
[   17.505601] iwlwifi 0000:00:14.3: 0x00000000 | isr status reg
[   17.505617] iwlwifi 0000:00:14.3: Fseq Registers:
[   17.505619] iwlwifi 0000:00:14.3: 0x20000021 | FSEQ_ERROR_CODE
[   17.505622] iwlwifi 0000:00:14.3: 0x80260000 | FSEQ_TOP_INIT_VERSION
[   17.505624] iwlwifi 0000:00:14.3: 0x80020006 | FSEQ_CNVIO_INIT_VERSION
[   17.505626] iwlwifi 0000:00:14.3: 0x0000A384 | FSEQ_OTP_VERSION
[   17.505629] iwlwifi 0000:00:14.3: 0xB0C1F95F | FSEQ_TOP_CONTENT_VERSION
[   17.505631] iwlwifi 0000:00:14.3: 0x4552414E | FSEQ_ALIVE_TOKEN
[   17.505633] iwlwifi 0000:00:14.3: 0x20000302 | FSEQ_CNVI_ID
[   17.505636] iwlwifi 0000:00:14.3: 0x01300202 | FSEQ_CNVR_ID
[   17.505638] iwlwifi 0000:00:14.3: 0x20000302 | CNVI_AUX_MISC_CHIP
[   17.505643] iwlwifi 0000:00:14.3: 0x01300202 | CNVR_AUX_MISC_CHIP
[   17.505647] iwlwifi 0000:00:14.3: 0x0000485B | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[   17.505682] iwlwifi 0000:00:14.3: 0xA5A5A5A2 | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[   17.505747] iwlwifi 0000:00:14.3: SecBoot CPU1 Status: 0x5c17, CPU2 Status: 0x3
[   17.505748] iwlwifi 0000:00:14.3: Failed to start RT ucode: -110
[   17.505759] iwlwifi 0000:00:14.3: Firmware not running - cannot dump error
[   17.517946] iwlwifi 0000:00:14.3: Failed to run INIT ucode: -110
Comment 4 Anders Kaseorg 2019-12-01 23:39:11 UTC 
I isolated the second bug using a `git bisect` with 968dcfb4905245dc64d65312c0d17692fa087b99 reverted at each step.  The implicated commit is

commit 9a47cb988338796b70c544919a8b6ba1f2245edb
Author:     Luca Coelho <luciano.coelho@intel.com>
AuthorDate: Sat Oct 19 13:03:30 2019 +0300
Commit:     Kalle Valo <kvalo@codeaurora.org>
CommitDate: Wed Oct 23 13:31:33 2019 +0300

    iwlwifi: pcie: add workaround for power gating in integrated 22000
    
    Add a workaround that forces power gating to be enabled on integrated
    22000 devices.  This improves power saving in certain situations.
    
    Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
    Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

I then verified that my wireless works after reverting both 968dcfb4905245dc64d65312c0d17692fa087b99 and 9a47cb988338796b70c544919a8b6ba1f2245edb, on top of either v5.4.1 or current master (81b6b96475ac7a4ebfceae9f16fb3758327adbfe).

I submitted the first revert here, as the dead assignments to ‘iwl_trans->cfg’ are clearly a bug:
https://marc.info/?l=linux-wireless&m=157524294422431

As for the second revert, I await the maintainers’ thoughts on whether that’s the right solution or there’s something else I should try.
Comment 5 Luca Coelho 2019-12-02 05:41:27 UTC 
Thanks for reporting and bisecting! I think there may be some other fixes pending after this one that didn't get in, or, more likely, merge damage because of the way things moved around while fixes were being made.

Please tell me the output of "lscpi -knn -d ::280", so I can check the exact PCI device ID and subsystem device ID of your device.
Comment 6 Anders Kaseorg 2019-12-02 06:54:57 UTC 
Sure (this is a subset of the output I posted in comment 1).

# lspci -knn -d ::280
00:14.3 Network controller [0280]: Intel Corporation Device [8086:02f0]
	Subsystem: Intel Corporation Device [8086:0030]
	Kernel driver in use: iwlwifi
	Kernel modules: iwlwifi
Comment 7 Luca Coelho 2019-12-03 14:12:27 UTC 
I've been investigating this issue and unfortunately I don't have a solution for it yet. :(

Meanwhile, can you try to use the latest FW I published a few days ago?

https://git.kernel.org/pub/scm/linux/kernel/git/iwlwifi/linux-firmware.git/plain/iwlwifi-QuZ-a0-hr-b0-50.ucode

After installing it, please do a cold reboot (power off, power on) of the machine so everything is loaded again from scratch.

I'm not sure this will help, but then at least we get a bit more data on what is going on.
Comment 8 Anders Kaseorg 2019-12-03 22:04:17 UTC 
I got iwlwifi-QuZ-a0-jf-b0-50.ucode instead of iwlwifi-QuZ-a0-hr-b0-50.ucode since that’s what `dmesg` said the driver was trying to load.

With both 968dcfb4905245dc64d65312c0d17692fa087b99 and 9a47cb988338796b70c544919a8b6ba1f2245edb reverted, it works fine.

With only 968dcfb4905245dc64d65312c0d17692fa087b99 reverted, it fails again with what looks like the same problem:

[   13.187705] Intel(R) Wireless WiFi driver for Linux
[   13.187706] Copyright(c) 2003- 2015 Intel Corporation
[   13.187910] iwlwifi 0000:00:14.3: enabling device (0000 -> 0002)
[   13.192283] iwlwifi 0000:00:14.3: TLV_FW_FSEQ_VERSION: FSEQ Version: 58.3.35.22
[   13.192802] iwlwifi 0000:00:14.3: loaded firmware version 50.3e391d3e.0 op_mode iwlmvm
[   13.415841] iwlwifi 0000:00:14.3: Detected Intel(R) Wireless-AC 9560 160MHz, REV=0x354
[   14.469484] iwlwifi 0000:00:14.3: Collecting data: trigger 15 fired.
[   14.720214] iwlwifi 0000:00:14.3: Start IWL Error Log Dump:
[   14.720215] iwlwifi 0000:00:14.3: Status: 0x00000000, count: 6
[   14.720216] iwlwifi 0000:00:14.3: Loaded firmware version: 50.3e391d3e.0
[   14.720217] iwlwifi 0000:00:14.3: 0x00000084 | NMI_INTERRUPT_UNKNOWN       
[   14.720218] iwlwifi 0000:00:14.3: 0x000022F0 | trm_hw_status0
[   14.720218] iwlwifi 0000:00:14.3: 0x00000000 | trm_hw_status1
[   14.720219] iwlwifi 0000:00:14.3: 0x004C1EB0 | branchlink2
[   14.720219] iwlwifi 0000:00:14.3: 0x004B0CE4 | interruptlink1
[   14.720220] iwlwifi 0000:00:14.3: 0x004B0CE4 | interruptlink2
[   14.720220] iwlwifi 0000:00:14.3: 0x0001919A | data1
[   14.720221] iwlwifi 0000:00:14.3: 0xFF000000 | data2
[   14.720221] iwlwifi 0000:00:14.3: 0x00000000 | data3
[   14.720222] iwlwifi 0000:00:14.3: 0x00000000 | beacon time
[   14.720222] iwlwifi 0000:00:14.3: 0x000F7EAB | tsf low
[   14.720223] iwlwifi 0000:00:14.3: 0x00000000 | tsf hi
[   14.720223] iwlwifi 0000:00:14.3: 0x00000000 | time gp1
[   14.720224] iwlwifi 0000:00:14.3: 0x000FDC71 | time gp2
[   14.720224] iwlwifi 0000:00:14.3: 0x00000001 | uCode revision type
[   14.720225] iwlwifi 0000:00:14.3: 0x00000032 | uCode version major
[   14.720225] iwlwifi 0000:00:14.3: 0x3E391D3E | uCode version minor
[   14.720226] iwlwifi 0000:00:14.3: 0x00000351 | hw version
[   14.720226] iwlwifi 0000:00:14.3: 0x00089004 | board version
[   14.720227] iwlwifi 0000:00:14.3: 0x8003F502 | hcmd
[   14.720227] iwlwifi 0000:00:14.3: 0x00020000 | isr0
[   14.720228] iwlwifi 0000:00:14.3: 0x00000000 | isr1
[   14.720228] iwlwifi 0000:00:14.3: 0x00F00002 | isr2
[   14.720229] iwlwifi 0000:00:14.3: 0x00C0001C | isr3
[   14.720229] iwlwifi 0000:00:14.3: 0x00000000 | isr4
[   14.720230] iwlwifi 0000:00:14.3: 0x00000000 | last cmd Id
[   14.720230] iwlwifi 0000:00:14.3: 0x0001919A | wait_event
[   14.720231] iwlwifi 0000:00:14.3: 0x00000000 | l2p_control
[   14.720231] iwlwifi 0000:00:14.3: 0x00000000 | l2p_duration
[   14.720232] iwlwifi 0000:00:14.3: 0x00000000 | l2p_mhvalid
[   14.720232] iwlwifi 0000:00:14.3: 0x00000000 | l2p_addr_match
[   14.720233] iwlwifi 0000:00:14.3: 0x0000000B | lmpm_pmg_sel
[   14.720233] iwlwifi 0000:00:14.3: 0x00000000 | timestamp
[   14.720234] iwlwifi 0000:00:14.3: 0x00AC0000 | flow_handler
[   14.720268] iwlwifi 0000:00:14.3: Start IWL Error Log Dump:
[   14.720269] iwlwifi 0000:00:14.3: Status: 0x00000000, count: 7
[   14.720269] iwlwifi 0000:00:14.3: 0x20000066 | NMI_INTERRUPT_HOST
[   14.720270] iwlwifi 0000:00:14.3: 0x00000000 | umac branchlink1
[   14.720270] iwlwifi 0000:00:14.3: 0x80462CBA | umac branchlink2
[   14.720271] iwlwifi 0000:00:14.3: 0x80480094 | umac interruptlink1
[   14.720271] iwlwifi 0000:00:14.3: 0x80480094 | umac interruptlink2
[   14.720272] iwlwifi 0000:00:14.3: 0x01000000 | umac data1
[   14.720272] iwlwifi 0000:00:14.3: 0x80480094 | umac data2
[   14.720273] iwlwifi 0000:00:14.3: 0x00000000 | umac data3
[   14.720273] iwlwifi 0000:00:14.3: 0x00000032 | umac major
[   14.720274] iwlwifi 0000:00:14.3: 0x3E391D3E | umac minor
[   14.720274] iwlwifi 0000:00:14.3: 0x000FDC6C | frame pointer
[   14.720275] iwlwifi 0000:00:14.3: 0xC0886284 | stack pointer
[   14.720275] iwlwifi 0000:00:14.3: 0x0000FFFF | last host cmd
[   14.720276] iwlwifi 0000:00:14.3: 0x00000000 | isr status reg
[   14.720291] iwlwifi 0000:00:14.3: Fseq Registers:
[   14.720293] iwlwifi 0000:00:14.3: 0x20000000 | FSEQ_ERROR_CODE
[   14.720295] iwlwifi 0000:00:14.3: 0x80260000 | FSEQ_TOP_INIT_VERSION
[   14.720298] iwlwifi 0000:00:14.3: 0x00020006 | FSEQ_CNVIO_INIT_VERSION
[   14.720300] iwlwifi 0000:00:14.3: 0x0000A384 | FSEQ_OTP_VERSION
[   14.720302] iwlwifi 0000:00:14.3: 0xB0C1F95F | FSEQ_TOP_CONTENT_VERSION
[   14.720304] iwlwifi 0000:00:14.3: 0x4552414E | FSEQ_ALIVE_TOKEN
[   14.720306] iwlwifi 0000:00:14.3: 0x20000302 | FSEQ_CNVI_ID
[   14.720308] iwlwifi 0000:00:14.3: 0x01300202 | FSEQ_CNVR_ID
[   14.720310] iwlwifi 0000:00:14.3: 0x20000302 | CNVI_AUX_MISC_CHIP
[   14.720314] iwlwifi 0000:00:14.3: 0x01300202 | CNVR_AUX_MISC_CHIP
[   14.720318] iwlwifi 0000:00:14.3: 0x0000485B | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[   14.720352] iwlwifi 0000:00:14.3: 0xA5A5A5A2 | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[   14.720381] iwlwifi 0000:00:14.3: SecBoot CPU1 Status: 0x59ab, CPU2 Status: 0x3
[   14.720382] iwlwifi 0000:00:14.3: Failed to start RT ucode: -110
[   14.720391] iwlwifi 0000:00:14.3: Firmware not running - cannot dump error
[   14.732536] iwlwifi 0000:00:14.3: Failed to run INIT ucode: -110
Comment 9 Anders Kaseorg 2019-12-04 09:22:36 UTC 
(In reply to Anders Kaseorg from comment #8)
> With both 968dcfb4905245dc64d65312c0d17692fa087b99 and
> 9a47cb988338796b70c544919a8b6ba1f2245edb reverted, it works fine.

I may have spoken too soon.  It works for a while after boot, but eventually it hits this error and stops working until I turn Wi-Fi off and on again.  I’ve seen this a couple times on different cold boots.

[  257.962509] iwlwifi 0000:00:14.3: Microcode SW error detected. Restarting 0x0.
[  257.962578] iwlwifi 0000:00:14.3: Start IWL Error Log Dump:
[  257.962579] iwlwifi 0000:00:14.3: Status: 0x00000040, count: 6
[  257.962580] iwlwifi 0000:00:14.3: Loaded firmware version: 50.3e391d3e.0
[  257.962582] iwlwifi 0000:00:14.3: 0x00000EDC | ADVANCED_SYSASSERT          
[  257.962582] iwlwifi 0000:00:14.3: 0x00A0A200 | trm_hw_status0
[  257.962583] iwlwifi 0000:00:14.3: 0x00000000 | trm_hw_status1
[  257.962584] iwlwifi 0000:00:14.3: 0x004C1EB0 | branchlink2
[  257.962585] iwlwifi 0000:00:14.3: 0x004B0CE4 | interruptlink1
[  257.962586] iwlwifi 0000:00:14.3: 0x004B0CE4 | interruptlink2
[  257.962586] iwlwifi 0000:00:14.3: 0x00080000 | data1
[  257.962587] iwlwifi 0000:00:14.3: 0x30000045 | data2
[  257.962588] iwlwifi 0000:00:14.3: 0x00002554 | data3
[  257.962589] iwlwifi 0000:00:14.3: 0x3DC0458C | beacon time
[  257.962590] iwlwifi 0000:00:14.3: 0xCDAAFA4B | tsf low
[  257.962591] iwlwifi 0000:00:14.3: 0x0000014A | tsf hi
[  257.962592] iwlwifi 0000:00:14.3: 0x00000000 | time gp1
[  257.962592] iwlwifi 0000:00:14.3: 0x0E38DE6B | time gp2
[  257.962593] iwlwifi 0000:00:14.3: 0x00000001 | uCode revision type
[  257.962594] iwlwifi 0000:00:14.3: 0x00000032 | uCode version major
[  257.962595] iwlwifi 0000:00:14.3: 0x3E391D3E | uCode version minor
[  257.962596] iwlwifi 0000:00:14.3: 0x00000351 | hw version
[  257.962597] iwlwifi 0000:00:14.3: 0x00489004 | board version
[  257.962597] iwlwifi 0000:00:14.3: 0x30000045 | hcmd
[  257.962598] iwlwifi 0000:00:14.3: 0x20028000 | isr0
[  257.962599] iwlwifi 0000:00:14.3: 0x00400000 | isr1
[  257.962600] iwlwifi 0000:00:14.3: 0x08F0000A | isr2
[  257.962600] iwlwifi 0000:00:14.3: 0x00C37ECD | isr3
[  257.962601] iwlwifi 0000:00:14.3: 0x00000000 | isr4
[  257.962602] iwlwifi 0000:00:14.3: 0x05F9001C | last cmd Id
[  257.962603] iwlwifi 0000:00:14.3: 0x0001455A | wait_event
[  257.962604] iwlwifi 0000:00:14.3: 0x00004288 | l2p_control
[  257.962605] iwlwifi 0000:00:14.3: 0x00018034 | l2p_duration
[  257.962605] iwlwifi 0000:00:14.3: 0x00000000 | l2p_mhvalid
[  257.962606] iwlwifi 0000:00:14.3: 0x000000EF | l2p_addr_match
[  257.962607] iwlwifi 0000:00:14.3: 0x00000009 | lmpm_pmg_sel
[  257.962608] iwlwifi 0000:00:14.3: 0x00000000 | timestamp
[  257.962609] iwlwifi 0000:00:14.3: 0x00003830 | flow_handler
[  257.962641] iwlwifi 0000:00:14.3: Start IWL Error Log Dump:
[  257.962642] iwlwifi 0000:00:14.3: Status: 0x00000040, count: 7
[  257.962643] iwlwifi 0000:00:14.3: 0x20000070 | NMI_INTERRUPT_LMAC_FATAL
[  257.962644] iwlwifi 0000:00:14.3: 0x00000000 | umac branchlink1
[  257.962645] iwlwifi 0000:00:14.3: 0x80462CBA | umac branchlink2
[  257.962645] iwlwifi 0000:00:14.3: 0x80480094 | umac interruptlink1
[  257.962646] iwlwifi 0000:00:14.3: 0x80480094 | umac interruptlink2
[  257.962647] iwlwifi 0000:00:14.3: 0x00000400 | umac data1
[  257.962648] iwlwifi 0000:00:14.3: 0x80480094 | umac data2
[  257.962649] iwlwifi 0000:00:14.3: 0x00000000 | umac data3
[  257.962650] iwlwifi 0000:00:14.3: 0x00000032 | umac major
[  257.962650] iwlwifi 0000:00:14.3: 0x3E391D3E | umac minor
[  257.962651] iwlwifi 0000:00:14.3: 0x0E38DE84 | frame pointer
[  257.962652] iwlwifi 0000:00:14.3: 0xC0886284 | stack pointer
[  257.962653] iwlwifi 0000:00:14.3: 0x00D7010C | last host cmd
[  257.962654] iwlwifi 0000:00:14.3: 0x00000000 | isr status reg
[  257.962669] iwlwifi 0000:00:14.3: Fseq Registers:
[  257.962671] iwlwifi 0000:00:14.3: 0x60000000 | FSEQ_ERROR_CODE
[  257.962674] iwlwifi 0000:00:14.3: 0x80260000 | FSEQ_TOP_INIT_VERSION
[  257.962676] iwlwifi 0000:00:14.3: 0x00020006 | FSEQ_CNVIO_INIT_VERSION
[  257.962679] iwlwifi 0000:00:14.3: 0x0000A384 | FSEQ_OTP_VERSION
[  257.962681] iwlwifi 0000:00:14.3: 0xB0C5F94F | FSEQ_TOP_CONTENT_VERSION
[  257.962683] iwlwifi 0000:00:14.3: 0x4552414E | FSEQ_ALIVE_TOKEN
[  257.962686] iwlwifi 0000:00:14.3: 0x20000302 | FSEQ_CNVI_ID
[  257.962688] iwlwifi 0000:00:14.3: 0x01300202 | FSEQ_CNVR_ID
[  257.962690] iwlwifi 0000:00:14.3: 0x20000302 | CNVI_AUX_MISC_CHIP
[  257.962695] iwlwifi 0000:00:14.3: 0x01300202 | CNVR_AUX_MISC_CHIP
[  257.962700] iwlwifi 0000:00:14.3: 0x0000485B | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[  257.962734] iwlwifi 0000:00:14.3: 0xA5A5A5A2 | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[  257.962838] iwlwifi 0000:00:14.3: Collecting data: trigger 2 fired.
[  257.962841] ieee80211 phy0: Hardware restart was requested
Comment 10 Luca Coelho 2019-12-04 10:04:04 UTC 
Thanks for testing the new firmware.  I now know the reason for the crash and I'm working with our system engineers to figure out what to do.

This new crash you're getting is something different.  It is related to bug 203315.  Can you try the patch linked there and see if it solves the issue for you?
Comment 11 Anders Kaseorg 2019-12-04 22:37:27 UTC 
I already have the patch for bug 203315, cfb21b11b891b08b79be07be57c40a85bb926668 “iwlwifi: mvm: disable TX-AMSDU on older NICs”; it was merged in v5.3-rc4, and I’m on v5.4.  But I also see that it only makes a change for device_family < IWL_DEVICE_FAMILY_9000, and I have IWL_DEVICE_FAMILY_22000.
Comment 12 Anders Kaseorg 2019-12-04 22:43:09 UTC 
Just after posting that, I reproduced with firmware 48 by doing an upload speed test at fast.com.  (So I guess it’s not a firmware 50 regression at least.)

[ 2186.673865] iwlwifi 0000:00:14.3: Microcode SW error detected. Restarting 0x0.
[ 2186.673934] iwlwifi 0000:00:14.3: Start IWL Error Log Dump:
[ 2186.673935] iwlwifi 0000:00:14.3: Status: 0x00000040, count: 6
[ 2186.673936] iwlwifi 0000:00:14.3: Loaded firmware version: 48.4fa0041f.0
[ 2186.673937] iwlwifi 0000:00:14.3: 0x00001043 | ADVANCED_SYSASSERT          
[ 2186.673937] iwlwifi 0000:00:14.3: 0x05B022F3 | trm_hw_status0
[ 2186.673938] iwlwifi 0000:00:14.3: 0x00000000 | trm_hw_status1
[ 2186.673938] iwlwifi 0000:00:14.3: 0x004C2214 | branchlink2
[ 2186.673939] iwlwifi 0000:00:14.3: 0x00000E36 | interruptlink1
[ 2186.673939] iwlwifi 0000:00:14.3: 0x00000E36 | interruptlink2
[ 2186.673940] iwlwifi 0000:00:14.3: 0x00002A14 | data1
[ 2186.673940] iwlwifi 0000:00:14.3: 0x00003630 | data2
[ 2186.673941] iwlwifi 0000:00:14.3: 0x000005F4 | data3
[ 2186.673941] iwlwifi 0000:00:14.3: 0xBEC18F66 | beacon time
[ 2186.673942] iwlwifi 0000:00:14.3: 0xFC10B072 | tsf low
[ 2186.673942] iwlwifi 0000:00:14.3: 0x00000155 | tsf hi
[ 2186.673943] iwlwifi 0000:00:14.3: 0x00000000 | time gp1
[ 2186.673943] iwlwifi 0000:00:14.3: 0x561B79DD | time gp2
[ 2186.673944] iwlwifi 0000:00:14.3: 0x00000001 | uCode revision type
[ 2186.673944] iwlwifi 0000:00:14.3: 0x00000030 | uCode version major
[ 2186.673945] iwlwifi 0000:00:14.3: 0x4FA0041F | uCode version minor
[ 2186.673945] iwlwifi 0000:00:14.3: 0x00000351 | hw version
[ 2186.673946] iwlwifi 0000:00:14.3: 0x00489004 | board version
[ 2186.673946] iwlwifi 0000:00:14.3: 0x05AD001C | hcmd
[ 2186.673947] iwlwifi 0000:00:14.3: 0xE6FA3800 | isr0
[ 2186.673947] iwlwifi 0000:00:14.3: 0x01440000 | isr1
[ 2186.673947] iwlwifi 0000:00:14.3: 0x08F80112 | isr2
[ 2186.673948] iwlwifi 0000:00:14.3: 0x0CC5FFDF | isr3
[ 2186.673948] iwlwifi 0000:00:14.3: 0x00000000 | isr4
[ 2186.673949] iwlwifi 0000:00:14.3: 0x0576001C | last cmd Id
[ 2186.673949] iwlwifi 0000:00:14.3: 0x00015134 | wait_event
[ 2186.673950] iwlwifi 0000:00:14.3: 0x000000C4 | l2p_control
[ 2186.673950] iwlwifi 0000:00:14.3: 0x00001C20 | l2p_duration
[ 2186.673951] iwlwifi 0000:00:14.3: 0x00000007 | l2p_mhvalid
[ 2186.673951] iwlwifi 0000:00:14.3: 0x00000081 | l2p_addr_match
[ 2186.673952] iwlwifi 0000:00:14.3: 0x00000009 | lmpm_pmg_sel
[ 2186.673952] iwlwifi 0000:00:14.3: 0x00000000 | timestamp
[ 2186.673953] iwlwifi 0000:00:14.3: 0x00000800 | flow_handler
[ 2186.673985] iwlwifi 0000:00:14.3: Start IWL Error Log Dump:
[ 2186.673985] iwlwifi 0000:00:14.3: Status: 0x00000040, count: 7
[ 2186.673986] iwlwifi 0000:00:14.3: 0x20000070 | NMI_INTERRUPT_LMAC_FATAL
[ 2186.673986] iwlwifi 0000:00:14.3: 0x00000000 | umac branchlink1
[ 2186.673987] iwlwifi 0000:00:14.3: 0xC008CF5C | umac branchlink2
[ 2186.673987] iwlwifi 0000:00:14.3: 0x8048A78A | umac interruptlink1
[ 2186.673988] iwlwifi 0000:00:14.3: 0x8048A78A | umac interruptlink2
[ 2186.673988] iwlwifi 0000:00:14.3: 0x00000400 | umac data1
[ 2186.673989] iwlwifi 0000:00:14.3: 0x8048A78A | umac data2
[ 2186.673989] iwlwifi 0000:00:14.3: 0x00000000 | umac data3
[ 2186.673990] iwlwifi 0000:00:14.3: 0x00000030 | umac major
[ 2186.673990] iwlwifi 0000:00:14.3: 0x4FA0041F | umac minor
[ 2186.673991] iwlwifi 0000:00:14.3: 0x561B79F6 | frame pointer
[ 2186.673991] iwlwifi 0000:00:14.3: 0xC0886284 | stack pointer
[ 2186.673992] iwlwifi 0000:00:14.3: 0x00C5010C | last host cmd
[ 2186.673992] iwlwifi 0000:00:14.3: 0x00000000 | isr status reg
[ 2186.674007] iwlwifi 0000:00:14.3: Fseq Registers:
[ 2186.674009] iwlwifi 0000:00:14.3: 0xE0000021 | FSEQ_ERROR_CODE
[ 2186.674012] iwlwifi 0000:00:14.3: 0x00260000 | FSEQ_TOP_INIT_VERSION
[ 2186.674014] iwlwifi 0000:00:14.3: 0x80020006 | FSEQ_CNVIO_INIT_VERSION
[ 2186.674016] iwlwifi 0000:00:14.3: 0x0000A384 | FSEQ_OTP_VERSION
[ 2186.674018] iwlwifi 0000:00:14.3: 0xB0C1D9DF | FSEQ_TOP_CONTENT_VERSION
[ 2186.674020] iwlwifi 0000:00:14.3: 0x4552414E | FSEQ_ALIVE_TOKEN
[ 2186.674022] iwlwifi 0000:00:14.3: 0x20000302 | FSEQ_CNVI_ID
[ 2186.674024] iwlwifi 0000:00:14.3: 0x01300202 | FSEQ_CNVR_ID
[ 2186.674026] iwlwifi 0000:00:14.3: 0x20000302 | CNVI_AUX_MISC_CHIP
[ 2186.674030] iwlwifi 0000:00:14.3: 0x01300202 | CNVR_AUX_MISC_CHIP
[ 2186.674034] iwlwifi 0000:00:14.3: 0x0000485B | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[ 2186.674068] iwlwifi 0000:00:14.3: 0xA5A5A5A2 | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[ 2186.674163] iwlwifi 0000:00:14.3: Collecting data: trigger 2 fired.
[ 2186.674166] ieee80211 phy0: Hardware restart was requested

I’ll try editing the code to disable TX-AMSDU on 22000 as well.  Maybe this should be a module parameter?
Comment 13 Luca Coelho 2019-12-04 22:51:05 UTC 
Created attachment 286183 [details]
Patch to fix the power gating issue.
Comment 14 Luca Coelho 2019-12-04 22:52:06 UTC 
That's another bug too.  Whenever the value beside the ADVANCED_SYSASSERT changes, it's likely to be a different issue.  I think this specific one 0x00001043 is also known, I'll check it later.

For the original issue in this bug (the power gating issue), I have a patch (attached).  Can you try it and let me know if it solves the crash during initialization for you?
Comment 15 Anders Kaseorg 2019-12-04 23:14:16 UTC 
Yes, this new patch (along with the first revert of 968dcfb4905245dc64d65312c0d17692fa087b99) has resolved the initialization crash.  Thanks!

I ran into a third ADVANCED_SYSASSERT error under load (0x0000103C), but that code led me to bug 205061, so I’ll try the patch there.
Comment 16 Luca Coelho 2019-12-05 06:17:45 UTC 
Thanks Anders!

I'll mark this bug as RESOLVED now, because the original problem is solved.  And then I'll mark it as CLOSED when the patch is on its way upstream.

You were right about the 0xEDC bug I mentioned.  The fix doesn't make sense for 9000 devices and above.

About the other SYSASSERTs, they probably caused by the bug I fixed with the patch on bug 205061.  If you see them again, please check if there is already a bug report with the same symptoms.  If there is, please post additional info you have in the existing bug.  If not, feel free to create a new bug and assign to linuxwifi@intel.com.

Thanks for the cooperation!
Comment 17 Luca Coelho 2019-12-05 07:08:17 UTC 
The patch is now on its way upstream: https://patchwork.kernel.org/patch/11274279/
Comment 18 leander 2020-01-21 18:27:17 UTC 
Was told to note this here on the Fedora-users list. I have a laptop with a AC Killer Wi-Fi 6 AX1650i that is failing to load firmware on 5.4 kernels on Fedora 31. Specifically I've tried it with 5.4.8-200.fc31.x86_64, 5.4.10-200.fc31.x86_64 and 5.4.12-200.fc31.x86_64 on F31. The card works fine on 5.3 series kernels. Output from dmesg below, thanks!


[   25.349791] iwlwifi 0000:00:14.3: enabling device (0000 -> 0002)
[   25.353683] iwlwifi 0000:00:14.3: Direct firmware load for iwlwifi-Qu-b0-hr-b0-50.ucode failed with error -2
[   25.353729] iwlwifi 0000:00:14.3: Direct firmware load for iwlwifi-Qu-b0-hr-b0-49.ucode failed with error -2
[   25.356322] iwlwifi 0000:00:14.3: TLV_FW_FSEQ_VERSION: FSEQ Version: 43.2.23.17
[   25.356327] iwlwifi 0000:00:14.3: Found debug destination: EXTERNAL_DRAM
[   25.356330] iwlwifi 0000:00:14.3: Found debug configuration: 0
[   25.356614] iwlwifi 0000:00:14.3: loaded firmware version 48.4fa0041f.0 op_mode iwlmvm
[   25.604880] iwlwifi 0000:00:14.3: Detected Killer(R) Wi-Fi 6 AX1650i 160MHz Wireless Network Adapter (201NGW), REV=0x338
[   25.611560] iwlwifi 0000:00:14.3: Applying debug destination EXTERNAL_DRAM
[   25.611842] iwlwifi 0000:00:14.3: Allocated 0x00400000 bytes for firmware monitor.
[   26.616821] iwlwifi 0000:00:14.3: Collecting data: trigger 15 fired.
[   26.617256] iwlwifi 0000:00:14.3: Start IWL Error Log Dump:
[   26.617258] iwlwifi 0000:00:14.3: Status: 0x00000000, count: 2127592660
[   26.617260] iwlwifi 0000:00:14.3: Loaded firmware version: 48.4fa0041f.0
[   26.617262] iwlwifi 0000:00:14.3: 0x6AD48FE2 | ADVANCED_SYSASSERT          
[   26.617263] iwlwifi 0000:00:14.3: 0xDF7C392E | trm_hw_status0
[   26.617265] iwlwifi 0000:00:14.3: 0x5893FF5E | trm_hw_status1
[   26.617266] iwlwifi 0000:00:14.3: 0xD42165AC | branchlink2
[   26.617267] iwlwifi 0000:00:14.3: 0x1AF12612 | interruptlink1
[   26.617268] iwlwifi 0000:00:14.3: 0xEC9D03FF | interruptlink2
[   26.617269] iwlwifi 0000:00:14.3: 0x4FF536FD | data1
[   26.617271] iwlwifi 0000:00:14.3: 0x08155044 | data2
[   26.617272] iwlwifi 0000:00:14.3: 0x0EB30A10 | data3
[   26.617273] iwlwifi 0000:00:14.3: 0xBCA8C2E9 | beacon time
[   26.617274] iwlwifi 0000:00:14.3: 0xA5CC5B59 | tsf low
[   26.617275] iwlwifi 0000:00:14.3: 0x4C694008 | tsf hi
[   26.617277] iwlwifi 0000:00:14.3: 0x8A24C99E | time gp1
[   26.617278] iwlwifi 0000:00:14.3: 0xD7DEE6A7 | time gp2
[   26.617279] iwlwifi 0000:00:14.3: 0xDB596FFD | uCode revision type
[   26.617280] iwlwifi 0000:00:14.3: 0x4616438F | uCode version major
[   26.617281] iwlwifi 0000:00:14.3: 0x3EA72F70 | uCode version minor
[   26.617282] iwlwifi 0000:00:14.3: 0xB19E6974 | hw version
[   26.617284] iwlwifi 0000:00:14.3: 0xF23A2BBA | board version
[   26.617285] iwlwifi 0000:00:14.3: 0xBC6D677E | hcmd
[   26.617286] iwlwifi 0000:00:14.3: 0xF0D072B7 | isr0
[   26.617287] iwlwifi 0000:00:14.3: 0xD0161135 | isr1
[   26.617290] iwlwifi 0000:00:14.3: 0x79236BEC | isr2
[   26.617291] iwlwifi 0000:00:14.3: 0x4EFCEC7C | isr3
[   26.617294] iwlwifi 0000:00:14.3: 0x42133502 | isr4
[   26.617295] iwlwifi 0000:00:14.3: 0x36594403 | last cmd Id
[   26.617297] iwlwifi 0000:00:14.3: 0xBE9DB9FD | wait_event
[   26.617298] iwlwifi 0000:00:14.3: 0x2DD37BE9 | l2p_control
[   26.617299] iwlwifi 0000:00:14.3: 0x1FAA7872 | l2p_duration
[   26.617300] iwlwifi 0000:00:14.3: 0x98C05A92 | l2p_mhvalid
[   26.617301] iwlwifi 0000:00:14.3: 0xB6DA7EF2 | l2p_addr_match
[   26.617302] iwlwifi 0000:00:14.3: 0xA9DE6EBC | lmpm_pmg_sel
[   26.617303] iwlwifi 0000:00:14.3: 0x51C070AC | timestamp
[   26.617304] iwlwifi 0000:00:14.3: 0x9CA17099 | flow_handler
[   26.617347] iwlwifi 0000:00:14.3: Start IWL Error Log Dump:
[   26.617348] iwlwifi 0000:00:14.3: Status: 0x00000000, count: 7
[   26.617350] iwlwifi 0000:00:14.3: 0x201013F1 | ADVANCED_SYSASSERT
[   26.617351] iwlwifi 0000:00:14.3: 0x00000000 | umac branchlink1
[   26.617352] iwlwifi 0000:00:14.3: 0xC008D49C | umac branchlink2
[   26.617353] iwlwifi 0000:00:14.3: 0x00000000 | umac interruptlink1
[   26.617354] iwlwifi 0000:00:14.3: 0x00000000 | umac interruptlink2
[   26.617356] iwlwifi 0000:00:14.3: 0x00000003 | umac data1
[   26.617357] iwlwifi 0000:00:14.3: 0x02000300 | umac data2
[   26.617358] iwlwifi 0000:00:14.3: 0x01300504 | umac data3
[   26.617359] iwlwifi 0000:00:14.3: 0x00000030 | umac major
[   26.617361] iwlwifi 0000:00:14.3: 0x4FA0041F | umac minor
[   26.617362] iwlwifi 0000:00:14.3: 0x0000607B | frame pointer
[   26.617363] iwlwifi 0000:00:14.3: 0xC0887F58 | stack pointer
[   26.617364] iwlwifi 0000:00:14.3: 0x00000000 | last host cmd
[   26.617366] iwlwifi 0000:00:14.3: 0x00000000 | isr status reg
[   26.617382] iwlwifi 0000:00:14.3: Fseq Registers:
[   26.617385] iwlwifi 0000:00:14.3: 0x00000003 | FSEQ_ERROR_CODE
[   26.617388] iwlwifi 0000:00:14.3: 0x00290033 | FSEQ_TOP_INIT_VERSION
[   26.617391] iwlwifi 0000:00:14.3: 0x80070043 | FSEQ_CNVIO_INIT_VERSION
[   26.617412] iwlwifi 0000:00:14.3: 0x0000A481 | FSEQ_OTP_VERSION
[   26.617415] iwlwifi 0000:00:14.3: 0x00000002 | FSEQ_TOP_CONTENT_VERSION
[   26.617419] iwlwifi 0000:00:14.3: 0x4552414E | FSEQ_ALIVE_TOKEN
[   26.617440] iwlwifi 0000:00:14.3: 0x02000300 | FSEQ_CNVI_ID
[   26.617443] iwlwifi 0000:00:14.3: 0x01300504 | FSEQ_CNVR_ID
[   26.617446] iwlwifi 0000:00:14.3: 0x02000300 | CNVI_AUX_MISC_CHIP
[   26.617451] iwlwifi 0000:00:14.3: 0x01300504 | CNVR_AUX_MISC_CHIP
[   26.617457] iwlwifi 0000:00:14.3: 0x05B0905B | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[   26.617481] iwlwifi 0000:00:14.3: 0x0000025B | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[   26.617545] iwlwifi 0000:00:14.3: SecBoot CPU1 Status: 0x5fe6, CPU2 Status: 0x3
[   26.617547] iwlwifi 0000:00:14.3: Failed to start RT ucode: -110
[   26.617552] iwlwifi 0000:00:14.3: Firmware not running - cannot dump error
[   26.628883] iwlwifi 0000:00:14.3: Failed to run INIT ucode: -110