Bug 207555

Summary: ixgbe: firmware spam on X520-T2 NIC
Product: Drivers Reporter: Nick Price (nick)
Component: NetworkAssignee: drivers_network (drivers_network)
Status: NEW ---    
Severity: normal CC: matorola, nick
Priority: P1 Keywords: trivial
Hardware: All   
OS: Linux   
Kernel Version: 5.6.8 Subsystem:
Regression: No Bisected commit-id:
Attachments: Check FWSM bit before determining we have should fire off an error message

Description Nick Price 2020-05-03 12:43:51 UTC
There is constant spam caused by the the check added in the commit at https://github.com/torvalds/linux/commit/59dd45d550c518a2c297b2888f194633cb8e5700

Intel X520 (and I believe X540) series NICs have no firmware updates available but trigger this message two times (likely one message per port) every two seconds
 
[  639.428033] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  639.652057] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  641.444090] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  641.668051] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  643.460080] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  643.684112] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  645.476108] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  645.700115] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  647.492131] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  647.716118] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  649.508152] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  649.732134] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  651.524153] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  651.748194] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  653.540216] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  653.764184] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  655.556234] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  655.780245] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  657.576286] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  657.796277] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  659.588294] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  659.812302] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  661.604312] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  661.828304] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  663.620370] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  663.844336] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  665.636377] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  665.860365] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  667.652358] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  667.876387] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  669.668444] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  669.892451] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
[  671.684469] ixgbe 0000:41:00.0: Warning firmware error detected FWSM: 0x00000000
[  671.908447] ixgbe 0000:41:00.1: Warning firmware error detected FWSM: 0x00000000
Comment 1 Nick Price 2020-05-03 13:35:15 UTC
https://patchwork.criu.org/patch/11882/raw/
Comment 2 Nick Price 2020-05-03 19:07:19 UTC
Posted in my comment above is a patch that simply disables the check and the error.  I've also attached a patch that checks for the FWSM bit to determine whether there's actually an error, which I believe is more correct.

Not sure if there's anything we can do with `hw->mac.type` around line 6317 in ixgbe_main.c that would be even *more* correct.
Comment 3 Nick Price 2020-05-03 19:08:09 UTC
Created attachment 288877 [details]
Check FWSM bit before determining we have should fire off an error message

I think this is *more* correct than simply disabling the check/warning altogether as others have done.
Comment 4 Nick Price 2020-05-18 14:36:39 UTC
After some more digging, my patch above is probably incorrect - this is an unpatched firmware issue with a series of cards where they don't set bit 15 of the fwsm register to 1 after they initialize.
Comment 5 Anatoly Pugachev 2020-07-22 08:32:18 UTC
don't have this firmware messages with fedora 31 and X540-AT2 :

$ uname -r
5.7.8-100.fc31.x86_64

$ ethtool -i eno1
driver: ixgbe
version: 5.1.0-k
firmware-version: 0x8000035a
...

$ lspci | grep X5
20:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
...