Bug 218589

Summary:	Intel ethernet card (8086:550a) make the system not enter S0ix when cable is plugged (driver: e1000e)
Product:	Drivers	Reporter:	Hui Wang (hui.wang)
Component:	Network	Assignee:	drivers_network (drivers_network)
Status:	NEW ---
Severity:	blocking	CC:	amir.avivi, anthony.l.nguyen, david.e.box, dima.ruinskiy, hui.wang, jesse.brandeburg, libin.charles, sasha.neftin, vitaly.lifshits
Priority:	P3
Hardware:	Intel
OS:	Linux
Kernel Version:	6.8.0-rc7	Subsystem:
Regression:	No	Bisected commit-id:
Attachments:	AMT status on the lenovo laptop.

Description Hui Wang 2024-03-12 02:01:22 UTC

kernel version: ubuntu-6.5.0 and most recent mainline kernel (linux-v6.8.0-rc7)


We have a couple of laptops (HP, Lenovo and Dell), they all have the Intel ethernet card as below. when the ethernet cable is plugged and we run system suspend, the system could enter suspend successfully, after we resume the system and check the /sys/kernel/debug/pmc_core/slp_s0_residency_usec, it returns 0, that means the system doesn't enter S0ix during the suspend. This makes our QA's testcase fail. And we also check the /sys/kernel/debug/pmc_core/package_cstate_show, it shows the system does could enter PC10. 


If we unplug the ethernet cable, the issue disappears, the system could enter S0ix successfully.


So in sumary:
With the cable plugged: system suspend could enter PC10, but couldn't enter S0ix.
With the cable unplugged: system suspend could enter PC10 and S0ix.


ubuntu@Oak2-1:~$ lspci -nn -s 00:1f.6
00:1f.6 Ethernet controller [0200]: Intel Corporation Device [8086:550a] (rev 20)

Comment 1 Hui Wang 2024-03-14 06:25:17 UTC

And 'ifconfig $NETDEV down' doesn't help for this issue. And 'ethtool -s $NETDEV wol d' deosn't help either.

Comment 2 Sasha Neftin 2024-03-17 17:06:24 UTC

What is the CSME (ME manageability) version of your system running?
AMT provisioned or AMT none provisioned? Please, check with laptop vendors.

Comment 3 Hui Wang 2024-03-19 09:28:54 UTC

Sorry for late response. This is the version of CSME on the lenovo laptop:

Intel(R) CSME Version Detection Tool
Copyright(C) 2017-2023, Intel Corporation, All rights reserved.

Application Version: 9.0.3.0
Scan date: 2024-03-19 09:25:52 GMT

*** Host Computer Information ***
Name: Oak2-1
Manufacturer: LENOVO
Model: 21KXSIT003
Processor Name: Intel(R) Core(TM) Ultra 5 135H
OS Version: Ubuntu 22.04.4 LTS (6.5.0-1014-oem)

*** Intel(R) ME Information ***
Engine: Intel(R) Converged Security and Management Engine
Version: 18.0.5.2066

*** Risk Assessment ***
Based on the analysis performed by this tool: This system is not vulnerable.

For more information refer to the Intel(R) CSME Version Detection Tool User Guide
or the related Intel Security Advisory list at:
https://www.intel.com/content/www/us/en/support/articles/000031784/technologies.html

Comment 4 Hui Wang 2024-03-19 09:37:44 UTC

Created attachment 306007 [details]
AMT status on the lenovo laptop.

AMT status on the lenovo laptop.

Comment 5 Hui Wang 2024-03-19 09:40:09 UTC

About the AMT status on the lenovo laptop, please see the attached picture.

And if provided info is not correct or not sufficient, please tell me, then I will ask for requested info from laptop vendors.

thanks.

Comment 6 Bin Li 2024-03-19 11:29:52 UTC

I219-LM 8086:550b also has the same issue.

00:1f.6 Ethernet controller [0200]: Intel Corporation Device [8086:550b] (rev 20)    
    Subsystem: Lenovo Device [17aa:50e1]  
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+    
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-    
    Latency: 0                                                                      
    Interrupt: pin D routed to IRQ 141                                              
    IOMMU group: 14                                                                 
    Region 0: Memory at ac400000 (32-bit, non-prefetchable) [size=128K]             
    Capabilities: [c8] Power Management version 3                                   
        Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)    
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-                      
    Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+                      
        Address: 00000000fee004b8  Data: 0000                                       
    Kernel driver in use: e1000e                                                    
    Kernel modules: e1000e

Comment 7 Bin Li 2024-03-19 11:31:29 UTC

(In reply to Bin Li from comment #6)
> I219-LM 8086:550b also has the same issue.
> 
> 00:1f.6 Ethernet controller [0200]: Intel Corporation Device [8086:550b]
> (rev 20)    
>     Subsystem: Lenovo Device [17aa:50e1]  

Sorry, it should be I219-V 8086:550b

Comment 8 Bin Li 2024-03-19 11:31:41 UTC

(In reply to Bin Li from comment #6)
> I219-LM 8086:550b also has the same issue.
> 
> 00:1f.6 Ethernet controller [0200]: Intel Corporation Device [8086:550b]
> (rev 20)    
>     Subsystem: Lenovo Device [17aa:50e1]  

Sorry, it should be I219-V 8086:550b

Comment 9 Hui Wang 2024-03-20 02:59:14 UTC

@Sasha, This is the info of a HP laptop.

$ lspci

00:1f.6 Ethernet controller: Intel Corporation Device 550a (rev 20)
 DeviceName: Onboard Ethernet
 Subsystem: Hewlett-Packard Company Device 8ca0
 Kernel driver in use: e1000e
 Kernel modules: e1000e

$ cat /sys/class/dmi/id/bios_version
W97 Ver. 92.02.00

$ ./intel_csme_version_detection_tool
Intel(R) CSME Version Detection Tool
Copyright(C) 2017-2023, Intel Corporation, All rights reserved.

Application Version: 9.0.3.0
Scan date: 2024-03-06 05:39:12 GMT

*** Host Computer Information ***
Name: 33400
Manufacturer: HP
Model: HP ZBook Power 16 inch G11 Mobile Workstation PC
Processor Name: Intel(R) Core(TM) Ultra 9 185H
OS Version: Ubuntu 22.04.3 LTS (6.5.0-2016-oem)

*** Intel(R) ME Information ***
Engine: Intel(R) Converged Security and Management Engine
Version: 18.0.5.2098

*** Risk Assessment ***
Based on the analysis performed by this tool: This system is not vulnerable.

For more information refer to the Intel(R) CSME Version Detection Tool User Guide
or the related Intel Security Advisory list at:
https://www.intel.com/content/www/us/en/support/articles/000031784/technologies.html

Comment 10 Sasha Neftin 2024-03-20 04:17:37 UTC

(In reply to Hui Wang from comment #9)
> @Sasha, This is the info of a HP laptop.
> 
> $ lspci
> 
> 00:1f.6 Ethernet controller: Intel Corporation Device 550a (rev 20)
>  DeviceName: Onboard Ethernet
>  Subsystem: Hewlett-Packard Company Device 8ca0
>  Kernel driver in use: e1000e
>  Kernel modules: e1000e
> 
> $ cat /sys/class/dmi/id/bios_version
> W97 Ver. 92.02.00
> 
> $ ./intel_csme_version_detection_tool
> Intel(R) CSME Version Detection Tool
> Copyright(C) 2017-2023, Intel Corporation, All rights reserved.
> 
> Application Version: 9.0.3.0
> Scan date: 2024-03-06 05:39:12 GMT
> 
> *** Host Computer Information ***
> Name: 33400
> Manufacturer: HP
> Model: HP ZBook Power 16 inch G11 Mobile Workstation PC
> Processor Name: Intel(R) Core(TM) Ultra 9 185H
> OS Version: Ubuntu 22.04.3 LTS (6.5.0-2016-oem)
> 
> *** Intel(R) ME Information ***
> Engine: Intel(R) Converged Security and Management Engine
> Version: 18.0.5.2098
> 
> *** Risk Assessment ***
> Based on the analysis performed by this tool: This system is not vulnerable.
> 
> For more information refer to the Intel(R) CSME Version Detection Tool User
> Guide
> or the related Intel Security Advisory list at:
> https://www.intel.com/content/www/us/en/support/articles/000031784/
> technologies.html

Thank you Hui. Question is CSME AMT provisioned or AMT none provisioned? Can you check with laptop vendors(Lenovo and HP)? 
I will try to find out about this internally too.

The reason for my question: is that CSME AMT provisioned systems do not support s0ix states in Linux. (only CSME not provisioned).

I would also suggest (as the next step) trying to disable the CSME from BIOS on your systems. I am pretty sure CSME runs some traffic when you connect the cable and therefore can not allow s0ix residency.

Comment 11 Hui Wang 2024-03-20 04:46:09 UTC

OK, got it. I will ask vendor about it. And try to disable the CSME and test on the machines.

thx.

Comment 12 Sasha Neftin 2024-03-20 08:04:12 UTC

(In reply to Hui Wang from comment #11)
> OK, got it. I will ask vendor about it. And try to disable the CSME and test
> on the machines.
> 
> thx.

alternatively, you can put your system in S state (rtcwake -v -m freeze -s 120) and check ping/traffic on the ethernet card.

Comment 13 Hui Wang 2024-03-20 12:17:26 UTC

Before running (rtcwake -v -m freeze -s 120):
$ping 192.168.6.106
64 bytes from 192.168.6.106: icmp_seq=118 ttl=64 time=1026 ms
64 bytes from 192.168.6.106: icmp_seq=119 ttl=64 time=1.74 ms
...

After running (rtcwake -v -m freeze -s 120):
$ping 192.168.6.106
From 192.168.6.1 icmp_seq=1 Destination Host Unreachable
From 192.168.6.1 icmp_seq=2 Destination Host Unreachable


And I got the reply from our PM, it is not provisioned.

Comment 14 Sasha Neftin 2024-03-21 09:47:03 UTC

(In reply to Hui Wang from comment #13)
> Before running (rtcwake -v -m freeze -s 120):
> $ping 192.168.6.106
> 64 bytes from 192.168.6.106: icmp_seq=118 ttl=64 time=1026 ms
> 64 bytes from 192.168.6.106: icmp_seq=119 ttl=64 time=1.74 ms
> ...
> 
> After running (rtcwake -v -m freeze -s 120):
> $ping 192.168.6.106
> From 192.168.6.1 icmp_seq=1 Destination Host Unreachable
> From 192.168.6.1 icmp_seq=2 Destination Host Unreachable
> 
> 
> And I got the reply from our PM, it is not provisioned.

what is last  Intel ethernet card (8086:????) works for you?
(we will need to check LTR ignore w/a with cable connected for new devices so)

Comment 15 Hui Wang 2024-03-21 13:15:14 UTC

At the moment, what I know is 8086:550b is better than 8086:550a, 550b could enter S0ix with cable plugged, but it stays in S0ix state shorter than expected (sth like wakeup earlier than expected from S0ix). 550a Couldn't enter S0ix at all.

For other net cards, I need to go to lab to check. I will update more info next week.

Comment 16 Sasha Neftin 2024-03-23 17:09:58 UTC

(In reply to Hui Wang from comment #15)
> At the moment, what I know is 8086:550b is better than 8086:550a, 550b could
> enter S0ix with cable plugged, but it stays in S0ix state shorter than
> expected (sth like wakeup earlier than expected from S0ix). 550a Couldn't
> enter S0ix at all.
> 
> For other net cards, I need to go to lab to check. I will update more info
> next week.

550b is -V SKU (no CSME). 550a is -LM SKU (with CSME). Could you confirm that s0ix residencies problem with -LM SKU? 0x550a?

Comment 17 Hui Wang 2024-03-24 10:13:39 UTC

I confirm that the system couldn't enter s0ix with the 550a SKU (cable plugged), the log below is collected on a Lenovo laptop with 550a SKU:

root@Oak2-1:/sys/kernel/debug/pmc_core# lspci -nn -s 00:1f.6
00:1f.6 Ethernet controller [0200]: Intel Corporation Device [8086:550a] (rev 20)
root@Oak2-1:/sys/kernel/debug/pmc_core# ifconfig enp0s31f6
enp0s31f6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 192.168.6.106 netmask 255.255.255.0 broadcast 192.168.6.255
        inet6 fe80::4312:f92:9674:e335 prefixlen 64 scopeid 0x20<link>
        ether fc:5c:ee:48:1e:bf txqueuelen 1000 (Ethernet)
        RX packets 18 bytes 1902 (1.9 KB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 139 bytes 13207 (13.2 KB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
        device interrupt 19 memory 0xab800000-ab820000

root@Oak2-1:/sys/kernel/debug/pmc_core# cat slp_s0_residency_usec
0
root@Oak2-1:/sys/kernel/debug/pmc_core# cat package_cstate_show
Package C2 : 325442386
Package C3 : 0
Package C6 : 0
Package C7 : 0
Package C8 : 0
Package C9 : 0
Package C10 : 0

root@Oak2-1:/sys/kernel/debug/pmc_core# rtcwake -s 20 -m mem
rtcwake: assuming RTC uses UTC ...
rtcwake: wakeup from "mem" using /dev/rtc0 at Fri Mar 22 00:16:06 2024

root@Oak2-1:/sys/kernel/debug/pmc_core#
root@Oak2-1:/sys/kernel/debug/pmc_core# cat slp_s0_residency_usec
0
root@Oak2-1:/sys/kernel/debug/pmc_core# cat package_cstate_show
Package C2 : 346374190
Package C3 : 28681
Package C6 : 1465318
Package C7 : 0
Package C8 : 32109
Package C9 : 0
Package C10 : 17417558
root@Oak2-1:/sys/kernel/debug/pmc_core#

Comment 18 Hui Wang 2024-03-26 06:13:26 UTC

The ethernet card 8086:0dc6 doesn't have this issue, the system could enter s0ix with cable plugged:

root@Jing-Intel-2:/sys/kernel/debug/pmc_core# lspci -nn -s 00:1f.6
00:1f.6 Ethernet controller [0200]: Intel Corporation Device [8086:0dc6] (rev 01)
root@Jing-Intel-2:/sys/kernel/debug/pmc_core# cat package_cstate_show 
Package C2 : 817747217
Package C3 : 4447659574
Package C6 : 0
Package C7 : 0
Package C8 : 0
Package C9 : 0
Package C10 : 0
root@Jing-Intel-2:/sys/kernel/debug/pmc_core# cat slp_s0_residency_usec 
0
root@Jing-Intel-2:/sys/kernel/debug/pmc_core# rtcwake  -s 20 -m mem 
rtcwake: assuming RTC uses UTC ...
rtcwake: wakeup from "mem" using /dev/rtc0 at Tue Mar 26 05:48:28 2024
root@Jing-Intel-2:/sys/kernel/debug/pmc_core# cat slp_s0_residency_usec 
19682504

Comment 19 Hui Wang 2024-03-26 06:19:06 UTC

The ethernet card 8086:0dc5 doesn't have this issue, the system could enter s0ix with cable plugged:

ubuntu@Roc2-P-4:~$ ifconfig
enp0s31f6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.1.1.114  netmask 255.255.255.0  broadcast 10.1.1.255
        inet6 fe80::5a66:1535:3816:3f46  prefixlen 64  scopeid 0x20<link>
        ether e8:80:88:a9:e3:18  txqueuelen 1000  (Ethernet)
        RX packets 28  bytes 3701 (3.7 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 41  bytes 5684 (5.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 16  memory 0xbe280000-be2a0000  

ubuntu@Roc2-P-4:~$ lspci -nn -s 00:1f.6
00:1f.6 Ethernet controller [0200]: Intel Corporation Device [8086:0dc5] (rev 01)
ubuntu@Roc2-P-4:~$ sudo su
root@Roc2-P-4:/home/ubuntu# cd /sys/kernel/debug/pmc_core/
root@Roc2-P-4:/sys/kernel/debug/pmc_core# cat slp_s0_residency_usec 
0
root@Roc2-P-4:/sys/kernel/debug/pmc_core# cat package_cstate_show 
Package C2 : 45785582
Package C3 : 3986261
Package C6 : 11594305
Package C7 : 0
Package C8 : 25924196
Package C9 : 0
Package C10 : 120111872
root@Roc2-P-4:/sys/kernel/debug/pmc_core# rtcwake -s 20 -m mem
rtcwake: assuming RTC uses UTC ...
rtcwake: wakeup from "mem" using /dev/rtc0 at Tue Mar 26 06:00:45 2024
root@Roc2-P-4:/sys/kernel/debug/pmc_core# cat package_cstate_show 
Package C2 : 49787179
Package C3 : 4170036
Package C6 : 13421572
Package C7 : 0
Package C8 : 26853893
Package C9 : 0
Package C10 : 137728780
root@Roc2-P-4:/sys/kernel/debug/pmc_core# cat slp_s0_residency_usec 
15807784
root@Roc2-P-4:/sys/kernel/debug/pmc_core#

Comment 20 Hui Wang 2024-03-27 01:10:08 UTC

BTW, I got ("Name" : "AMT [unprovisioned]") from all machines I tested.

Comment 21 Sasha Neftin 2024-03-27 12:42:33 UTC

(In reply to Hui Wang from comment #20)
> BTW, I got ("Name" : "AMT [unprovisioned]") from all machines I tested.

Thank you Hui,
Please, tell Lenovo, HP, DELL to contact Intel's PAE and Intel will take a debug. (I know part of them already contacted PAE.)