Bug 85621

Summary: pcie_aspm.policy=powersave pcie_aspm=force does not enable L0sL1 (L0s L1) on each ASPM PCIe device - Intel Xeon E3-1200
Product: Power Management Reporter: linux-ide
Component: Run-Time-PMAssignee: Lv Zheng (lv.zheng)
Status: CLOSED DOCUMENTED    
Severity: normal CC: boniek83, lenb, rui.zhang
Priority: P1    
Hardware: Intel   
OS: Linux   
Kernel Version: 3.17 20140928 Subsystem:
Regression: No Bisected commit-id:
Attachments: lspci -vnvn

Description linux-ide 2014-10-05 13:06:52 UTC
Created attachment 152461 [details]
lspci -vnvn

4) What happened instead
===============================================
# dmesg | grep -e _OSC -e ASPM
[ 0.000000] PCIe ASPM is forcibly enabled
[ 0.085767] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
[ 0.116506] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[ 0.116664] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME]
[ 0.116764] acpi PNP0A08:00: _OSC: OS now controls [AER PCIeCapability]
[ 0.125997] acpi PNP0A08:00: Disabling ASPM (FADT indicates it is unsupported)
# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-3.17.0-999-generic root=/dev/sda1 ro biosdevname=0 intel_pstate=enable ipv6.disabled=1 debug ignore_loglevel panic=10 pcie_aspm.policy=powersave pcie_aspm=force r8168.aspm=1 r8168.eee_enable=1 oops=panic
# lspci -vvvv | grep ASPM
LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <8us
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk-
LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us
LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- CommClk+
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- CommClk+

3) What you expected to happen
===============================================
# lspci -vvvv | grep ASPM
LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <8us
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- CommClk-
LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- CommClk-
LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- CommClk+
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- CommClk+

Corresponding PCIe devices:
# lspci -s 00:01.00 -nn
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)
# lspci -s 00:1c.00 -nn
00:1c.0 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 [8086:8c10] (rev d5)
# lspci -s 00:1c.05 -nn
00:1c.5 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #6 [8086:8c1a] (rev d5)
# lspci -s 03:00.00 -nn
03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 11)

Current workaround:
ASPM L0s L1 Enabled on each PCIe device can be created using manually using the correct setpci commands. This reliably enables ASPM L0s and L1 saving 25% in power consumption when measured at wall over at least 8 hours:
# setpci -s 00:01.00 0xB0.B=0x3 # Intel Xeon E3-1200 PCIe controller [8086:0c01]
# setpci -s 00:1c.05 0x50.B=0x43 # Intel 8 series PCIe root port #6 [8086:8c1a]
# setpci -s 00:1c.00 0x50.B=0x43 # Intel 8 series PCIe root port #1 [8086:8c10]
# setpci -s 03:00.00 0x80.B=0x43 # Realtek Gigabit Ethernet [10ec:8168]

2) The version of the kernel you are using
===============================================
# uname -a
Linux ubuntu14 3.17.0-999-generic #201409282205 SMP Mon Sep 29 02:06:39 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

1) The release of Ubuntu
===============================================
Description: Ubuntu 14.04.1 LTS
Release: 14.04

Version signature is missing due to:
# cat /proc/version_signature
cat: /proc/version_signature: No such file or directory
Comment 1 Len Brown 2014-10-28 05:02:41 UTC
*** Bug 85401 has been marked as a duplicate of this bug. ***
Comment 2 Lv Zheng 2014-10-28 07:52:14 UTC
Hi,

pcie_aspm=force cannot force enabling aspm, it only ensures the OSPM <-> BIOS negotiation will not result in an aspm disabling operation. It doesn't mean if aspm is disabled previously in the PCI layer, it can be enabled by this ACPI related option. So this is a PCI issue.

Thanks
Comment 3 Lv Zheng 2015-02-11 02:19:43 UTC
Closing it due to no need for now to fix it.