Current 3.13 and mainline 3.17 kernel do not enable PCIe ASPM when BOOT_ARCH[4] in the ACPI FADT is set, not even with pcie_aspm=force pcie_aspm.policy=powersave: no boot parameter: LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <8us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ pcie_aspm.policy=powersave: LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <8us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ pcie_aspm=force: LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <8us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk+ pcie_aspm=force pcie_aspm.policy=powersave: LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <8us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk+ # lspci -s 00:01.00 -nn 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06) # lspci -s 00:1c.00 -nn 00:1c.0 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 [8086:8c10] (rev d5) # lspci -s 00:1c.05 -nn 00:1c.5 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #6 [8086:8c1a] (rev d5) # lspci -s 03:00.00 -nn 03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 11) These setpci command do reliably enable ASPM L0s and L1 saving 25% in power consumption when measured at wall over at least 8 hours: setpci -s 00:01.00 0xB0.B=0x3 # Intel Xeon E3-1200 PCIe controller [8086:0c01] setpci -s 00:1c.05 0x50.B=0x43 # Intel 8 series PCIe root port #6 [8086:8c1a] setpci -s 00:1c.00 0x50.B=0x43 # Intel 8 series PCIe root port #1 [8086:8c10] setpci -s 03:00.00 0x80.B=0x43 # Realtek [10ec:8168] From 11 out of 11 tested recent (Ivy Bridge: Intel Corporation 7 Series/C210 or Haswell Intel Corporation 8 Series/C220) desktop motherboards, all have the PCIe ASPM Not supported (V4) bit set. Only the Asus H81M-A has enough BIOS configuration options to clear this "PCIe ASPM Not supported"-bit (under Advanced\Platform Misc Configuration > Native ASPM [Disabled]->[Enabled]). The 11 tested mainboards and their chipset: === ASRock, H77 Pro4-M P2.00, H77: PCIe ASPM Not supported (V4): 0x1 Intel, DC3217BY 0053, QS77: PCIe ASPM Not supported (V4): 0x1 ASRock, H81 Pro BTC P1.80, H81: PCIe ASPM Not supported (V4): 0x1 ASRock, H81M-VG4, H81: PCIe ASPM Not supported (V4): 0x1 ASUSTeK, H81M-A, H81: PCIe ASPM Not supported (V4): 0x1 Gigabyte, H81M-H, H81: PCIe ASPM Not supported (V4): 0x1 MSI, H81M-P33 (MS-7817), H81: PCIe ASPM Not supported (V4): 0x1 ASRock, B85M Pro4 P2.10, B85: PCIe ASPM Not supported (V4): 0x1 ASRock, B85M-ITX P2.20, B85: PCIe ASPM Not supported (V4): 0x1 ASUS, Q87T, Q87: PCIe ASPM Not supported (V4): 0x1 ASRock, Z87E-ITX, Z87: PCIe ASPM Not supported (V4): 0x1 === A side-effect on at least 3 Intel 8 series chipset Haswell is that processor package P-state does only reach level pc2 or pc3. On the other hand the ASRock H77 Pro4-M P2.00 (Intel 7 series chipset) doesn't suffer from this issue and reached pc6. Three mainboards, their best processor package P-state, and NIC: === ASRock, H81 Pro BTC: pc3 RTL8111GR* Gigabyte, B85M-HD3: pc2 RTL8111F ASRock, Z87E-ITX: pc3 Intel I217V === *The ASRock H81 Pro BTC even runs the RTL-8168 driver (r8168 module) version 8.039. Without this version the package state will not reach PC6 at all.
Created attachment 152171 [details] fwts --batch output Canonical (Ubuntu) Firmware Test Suite 14.09 output for ASRock H81 Pro BTC motherboard.
Created attachment 152181 [details] dmesg output
There seems to be no _OSC issue prohibiting (request failed or not granting) the enablement of ASPM: $ dmesg | grep -e _OSC -e PCIe -e ASPM [ 0.084223] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it [ 0.116621] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] [ 0.116775] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME] [ 0.116874] acpi PNP0A08:00: _OSC: OS now controls [AER PCIeCapability] [ 0.133733] acpi PNP0A08:00: Disabling ASPM (FADT indicates it is unsupported) [ 0.554012] r8169 0000:03:00.0: can't disable ASPM; OS doesn't have ASPM control $ cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.13.0-35-generic root=/dev/sda1 ro biosdevname=0 intel_pstate=enable ipv6.disable=1 debug ignore_loglevel panic=10 oops=panic
The 3.17 04-Oct-2014 mainline kernel is even worse, it disabled all PCIe ASPM on all devices whether or not any kernel pcie_aspm parameter is set: # cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.17.0-999-generic root=/dev/sda1 ro biosdevname=0 intel_pstate=enable ipv6.disable=1 debug ignore_loglevel panic=10 pcie_aspm.policy=powersave pcie_aspm=force oops=panic # lspci -vvvv | grep ASPM LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <256ns, L1 <8us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ # dmesg | grep -e _OSC -e PCIe -e ASPM [ 0.000000] PCIe ASPM is forcibly enabled [ 0.090565] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it [ 0.121461] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] [ 0.121623] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME] [ 0.121726] acpi PNP0A08:00: _OSC: OS now controls [AER PCIeCapability] [ 0.138681] acpi PNP0A08:00: Disabling ASPM (FADT indicates it is unsupported) When removing the pcie_aspm boot parameters: # cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.17.0-999-generic root=/dev/sda1 ro biosdevname=0 intel_pstate=enable ipv6.disable=1 debug ignore_loglevel panic=10 oops=panic # lspci -vvvv | grep ASPM LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <256ns, L1 <8us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
The 2014-10-04 output was where firmware/BIOS chipset setting "Program PCIe ASPM after OpROM" was set to [Disabled] = PCIe ASPM will be programmed before OpROM. The machine boots UEFI with CSM disabled. After changing firmware/BIOS chipset setting "Program PCIe ASPM after OpROM" from [Disabled] to [Enabled], the results differ: =========== Kernel 3.13 =========== # dmesg | grep -e _OSC -e ASPM [ 0.080511] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it [ 0.113071] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] [ 0.113223] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME] [ 0.113319] acpi PNP0A08:00: _OSC: OS now controls [AER PCIeCapability] [ 0.122196] acpi PNP0A08:00: Disabling ASPM (FADT indicates it is unsupported) # cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.13.0-35-generic root=/dev/sda1 ro biosdevname=0 intel_pstate=enable ipv6.disabled=1 debug ignore_loglevel panic=10 oops=panic # lspci -vvvv | grep ASPM LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <8us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ =========================== Kernel 3.13 force powersave =========================== # dmesg | grep -e _OSC -e ASPM [ 0.000000] PCIe ASPM is forcibly enabled [ 0.080443] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it [ 0.113215] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] [ 0.113379] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME] [ 0.113488] acpi PNP0A08:00: _OSC: OS now controls [AER PCIeCapability] [ 0.122094] acpi PNP0A08:00: Disabling ASPM (FADT indicates it is unsupported) # cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.13.0-35-generic root=/dev/sda1 ro biosdevname=0 intel_pstate=enable ipv6.disabled=1 debug ignore_loglevel panic=10 pcie_aspm.policy=powersave pcie_aspm=force r8168.aspm=1 r8168.eee_enable=1 oops=panic # lspci -vvvv | grep ASPM LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <8us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk+ =============================== Kernel 3.17 mainline 4-Oct-2014 =============================== # dmesg | grep -e _OSC -e ASPM [ 0.085762] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it [ 0.116538] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] [ 0.116696] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME] [ 0.116796] acpi PNP0A08:00: _OSC: OS now controls [AER PCIeCapability] [ 0.126042] acpi PNP0A08:00: Disabling ASPM (FADT indicates it is unsupported) # cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.17.0-999-generic root=/dev/sda1 ro biosdevname=0 intel_pstate=enable ipv6.disabled=1 debug ignore_loglevel panic=10 oops=panic # lspci -vvvv | grep ASPM LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <8us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ =============================================== Kernel 3.17 mainline 4-Oct-2014 force powersave =============================================== # dmesg | grep -e _OSC -e ASPM [ 0.000000] PCIe ASPM is forcibly enabled [ 0.085767] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it [ 0.116506] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] [ 0.116664] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME] [ 0.116764] acpi PNP0A08:00: _OSC: OS now controls [AER PCIeCapability] [ 0.125997] acpi PNP0A08:00: Disabling ASPM (FADT indicates it is unsupported) # cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.17.0-999-generic root=/dev/sda1 ro biosdevname=0 intel_pstate=enable ipv6.disabled=1 debug ignore_loglevel panic=10 pcie_aspm.policy=powersave pcie_aspm=force r8168.aspm=1 r8168.eee_enable=1 oops=panic # lspci -vvvv | grep ASPM LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <8us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk- LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- CommClk+ LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- CommClk+
Moved to "Power Management" under a new title " pcie_aspm.policy=powersave pcie_aspm=force does not enable L0sL1 (L0s L1) on each ASPM PCIe device". See https://bugzilla.kernel.org/show_bug.cgi?id=85621
*** This bug has been marked as a duplicate of bug 85621 ***