Bug 72211 - intel_pstate Haswell CPU is not entering hardware package idle state
Summary: intel_pstate Haswell CPU is not entering hardware package idle state
Status: NEW
Alias: None
Product: Drivers
Classification: Unclassified
Component: Network (show other bugs)
Hardware: x86-64 Linux
: P1 normal
Assignee: drivers_network@kernel-bugs.osdl.org
Depends on:
Reported: 2014-03-16 16:30 UTC by opensource
Modified: 2017-12-16 19:47 UTC (History)
10 users (show)

See Also:
Kernel Version: 3.13.6
Regression: No
Bisected commit-id:

Kernel Config (Archlinux) (35.60 KB, application/gzip)
2014-03-16 16:30 UTC, opensource

Description opensource 2014-03-16 16:30:22 UTC
Created attachment 129641 [details]
Kernel Config (Archlinux)

Intel Pentium G3220 is not reaching package idle states above PC3. 

CPU is in C6 for over 99% (no C7 on pentium). Package idle states start with 97% in PC2. After five minute idle they drop to 97% PC3. But they never enter PC6. 

Kernel (archlinux), 
Linux eagle 3.13.6-1-ARCH #1 SMP PREEMPT Fri Mar 7 22:47:48 CET 2014 x86_64 GNU/Linux

config is attached. I tried 3.14-rc6 with similiar results. 

turbostat output is below. 

[root@eagle ~]# turbostat -v
turbostat v3.5 April 26, 2013 - Len Brown <lenb@kernel.org>
CPUID(0): GenuineIntel 13 CPUID levels; family:model:stepping 0x6:3c:3 (6:60:3)
30 * 100 = 3000 MHz TSC frequency
cpu0: MSR_IA32_POWER_CTL: 0x0004005d (C1E auto-promotion: DISabled)
cpu0: MSR_NHM_SNB_PKG_CST_CFG_CTL: 0x1e000403 (UNdemote-C3, UNdemote-C1, demote-C3, demote-C1, UNlocked: pkg-cstate-limit=3: pc6)
cpu0: MSR_NHM_TURBO_RATIO_LIMIT: 0x1e1e1e1e
30 * 100 = 3000 MHz max turbo 4 active cores
30 * 100 = 3000 MHz max turbo 3 active cores
30 * 100 = 3000 MHz max turbo 2 active cores
30 * 100 = 3000 MHz max turbo 1 active cores
cpu0: MSR_IA32_ENERGY_PERF_BIAS: 0x00000006 (balanced)
cpu0: MSR_RAPL_POWER_UNIT: 0x000a0e03 (0.125000 Watts, 0.000061 Joules, 0.000977 sec.)
cpu0: MSR_PKG_POWER_INFO: 0x000001a8 (53 W TDP, RAPL 0 - 0 W, 0.000000 sec.)
cpu0: MSR_PKG_POWER_LIMIT: 0x428212001a81a8 (UNlocked)
cpu0: PKG Limit #1: ENabled (53.000000 Watts, 8.000000 sec, clamp DISabled)
cpu0: PKG Limit #2: ENabled (66.250000 Watts, 0.002441* sec, clamp DISabled)
cpu0: MSR_PP0_POLICY: 0
cpu0: MSR_PP0_POWER_LIMIT: 0x00000000 (UNlocked)
cpu0: Cores Limit: DISabled (0.000000 Watts, 0.000977 sec, clamp DISabled)
cpu0: MSR_PP1_POLICY: 0
cpu0: MSR_PP1_POWER_LIMIT: 0x00000000 (UNlocked)
cpu0: GFX Limit: DISabled (0.000000 Watts, 0.000977 sec, clamp DISabled)
cpu0: MSR_IA32_TEMPERATURE_TARGET: 0x00641400 (100 C)
cpu0: MSR_IA32_PACKAGE_THERM_STATUS: 0x88490000 (27 C)
cpu0: MSR_IA32_THERM_STATUS: 0x88490000 (27 C +/- 1)
cpu1: MSR_IA32_THERM_STATUS: 0x88490000 (27 C +/- 1)

cor CPU    %c0  GHz  TSC SMI    %c1    %c3    %c6    %c7 
          0.13 2.99 2.99   0   0.02   0.00  99.85   0.00
  0   0   0.22 2.99 2.99   0   0.02   0.00  99.77   0.00
  1   1   0.03 2.99 2.99   0   0.03   0.00  99.94   0.00

CTMP  PTMP   %pc2   %pc3   %pc6   %pc7  Pkg_W  Cor_W GFX_W
   26   28   2.11  97.50   0.00   0.00  10.30   0.03  0.00
   26   28   2.11  97.50   0.00   0.00  10.30   0.03  0.00
Comment 1 Alan 2014-03-20 12:22:11 UTC
(adding Len)
Comment 2 opensource 2014-03-24 21:14:19 UTC
the onboard network device (realtek r8168) was the issue. I replaced the vanilla kernel module (r8169) with realteks r8168. Once the monitor is switched off via dpms i reach 97% package c state pc6. 

I assume the r8169 does not support aspm. 

Should this bug be closed or moved to drivers?
Comment 3 Tobias Jakobi 2014-06-29 13:40:01 UTC
Similar problem here on a i7-4700HQ. According to i7z the CPU doesn't enter anything below C1 (Halt). And yes, there is nothing (except for init, which is OpenRC here on Gentoo) running when I measured this.

With the same workload, my other system, a i5 M-450, stays in C6 ~90% of the time.
Comment 4 Tobias Jakobi 2014-07-17 00:05:53 UTC
OK, this turned out to be just a false alarm. The latest stable version of i7z doesn't correctly display statistics from the Haswell.

Using the git version and also turbostat reveals that the cores are staying mostly in C7 when the system is idle. So everything is like it should be.

Sorry for the noise!
Comment 5 Łukasz Siudut 2014-07-31 08:38:28 UTC
I'm struggling with this problem on Lenovo T440S. It's never going below PC3, moreover to achieve PC3 I need to suspend/resume my system. As a result my average power usage on idle is fixing around 7W, while in Windows I can easily get down to 5W.

Is there any way I can determine what keeps my system to go down below PC3? Btw. I don't experience this problem on my second laptop, also Lenovo, model W540. It achieve PC7 easily once screen is switched of via dkms (hi-res LCD).
Comment 6 linux-ide 2014-08-30 17:20:56 UTC
r8169 is not working as expected: it is not being able to make the processor enter Package State C6 (pc6) for RTL8111 hardware where ASPM is enabled.

This issue is still present in todays mainline built of kernel 3.17 (Aug 30th 2014).

With r8168-dkms Haswell G1820 and Haswell-R G1840 processors will enter pc6.
With r8169 and kernel 3.13 or 3.17 they don't.

Affected hardware: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 11)
Comment 7 Joerg Beyer 2014-09-13 15:24:33 UTC
In my case the change to the r8168-dkms net work driver package (on ubuntu-14.04) vastly improved the power consumption. I can reach Package Stage C6 with the new driver, while I couldn't with the vanilla driver.

That brought power consumption from about 20 watts to 13.
Comment 8 Joerg Beyer 2014-09-13 19:39:09 UTC
... but, r8168-dkms slowed my network connections down by a factor around 100. So that is not working.
Comment 9 linux-ide 2014-09-16 08:59:49 UTC
How did you measure this?

bugzilla-daemon@bugzilla.kernel.org schreef op 13-09-14 om 21:39:
> https://bugzilla.kernel.org/show_bug.cgi?id=72211
> --- Comment #8 from Joerg Beyer <joerg.beyer@gmail.com> ---
> ... but, r8168-dkms slowed my network connections down by a factor around
> 100.
> So that is not working.

Note You need to log in before you can comment on or make changes to this bug.