Bug 15064

I should also note that I've seen this problem on ubuntu and gentoo, on both ubuntu 32bit and 64bit, and gentoo 64bit.  This includes testing on vanilla kernel.

I also suffer from this issue when using the current and older Linux kernels (<= 2.6.31) on my laptop. I'm aware that the Turbo Boost feature in Intel's CORE architecture, is hardware controlled, and will only kick in when other cores remain idle. I've read that older CORE cpu's correctly handle Turbo Boost under Linux. I think this is an issue with newer models.

I've witness that all CPUs' multipliers remain capped @ 13x at all time. Using the i7z program I've noticed that the cores will never reach C-states beyond C1. Now, when I turn off ACPI support from the boot options @ grub, two of my cores are not accessible, hence idle, and the remaining cores will have their multiplier boosted up to 18x. This proofs that Turbo boost indeed works for this CPU's under a Linux environment, but the current ACPI modules are not able to use higher C-states, which might be the reason why the CPU multipliers won't automatically step up.

SYSTEM INFORMATION
        Running Ubuntu Linux, the Ubuntu 9.10 (karmic) release.
        GNOME: 2.28.1 (Ubuntu 2009-11-03)
        Kernel version: 2.6.31-18-server (#55-Ubuntu SMP Fri Jan 8 15:51:55 UTC 2010)

CPU INFORMATION
        GenuineIntel, Intel(R) Core(TM) i7 CPU       Q 720  @ 1.60GHz
        Number of CPUs: 8
        CPU clock currently at 933.000 MHz with 6144 KB cache
        Numbering: family(6) model(30) stepping(5)
        Bogomips: 3199.58
        Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology tsc_reliable nonstop_tsc pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid

please attach the acpidump output of your box.

Created attachment 24611 [details]
acpidump + sysinfo

Created attachment 24612 [details]
acpidump alienware m15x

Anders,

please
acpidump --addr 0xC771AA98 --length 0x000002DA > cpu0ist.dat
acpidump --addr 0xC771A718 --length 0x00000303 > apist.dat
acpidump --addr 0xC7719618 --length 0x000005CD > cpu0cst.dat
acpidump --addr 0xC7718D98 --length 0x00000119 > apcst.dat

and attach the *.dat files here.

please also paste the output of 

powertop -d

Created attachment 24625 [details]
[Asus G51J] powertop -d

Collecting data for 15 seconds 


< Detailed C-state information is not available.>




you have no C states to speak of.
does the dmesg have any acpi errors or somesuch? if you don't have real C states then yeah, turbo isn't going to help. This is the part that needs fixing first...

Created attachment 24626 [details]
[Asus G51J] dmesg

I do see some ACPI related warnings (lines 103,104), but no errors.

does adding acpi_osi=Linux to the kernel command line help?
(it should not, it would be a bios bug if it does ;-)

Nope, adding 'acpi_osi=Linux' had no apparent effect, I still see the same dmesg's ACPI warnings and I don't see any C-states on powertop's verbose.

please attach the kernel .config

note that it should include the following:

CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_CPU_IDLE_GOV_MENU=y

CONFIG_ACPI_PROCESSOR=y

and when the processor driver is loaded and working,
you should be able to observe the C-states it is providing
both via ACPI:

cat /proc/acpi/processor/*/power

and via cpuidle:

grep .  /sys/devices/system/cpu/cpu*/cpuidle/*/*

Created attachment 24629 [details]
cpustat utility source code

please compile this cpustat utility, run as root for
10 seconds or so, interrupt, and paste the output to this report.

cpustat will print out the maximum turbo capabilities of your Nehalem,
and will also print the idle time and the observed processor
frequency at 2 second intervals.

eg. when run on my desktop it shows that a single threaded program
can cause the 2.93 GHz CPU to run a core up to 3.16 GHz
for a 2 second interval, and if I run two cycle soakers,
they both run at 3.06 GHz:

# ./cpustat
Nehalem multiplier 22, TSC frequency 2933 MHz
Nehalem 4 cores active: 23 mult, max turbo frequency = 3067 MHz
Nehalem 3 cores active: 23 mult, max turbo frequency = 3067 MHz
Nehalem 2 cores active: 23 mult, max turbo frequency = 3067 MHz
Nehalem 1 core active: 24 mult, max turbo frequency = 3200 MHz
CPU:      0      1      2      3      4      5      6      7 
c0%:   0.04   0.37   0.06   2.18  50.22   0.75   0.05   0.01 
c1%:  50.50   2.44   0.16   4.59   0.32   2.06   0.17   6.76 
c3%:  34.10  76.97  42.27  93.18  34.10  76.98  42.27  93.17 
c6%:  15.37  20.22  57.51   0.05  15.36  20.22  57.51   0.05 
GHz:   2.02   2.87   2.76   2.97   3.16   2.99   2.11   2.16 
c0%:   0.02   0.62   0.13   4.01  99.73   1.41   0.02   0.00 
c1%:  99.98   5.20   0.12  48.32   0.27   4.41   0.22  52.33 
c3%:   0.00  94.17  99.75  47.63   0.00  94.17  99.75  47.64 
c6%:   0.00   0.01   0.00   0.03   0.00   0.01   0.00   0.03 
GHz:   3.06   3.06   3.06   3.06   3.12   3.06   3.06   3.06 
c0%:   0.02   0.67   5.37   4.84  99.52   1.56   0.06   0.00 
c1%:  99.98   5.40   0.15   5.38   0.48   4.50   5.47  10.22 
c3%:   0.00  64.27  60.12  89.69   0.00  64.27  60.12  89.69 
c6%:   0.00  29.66  34.35   0.09   0.00  29.67  34.35   0.09 
GHz:   3.06   3.06   3.06   3.06   3.17   3.06   3.06   3.06 
c0%:   0.02   0.64 100.00   5.62 100.00   1.55   0.03   0.01 
c1%:  99.98   7.01   0.00  48.47   0.00   6.10  99.97  54.08 
c3%:   0.00  61.15   0.00  45.82   0.00  61.15   0.00  45.82 
c6%:   0.00  31.21   0.00   0.09   0.00  31.21   0.00   0.09 
GHz:   3.06   3.06   3.06   3.06   3.06   3.06   3.06   3.06 
c0%:   0.69   0.35  97.50   4.33  99.99   0.80   0.03   0.01 
c1%:  99.31   3.77   0.08   3.92   0.01   3.32  97.56   8.24 
c3%:   0.00  63.16   2.38  91.72   0.00  63.17   2.38  91.72 
c6%:   0.00  32.72   0.03   0.03   0.00  32.71   0.03   0.03 
GHz:   3.06   3.06   3.06   3.06   3.06   3.06   3.06   3.06 
c0%:   1.00   0.04   0.07   0.83  75.32   0.05   0.03   0.01 
c1%:  74.59   0.17   0.08  51.00   0.26   0.16   0.12  51.82 
c3%:  24.38  71.13  94.71  48.11  24.38  71.12  94.71  48.12 
c6%:   0.04  28.66   5.14   0.06   0.04  28.67   5.14   0.06 
GHz:   2.98   2.26   3.04   2.91   3.10   2.74   2.01   2.66 
c0%:   0.04   0.01   0.06   0.09   0.01   0.02   0.04   0.01 
c1%:   0.07   0.03   0.07   0.05   0.09   0.02   0.09   0.13 
c3%:  49.92  99.88  39.90  99.85  49.92  99.87  39.90  99.84 
c6%:  49.97   0.08  59.97   0.02  49.98   0.09  59.97   0.02 
GHz:   1.60   1.60   1.60   1.60   1.60   1.60   1.60   1.60 
^C

Created attachment 24631 [details]
acpidump specific addresses, powertop -d, config

I also see "< Detailed C-state information is not available.>"

The directories /sys/devices/system/cpu/*/cpuidle does not exist

cat /sys/devices/system/cpu/cpuidle/*
    acpi_idle
    menu
So it's there (and .config confirms it), but there's no cpuidle directory in the /sys/devices/system/cpu/* dirs.


The cpustat utility shows cpu C0/C1 state but it never touches C3 or C6, under load or idle.

Created attachment 24632 [details]
dmesg.log, on a clean reboot

Adding dmesg.log, note that it was done on a clean reboot, and the tests were run after several s2ram cycles.  I see this issue on clean boots as well though.

There's also been an update the issue in i7z's issue tracker:
http://code.google.com/p/i7z/issues/detail?id=1
comment #8:
"I am trying to disable cores and logically that should enable the disabled cores to go into C6 state, but i am finding its not and they goes into C1 state (idle power is equivalent to C1 state rather than C6 state).
"

Created attachment 24634 [details]
[Asus G51J] Kernel's .config

All the following flags are enabled inside the kernel's config:
 CONFIG_CPU_IDLE=y
 CONFIG_CPU_IDLE_GOV_LADDER=y
 CONFIG_CPU_IDLE_GOV_MENU=y

 CONFIG_ACPI_PROCESSOR=y

cat /proc/acpi/processor/*/power displays the following output for each cpu:

 active state:            C0
 max_cstate:              C8
 maximum allowed latency: 2000000000 usec
 states:
     C1:                  type[C1] promotion[--] demotion[--] latency[003] usage[00000000] duration[00000000000000000000]

And there is no 'cpuidle' folder, for any /sys/devices/system/cpu/cpu*/

Created attachment 24635 [details]
[Asus G51J] cpustat output

The Turbo capabilities stated by cpustat, appear to be correct. Yet is shows the idling processors are always stuck @ (C0,C1) states and so their clock capped @ 1.73Ghz, while the Turbo speeds are 2.4Ghz - 2.8Ghz.

re: comment #16 about offline cores not entering c6
that is bug 5471

Both the Asus G51J and the asus G51J are laptops.
If you pull out the AC and run on battery,
do the C0-states appear? (show power file above)
If no, what if you boot on DC?

Created attachment 24644 [details]
[Asus G51J] On Battery Results

This are some interesting results.

When booting from battery power, I get the extra C-states (C3,C6), plus Turbo Boost appears to be working.
Once I plug in back AC power, I loose both the extra C-states and Turbo Boost.
If I unplug the AC power and go back to battery, the extra C-states appear once again, but Turbo Boost won't kick in.

I attached new versions of the outputs you've requested. They were created in the following chronological order:

1) .dc files, are the outputs while booting on battery power.
2) .postdc files, are the outputs after going back to AC power.
3) .postac files, are the outputs after going back to battery power.

This is apparently due to an issue we have in Linux where we
disqualify C2 state because of its advertised latency of > 100usec.
This causes both a loss in power savings and a loss of ability
to enter turbo mode.

If you boot with "processor.nocst=1", then we'll use the legacy
method of probing C-states.  totally bogus, but you may get
C3 that way, since it is set to 1000, the max legal value:

[060h 096  2]                   C2 Latency : 0065
[062h 098  2]                   C3 Latency : 03E9

Windows apparently doesn't check the latency of C2
before blindly using it.  As the ACPI spec says that
there is no limit on latency for C-states described
by _CST, apparently Linux should do the same.

Created attachment 24649 [details]
patch to allow > 100usec C2 via _CST

Please test this patch, and show the output from
cat /proc/acpi/processor/*/power

Created attachment 24650 [details]
patch to allow > 100usec C2 via _CST (take 2)

test this one instead -- fixes typo in previous version
which caused it not to build with CONFIG_ACPI_DEBUG

Works on my HP Envy 15.

achiang@dre:/proc/acpi$ grep . processor/CPU0/power 
active state:            C0
max_cstate:              C8
maximum allowed latency: 139319 usec
states:
    C1:                  type[C1] promotion[--] demotion[--] latency[003] usage[00002494] duration[00000000000000000000]
    C2:                  type[C2] promotion[--] demotion[--] latency[245] usage[00017339] duration[00000000090703736486]

Note the C2 latency of 245. Without this patch, I would not have C2, since it is obviously > 100 and would have failed the test.

Tested-by: Alex Chiang <achiang@hp.com>

Created attachment 24658 [details]
patch to allow > 100usec C2 via _CST (take 3)

thanks Alex.
refreshed patch attached to reflect what is now in the tree.

Worked on my Asus G51J.

active state:            C0
max_cstate:              C8
maximum allowed latency: 2000000000 usec
states:
    C1:                  type[C1] promotion[--] demotion[--] latency[003] usage[00026036] duration[00000000000000000000]
    C2:                  type[C2] promotion[--] demotion[--] latency[245] usage[00183647] duration[00000000078609016666]

Thanks Len.

On my Dell Studio 17 (model 1557) with Core i7 Q720 proc, Len's fine cpustat tool shows C6 states being used and Turbo Boost working effectively, though when booting normally with all 8 hyperthreaded pseudo-cores enabled.

Booting with 'maxcpus=4' to get 4 cores reduces cache thrashing + latency penalties from stalling, but we don't hit C6 and don't get Turbo Boost. Dell doesn't have a BIOS option to disable HyperThreading, which is poor.

Is there possibility to improve this situation from Linux's PoV? I'm submitting feedback to Dell, but don't expect anything...

The original bugreport here was solved quickly and cleanly, your problem is a different one.

You can try booting with noht to disable hyperthreading, but either way you dont really want to on an i7, which is a very different beast from a hyperthreading point of view than the older intel's.  Keep future discussions on this somewhere else though.

Daniel,
FYI, my Nehalem Dell Studio XPS 435 MT
has an F2/SETUP option to disable HT:
Advanced BIOS Features/CPU Features/Hyper-threading [enable|disable]

Created attachment 24673 [details]
turbostat.c

FYI, I've updated the utility above, now named "turbostat".
It is now slightly less dumb, checking cpuid and more counters.

Here is an example of a 2.93Ghz Nehalem running at 3.17GHz

(No package C-states enabled on this BIOS/stepping)

root@nehalem lenb]# ./turbostat
Nehalem multiplier 22, TSC frequency 2933 MHz
Nehalem 4 cores active: 23 mult, max turbo frequency = 3067 MHz
Nehalem 3 cores active: 23 mult, max turbo frequency = 3067 MHz
Nehalem 2 cores active: 23 mult, max turbo frequency = 3067 MHz
Nehalem 1 core active: 24 mult, max turbo frequency = 3200 MHz
 CPU   GHz    TSC    %c0    %c1    %c3    %c6   %pc3   %pc6   %pc7 
   0   3.06   2.93   1.11   0.12  88.46  10.30   0.00   0.00   0.00
   1   3.06   2.93   1.59   4.00  85.01   9.40   0.00   0.00   0.00
   2   3.15   2.93   0.22  99.78   0.00   0.00   0.00   0.00   0.00
   3   3.06   2.93   4.30   5.59  90.02   0.09   0.00   0.00   0.00
   4   3.06   2.93   0.01   1.22  88.47  10.30   0.00   0.00   0.00
   5   3.06   2.93   0.11   5.47  85.01   9.40   0.00   0.00   0.00
   6   3.17   2.93  99.50   0.50   0.00   0.00   0.00   0.00   0.00
   7   3.06   2.93   0.01   9.88  90.02   0.09   0.00   0.00   0.00

patch in comment #26 shipped in Linux-2.6.33-rc5

closed.

for the record, the latest turbostat utility
is now published in the latest pmtools package:

http://ftp.kernel.org/pub/linux/kernel/people/lenb/acpi/utils/

I'm not sure this bug is solved (I'm running vanilla 2.6.33-rc8):

Intel Core i5 650:
CPU family:            6
Model:                 37
Stepping:              2
CPU MHz:               3311.733 (slightly overclocked, BCLK=137 vs 133 normal)

./turbostat (idle)
 CPU   GHz    TSC    %c0    %c1    %c3    %c6   %pc3   %pc6   %pc7
   0   1.29   3.31   0.59  99.41   0.00   0.00   0.00   0.00   0.00
   1   1.77   3.31   0.91  99.09   0.00   0.00   0.00   0.00   0.00
   2   2.50   3.31   1.37  98.63   0.00   0.00   0.00   0.00   0.00
   3   1.59   3.31   0.83  99.17   0.00   0.00   0.00   0.00   0.00
 CPU   GHz    TSC    %c0    %c1    %c3    %c6   %pc3   %pc6   %pc7
   0   1.29   3.31   0.67  99.33   0.00   0.00   0.00   0.00   0.00
   1   1.57   3.31   0.71  99.29   0.00   0.00   0.00   0.00   0.00
   2   2.45   3.31   1.51  98.49   0.00   0.00   0.00   0.00   0.00
   3   1.62   3.31   0.78  99.22   0.00   0.00   0.00   0.00   0.00

./turbostat (one CPU intensive running application)
 CPU   GHz    TSC    %c0    %c1    %c3    %c6   %pc3   %pc6   %pc7
   0   3.31   3.31   1.25  98.75   0.00   0.00   0.00   0.00   0.00
   1   3.31   3.31   0.73  99.27   0.00   0.00   0.00   0.00   0.00
   2   3.31   3.31 100.00   0.00   0.00   0.00   0.00   0.00   0.00
   3   3.31   3.31   0.61  99.39   0.00   0.00   0.00   0.00   0.00
 CPU   GHz    TSC    %c0    %c1    %c3    %c6   %pc3   %pc6   %pc7
   0   3.31   3.31   1.26  98.74   0.00   0.00   0.00   0.00   0.00
   1   3.31   3.31   0.72  99.28   0.00   0.00   0.00   0.00   0.00
   2   3.31   3.31 100.00   0.00   0.00   0.00   0.00   0.00   0.00
   3   3.31   3.31   0.55  99.45   0.00   0.00   0.00   0.00   0.00

There's no sign of turbo boost. I have Asrock H55DE3 (bios v2.0) motherboard with most BIOS settings having default values.

Created attachment 25091 [details]
ASRock H55DE3 ACPI dump

Hm,

# ./acpidump > acpidump.txt
Wrong checksum for generic table!

What could that mean?

It's a false alarm, the corresponding BIOS CPU settings were disabled.