Bug 15157 - turbostat shows No Deep C-states, No deep P-states, limited Turbo mode - Supermicro X8DTN
Summary: turbostat shows No Deep C-states, No deep P-states, limited Turbo mode - Supe...
Status: CLOSED DOCUMENTED
Alias: None
Product: ACPI
Classification: Unclassified
Component: BIOS (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: acpi_bios
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-01-28 05:48 UTC by Len Brown
Modified: 2010-01-28 05:57 UTC (History)
1 user (show)

See Also:
Kernel Version: 2.6.31.6
Subsystem:
Regression: No
Bisected commit-id:


Attachments

Description Len Brown 2010-01-28 05:48:57 UTC
Dual-Xeon Nehalem EP system from supermicro, sitting idle.
BIOS version 10/06/2009

In this example, booted with "maxcpus=1"

Powertop says it is almost all in C3 mwait, and at 1.60 GHz:

Cn	          Avg residency
C0 (cpu running)        ( 0.0%)
polling		 76.9ms ( 6.7%)
C1 halt		  0.2ms ( 0.0%)
C2		  0.0ms ( 0.0%)
C3 mwait	 70.3ms (93.3%)
P-states (frequencies)
  2.80 Ghz     0.3%
  2.80 Ghz     0.0%
  2.67 Ghz     0.0%
  2.53 Ghz     0.0%
  1.60 Ghz    99.7%

However, turbostat
http://ftp.kernel.org/pub/linux/kernel/people/lenb/acpi/utils/pmtools-latest/turbostat/

shows the system is running at 2.8 GHz and never goes deeper than c1:

# ./turbostat -v
CPUID GenuineIntel 11 levels family:model:stepping 6:26:5
Nehalem multiplier 21, TSC frequency 2800 MHz
Nehalem 4 cores active: 23 mult, max turbo frequency = 3067 MHz
Nehalem 3 cores active: 23 mult, max turbo frequency = 3067 MHz
Nehalem 2 cores active: 24 mult, max turbo frequency = 3200 MHz
Nehalem 1 core active: 24 mult, max turbo frequency = 3200 MHz
 CPU   GHz    TSC    %c0    %c1    %c3    %c6   %pc3   %pc6   %pc7 
   0   2.80   2.80   0.09  99.91   0.00   0.00   0.00   0.00   0.00

Further, when a cycle soaker is executed:

# cat /dev/zero > /dev/null

the turbo frequency tops out at 3.07 GHz, when we would hope for 3.2 Ghz:

# ./turbostat
 CPU   GHz    TSC    %c0    %c1    %c3    %c6   %pc3   %pc6   %pc7 
   0   3.07   2.80 100.00   0.00   0.00   0.00   0.00   0.00   0.00
Comment 1 Len Brown 2010-01-28 05:56:51 UTC
booting without maxcpus=1, this problem goes away.

All the cores are able to enter c6 and get their frequency down to 1.6GHz
for the short time they are in c0:
 
CPU   GHz    TSC    %c0    %c1    %c3    %c6   %pc3   %pc6   %pc7 
   0   1.60   2.80   0.16   0.23   0.00  99.61   0.00  98.95   0.00
   1   1.60   2.80   0.08   0.09   0.00  99.83   0.00  98.95   0.00
   2   1.60   2.80   0.01   0.02   0.00  99.96   0.00  98.95   0.00
   3   1.63   2.80   0.01   0.03   0.00  99.96   0.00  98.95   0.00
   4   1.60   2.80   0.01   0.02   0.00  99.98   0.00  98.96   0.00
   5   1.60   2.80   0.06   0.13   0.00  99.81   0.00  98.96   0.00
   6   1.58   2.80   0.01   0.02   0.00  99.97   0.00  98.96   0.00
   7   1.60   2.80   0.01   0.02   0.00  99.98   0.00  98.96   0.00
   8   1.60   2.80   0.04   0.35   0.00  99.61   0.00  98.95   0.00
   9   1.60   2.80   0.02   0.15   0.00  99.83   0.00  98.95   0.00
  10   1.60   2.80   0.01   0.03   0.00  99.96   0.00  98.95   0.00
  11   1.59   2.80   0.01   0.03   0.00  99.96   0.00  98.95   0.00
  12   1.57   2.80   0.01   0.02   0.00  99.98   0.00  98.96   0.00
  13   1.60   2.80   0.04   0.15   0.00  99.81   0.00  98.96   0.00
  14   1.63   2.80   0.01   0.02   0.00  99.97   0.00  98.96   0.00
  15   1.60   2.80   0.01   0.01   0.00  99.98   0.00  98.96   0.00

When a cycle soaker is run, it is able to turbo-up to 3.2GHz as expected:

 CPU   GHz    TSC    %c0    %c1    %c3    %c6   %pc3   %pc6   %pc7 
   0   3.20   2.80   1.91   6.24   3.63  88.22   0.00   0.00   0.00
   1   3.19   2.80   0.05  99.95   0.00   0.00   0.00   0.00   0.00
   2   2.90   2.80   0.02   0.09  84.97  14.92   0.00   0.00   0.00
   3   2.49   2.80   0.01   0.09  84.98  14.92   0.00   0.00   0.00
   4   1.60   2.80   0.14   0.21   0.00  99.65   0.00   0.00   0.00
   5   1.60   2.80   0.05   0.10   0.00  99.85   0.00   0.00   0.00
   6   1.63   2.80   0.01   0.02   0.00  99.98   0.00   0.00   0.00
   7   1.60   2.80   0.01   0.02   0.00  99.98   0.00   0.00   0.00
   8   3.20   2.80   2.24   5.91   3.63  88.22   0.00   0.00   0.00
   9   3.20   2.80  99.99   0.01   0.00   0.00   0.00   0.00   0.00
  10   2.53   2.80   0.01   0.10  84.98  14.91   0.00   0.00   0.00
  11   2.51   2.80   0.01   0.09  84.98  14.92   0.00   0.00   0.00
  12   1.60   2.80   0.07   0.29   0.00  99.65   0.00   0.00   0.00
  13   1.60   2.80   0.04   0.10   0.00  99.85   0.00   0.00   0.00
  14   1.60   2.80   0.01   0.02   0.00  99.98   0.00   0.00   0.00
  15   1.60   2.80   0.01   0.01   0.00  99.98   0.00   0.00   0.00

The "maxcpus=1" power-management failure is due to a BIOS bug.
The BIOS is supposed to put non-boot CPUSs into c6
so that if the OS does not start them, they will not interfere with
power management.  However, this BIOS is clearly leaving the non-boot
CPUs in c1, which interferes with the power-management of the online cpus.

This same symptom is seen when a kernel built for MAX_CPUS=8
is run on a 16-thread system (which is how I first noticed this problem).

Note You need to log in before you can comment on or make changes to this bug.