Bug 215729 - amd-pstate driver has a much higher idle power consumption for a desktop Zen 3 CPU
Summary: amd-pstate driver has a much higher idle power consumption for a desktop Zen ...
Status: RESOLVED OBSOLETE
Alias: None
Product: Power Management
Classification: Unclassified
Component: cpufreq (show other bugs)
Hardware: x86-64 Linux
: P1 normal
Assignee: linux-pm@vger.kernel.org
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-03-22 23:58 UTC by Artem S. Tashkinov
Modified: 2023-09-15 10:24 UTC (History)
9 users (show)

See Also:
Kernel Version: 5.17
Subsystem:
Regression: No
Bisected commit-id:


Attachments

Description Artem S. Tashkinov 2022-03-22 23:58:03 UTC
I have a Ryzen 7 5800X CPU.

With the acpi-cpufreq driver the CPU idles around 20W, frequencies are in the range of 2.2-2.8GHz.

The very same system with the amd-pstate driver idles around 25W frequencies are in the range of 3.9-4.75GHz.

This doesn't look right.

Changing governor from ondemand to powersave reduces idle power consumption to around 21.5W (still higher than with acpi-cpufreq) but this obvious completely destroys performance.
Comment 1 Artem S. Tashkinov 2022-03-22 23:59:11 UTC
At least under Windows 10 I see effective frequencies around 50MHz (i.e. cores are idling as much as possible), that doesn't seem to be the case with Linux.
Comment 2 Artem S. Tashkinov 2022-03-23 00:01:18 UTC
CC'ing Huang Rui - please check.

I'm not changing any CPU frequency related options - i.e. everything is running by default.
Comment 3 Huang Rui 2022-03-23 02:09:55 UTC
(In reply to Artem S. Tashkinov from comment #0)
> I have a Ryzen 7 5800X CPU.
> 
> With the acpi-cpufreq driver the CPU idles around 20W, frequencies are in
> the range of 2.2-2.8GHz.
> 
> The very same system with the amd-pstate driver idles around 25W frequencies
> are in the range of 3.9-4.75GHz.
> 
> This doesn't look right.
> 
> Changing governor from ondemand to powersave reduces idle power consumption
> to around 21.5W (still higher than with acpi-cpufreq) but this obvious
> completely destroys performance.

Thanks to report the issue. Could you please dump the print of "lscpu" and use the latest bleeding-edge source code include the CPUPower support (https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git/log/?h=bleeding-edge) to print "cpupower frequency-info"?

Thanks,
Ray
Comment 4 Artem S. Tashkinov 2022-03-23 15:35:54 UTC
With acpi-cpufreq:

# lscpu

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         48 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  16
  On-line CPU(s) list:   0-15
Vendor ID:               AuthenticAMD
  BIOS Vendor ID:        Advanced Micro Devices, Inc.
  Model name:            AMD Ryzen 7 5800X 8-Core Processor
    BIOS Model name:     AMD Ryzen 7 5800X 8-Core Processor             
    CPU family:          25
    Model:               33
    Thread(s) per core:  2
    Core(s) per socket:  8
    Socket(s):           1
    Stepping:            0
    Frequency boost:     enabled
    CPU max MHz:         4928.1250
    CPU min MHz:         2200.0000
    BogoMIPS:            7600.48
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmul
                         qdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l
                         lc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total 
                         cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip pku vaes vpclmulqdq rdpid ov
                         erflow_recov succor smca fsrm
Virtualization features: 
  Virtualization:        AMD-V
Caches (sum of all):     
  L1d:                   256 KiB (8 instances)
  L1i:                   256 KiB (8 instances)
  L2:                    4 MiB (8 instances)
  L3:                    32 MiB (1 instance)
Vulnerabilities:         
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP always-on, RSB filling
  Srbds:                 Not affected
  Tsx async abort:       Not affected

# ./cpupower frequency-info
analyzing CPU 0:
  driver: acpi-cpufreq
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency:  Cannot determine or is not supported.
  hardware limits: 2.20 GHz - 4.93 GHz
  available frequency steps:  3.80 GHz, 2.80 GHz, 2.20 GHz
  available cpufreq governors: conservative ondemand userspace powersave performance schedutil
  current policy: frequency should be within 2.20 GHz and 3.80 GHz.
                  The governor "ondemand" may decide which speed to use
                  within this range.
  current CPU frequency: 2.20 GHz (asserted by call to hardware)
  boost state support:
    Supported: yes
    Active: yes
    Boost States: 0
    Total States: 3
    Pstate-P0:  3800MHz
    Pstate-P1:  2800MHz
    Pstate-P2:  2200MHz

Will post the results of `cpupower frequency-info` under amd-pstate a tad later.
Comment 5 Artem S. Tashkinov 2022-03-23 15:41:22 UTC
Under amd-pstate:

# lscpu

    Frequency boost:     enabled
    CPU max MHz:         4929.0000
    CPU min MHz:         550.0000
    BogoMIPS:            7599.66

(everything else is obviously the same)

# ./cpupower frequency-info
analyzing CPU 0:
  driver: amd-pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 131 us
  hardware limits: 550 MHz - 4.93 GHz
  available cpufreq governors: conservative ondemand userspace powersave performance schedutil
  current policy: frequency should be within 550 MHz and 4.93 GHz.
                  The governor "ondemand" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 903 MHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: yes
    AMD PSTATE Highest Performance: 166. Maximum Frequency: 4.93 GHz.
    AMD PSTATE Nominal Performance: 128. Nominal Frequency: 3.80 GHz.
    AMD PSTATE Lowest Non-linear Performance: 59. Lowest Non-linear Frequency: 1.75 GHz.
    AMD PSTATE Lowest Performance: 19. Lowest Frequency: 550 MHz.
Comment 6 Artem S. Tashkinov 2022-03-23 15:58:30 UTC
After rebooting again with amd-pstate the situation has kinda improved:

Max CPU frequency in idle is now around 1.2GHz with occasional spikes to 2.2GHz.

Power consumption is now around 23W - not a big difference with acpi-cpufreq but I was under the impression that amd-pstate should be more power efficient, not less.

'Minimum' governor is now at around ~20W in idle which matches acpi-cpufreq.

----

Here's the most important thing:

The ondemand governor in idle makes the CPU consume around 23.5W

The performance governor in idle makes the CPU consume around 20.5W which is near perfect.

So, it looks like the ondemand governor falters with amd-pstate for some reasons. Maybe it does more work than necessary, maybe it gets called far too often, I've no idea.
Comment 7 Artem S. Tashkinov 2022-03-23 16:04:11 UTC
The conservative governor idles around 21W which is good but this governor is very undesirable for a desktop system: it's too slow to react to instantaneous spikes in workload and it makes e.g. web browsing far from comfortable.

So, with amd-pstate in idle:

Powersave and performance (!) governors ~ 20W.
Conservative ~ 21W.
Ondemand ~ 23.5W.
Comment 8 Joe 2022-04-28 08:49:40 UTC
Hello,

I reproduce this issue on my local.  

Your device (CPU family:  25, Model:  33) should be shared memory solution with AMD P-state driver. Updating CPPC request with shared memory will need more energy  than writing MSR register directly. We are checking this issue now.

For ondemand governor, could you please try to reduce the sampling_rate and then check to energy consumption again? Thanks a lot!

Here is the command 

cd /sys/devices/system/cpu/cpufreq/ondemand
echo 50000 > sampling_rate

Thanks
Joe
Comment 9 Artem S. Tashkinov 2022-05-09 08:38:00 UTC
(In reply to Joe from comment #8)
> Hello,
> 
> I reproduce this issue on my local.  
> 
> Your device (CPU family:  25, Model:  33) should be shared memory solution
> with AMD P-state driver. Updating CPPC request with shared memory will need
> more energy  than writing MSR register directly. We are checking this issue
> now.
> 
> For ondemand governor, could you please try to reduce the sampling_rate and
> then check to energy consumption again? Thanks a lot!
> 
> Here is the command 
> 
> cd /sys/devices/system/cpu/cpufreq/ondemand
> echo 50000 > sampling_rate
> 
> Thanks
> Joe

Changing sampling_rate from default 2000 to 50000 has improved the situation considerably but still acpi-cpufreq has a tiny better idle power consumption.

acpi-cpufreq: ~20W
amd-pstate with default ondemand settings: ~23W
amd-pstate with ondemand/sampling_rate=50k: ~21W

Looks like the ondemand scheduler is not super efficient with the amd-pstate driver.
Comment 10 Artem S. Tashkinov 2023-07-07 13:24:20 UTC
Kernel 6.3.12, default settings:

acpi-cpufreq: ~19.4W
amd-pstate: ~20.7W

Much better but not perfect. I can live with that but I still expected the amd-pstate driver to be more effective considering its using native interfaces.



Not using any boot flags except: amd_pstate=active

/sys/devices/system/cpu/cpufreq/policy0 $ grep -r . 
energy_performance_available_preferences:default performance balance_performance balance_power power 
amd_pstate_highest_perf:166
scaling_min_freq:550000
scaling_available_governors:performance powersave
scaling_governor:powersave
cpuinfo_max_freq:4929000
amd_pstate_lowest_nonlinear_freq:1752000
amd_pstate_max_freq:4929000
related_cpus:0
scaling_cur_freq:550000
scaling_setspeed:<unsupported>
affected_cpus:0
scaling_max_freq:4929000
cpuinfo_transition_latency:0
energy_performance_preference:performance
scaling_driver:amd-pstate-epp
cpuinfo_min_freq:550000
Comment 11 Artem S. Tashkinov 2023-09-15 10:24:19 UTC
~1W is nothing to worry about, let's close this bug report.

For mobile CPUs there's seemingly no difference at all.

Note You need to log in before you can comment on or make changes to this bug.