Bug 215938 - amd-pstate ignoring scaling_max_freq after waking from suspend
Summary: amd-pstate ignoring scaling_max_freq after waking from suspend
Status: RESOLVED CODE_FIX
Alias: None
Product: Power Management
Classification: Unclassified
Component: cpufreq (show other bugs)
Hardware: AMD Linux
: P1 normal
Assignee: linux-pm@vger.kernel.org
URL:
Keywords:
: 216189 (view as bug list)
Depends on:
Blocks:
 
Reported: 2022-05-03 15:09 UTC by Alex Maras
Modified: 2022-07-07 22:36 UTC (History)
5 users (show)

See Also:
Kernel Version: 5.17.5
Subsystem:
Regression: No
Bisected commit-id:


Attachments
dmesg output after sleeping (103.14 KB, text/plain)
2022-05-20 09:55 UTC, Alex Maras
Details
cpuinfo (23.80 KB, text/plain)
2022-05-20 09:56 UTC, Alex Maras
Details
dmesg output after sleeping - with debug params (3.78 MB, text/plain)
2022-05-31 07:32 UTC, Alex Maras
Details
dmesg output - 5.18 (3.31 MB, text/plain)
2022-06-09 14:59 UTC, Alex Maras
Details

Description Alex Maras 2022-05-03 15:09:29 UTC
Using the amd-pstate driver for cpu scaling, I'm only able to enforce a maximum clock speed when my laptop has not slept. I do the following:

echo 'ondemand' | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

echo '1400000' | sudo tee /sys/devices/system/cpu/cpufreq/policy*/scaling_max_freq

cat /sys/devices/system/cpu/cpufreq/policy*/scaling_cur_freq

At this point, all reported frequencies for all cores are below 1.4ghz. If I then suspend the laptop, then wake it up, then checking the core frequencies again will show that it is behaving as if the ondemand governor was not set. This behavior will persist until a reboot, even if the governor is switched to another mode and then back, or if the scaling_max_freq value is changed. All settings will be ignored. 

I have replicated this behavior on another identical model of laptop (Lenovo IdeaPad 5 14ARE05), 

cpupower frequency-info also reports that the current CPU frequency is set to 1.4GHZ and that the governor is set to ondemand.
Comment 1 Artem S. Tashkinov 2022-05-04 06:16:23 UTC
Could you please check if bug 215800 affects you?
Comment 2 Alex Maras 2022-05-04 13:46:04 UTC
(In reply to Artem S. Tashkinov from comment #1)
> Could you please check if bug 215800 affects you?

No, I can set my frequency to 3GHZ and run stress and my scaling_cur_freq value reports ~3GHZ. I'm also on a AMD Ryzen 7 4800U, which is a Zen2 model.
Comment 3 perry_yuan 2022-05-20 08:37:19 UTC
(In reply to Alex Maras from comment #2)
> (In reply to Artem S. Tashkinov from comment #1)
> > Could you please check if bug 215800 affects you?
> 
> No, I can set my frequency to 3GHZ and run stress and my scaling_cur_freq
> value reports ~3GHZ. I'm also on a AMD Ryzen 7 4800U, which is a Zen2 model.

Hi Alex.
Could you help to attach the dmesg log and all CPU model info ?
Meanwhile I will check if I can reproduce the issue on my side if I have the any similar CPU model.

Perry.
Comment 4 Alex Maras 2022-05-20 09:55:03 UTC
Created attachment 301003 [details]
dmesg output after sleeping

1. Started up laptop fresh
2. Set governor to ondemand and frequency to 1.4GHZ
3. Checked CPU frequency, settings respected
4. Shut laptop lid, waited 10 seconds, opened laptop lid
5. Checked CPU frequency, settings disregarded
6. Grabbing dmesg output
Comment 5 Alex Maras 2022-05-20 09:56:46 UTC
Created attachment 301004 [details]
cpuinfo

CPUInfo output
Comment 6 Alex Maras 2022-05-20 09:57:35 UTC
Hey Perry,

I've attached the /proc/cpuinfo output and dmesg output after the listed process. Do you want anything more for the moment? Let me know if you need anything, happy to help.

Alex
Comment 7 perry_yuan 2022-05-31 02:55:41 UTC
(In reply to Alex Maras from comment #6)
> Hey Perry,
> 
> I've attached the /proc/cpuinfo output and dmesg output after the listed
> process. Do you want anything more for the moment? Let me know if you need
> anything, happy to help.
> 
> Alex

Thanks for your info.
Could you try this kernel ?

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/drivers/cpufreq/amd-pstate.c?h=v5.18

add this to grub.

amd-pstate.dyndbg=+p  acpi-cpufreq.dyndbg=+p freq_table.dyndbg=+p cpufreq.dyndbg=+p cpufreq_governor.dyndbg=+p

I didn't find amd pstate driver related log in your last dmesg.
more debug logs will be helpful.

Thanks.
Perry.
Comment 8 Alex Maras 2022-05-31 07:32:05 UTC
Created attachment 301080 [details]
dmesg output after sleeping - with debug params

I've attached the dmesg output here after applying those kernel params. My kernel version has the latest commit. I'll compile from git to ensure the results are the same in the meantime. 

The dmesg results are interesting - cat-ing scaling_cur_freq still shows the same behavior, but dmesg doesn't show any values above 1.4gHz. I ran `stress` either side of sleeping and watched the scaling_cur_freq, and can confirm that it was maxing out at 1.4gHz before suspend, and not limited after suspend.
Comment 9 Alex Maras 2022-06-09 14:59:11 UTC
Created attachment 301137 [details]
dmesg output - 5.18

same process with dmesg output with kernel built from 5.18 release tag
Comment 10 Alex Maras 2022-06-09 15:00:53 UTC
Note - in the most recent test, I was limiting to 1gHz instead of 1.4 - results are the same, successfully stating at 1gHz prior to sleeping, unbound after waking from sleep.
Comment 11 Perry_Yuan 2022-06-09 15:15:04 UTC
(In reply to Alex Maras from comment #10)
> Note - in the most recent test, I was limiting to 1gHz instead of 1.4 -
> results are the same, successfully stating at 1gHz prior to sleeping,
> unbound after waking from sleep.

Hi Alex.
Thanks for your detail and info share, I will check the logs soon. then will give you some feedback.
Comment 12 perry_yuan 2022-06-10 17:03:36 UTC
(In reply to Alex Maras from comment #8)
> Created attachment 301080 [details]
> dmesg output after sleeping - with debug params
> 
> I've attached the dmesg output here after applying those kernel params. My
> kernel version has the latest commit. I'll compile from git to ensure the
> results are the same in the meantime. 
> 
> The dmesg results are interesting - cat-ing scaling_cur_freq still shows the
> same behavior, but dmesg doesn't show any values above 1.4gHz. I ran
> `stress` either side of sleeping and watched the scaling_cur_freq, and can
> confirm that it was maxing out at 1.4gHz before suspend, and not limited
> after suspend.

Hi Alex.
Could you help to try if you can change the cores frequency when you switch to userspace governor after suspend/resume test?


 # sudo cpupower frequency-set -g userspace
 # systemctl suspend 
 Resume from Suspend pressing power button or keyboards.
 # cd /sys/devices/system/cpu/cpufreq/policy0
 # echo 1752589 > scaling_setspeed  //change this core freq to see if it can be changed 
 # cat scaling_setspeed
 # cat scaling_governor
 # cat scaling_cur_freq


I would like to see if the cpu cores still can change freq or not after resuming.

If the cpu freq cannot be changed, even you change the cpuinfo_max_freq, core freq(scaling_cur_freq) will not be limited  successfully as well. 


Perry.
Comment 13 Alex Maras 2022-06-11 03:33:18 UTC
Hey Perry, results below:

[I] alex@alex-laptop ~> sudo cpupower frequency-set -g userspace
[sudo] password for alex:
Setting cpu: 0
Setting cpu: 1
Setting cpu: 2
Setting cpu: 3
Setting cpu: 4
Setting cpu: 5
Setting cpu: 6
Setting cpu: 7
Setting cpu: 8
Setting cpu: 9
Setting cpu: 10
Setting cpu: 11
Setting cpu: 12
Setting cpu: 13
Setting cpu: 14
Setting cpu: 15
[I] alex@alex-laptop ~> systemctl suspend
[I] alex@alex-laptop ~> cd /sys/devices/system/cpu/cpufreq/policy0
[I] alex@alex-laptop /s/d/s/c/c/policy0> echo 1752589 | sudo tee scaling_setspeed
1752589
[I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_setspeed
1752589
[I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_governor
userspace
[I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_cur_freq
2796520
[I] alex@alex-laptop /s/d/s/c/c/policy0>


Looks like the same issue continuing. Do you want the same dmesg with pstate logging turned on?
Comment 14 perry_yuan 2022-06-11 04:11:10 UTC
(In reply to Alex Maras from comment #13)
> Hey Perry, results below:
> 
> [I] alex@alex-laptop ~> sudo cpupower frequency-set -g userspace
> [sudo] password for alex:
> Setting cpu: 0
> Setting cpu: 1
> Setting cpu: 2
> Setting cpu: 3
> Setting cpu: 4
> Setting cpu: 5
> Setting cpu: 6
> Setting cpu: 7
> Setting cpu: 8
> Setting cpu: 9
> Setting cpu: 10
> Setting cpu: 11
> Setting cpu: 12
> Setting cpu: 13
> Setting cpu: 14
> Setting cpu: 15
> [I] alex@alex-laptop ~> systemctl suspend
> [I] alex@alex-laptop ~> cd /sys/devices/system/cpu/cpufreq/policy0
> [I] alex@alex-laptop /s/d/s/c/c/policy0> echo 1752589 | sudo tee
> scaling_setspeed
> 1752589
> [I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_setspeed
> 1752589
> [I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_governor
> userspace
> [I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_cur_freq
> 2796520
> [I] alex@alex-laptop /s/d/s/c/c/policy0>
> 
> 
> Looks like the same issue continuing. Do you want the same dmesg with pstate
> logging turned on?

Seeing from you result, issue can be reproduced by my side, the core frequency is not updated by low level firmware.
I suppose that the issue was caused by suspend/resume , firmware has problem to resume the CPU frequency control, so any new frequency set will be ignored.
I will further do some debugging and try to resolve the problem.
Thanks for your confirm.

Perry.
Comment 15 perry_yuan 2022-06-13 16:25:19 UTC
Hi Alex.

I have been preparing some fix patches to resolve the prolem.
Will update here after patches sent.

Perry.
Comment 16 a 2022-06-29 15:35:35 UTC
*** Bug 216189 has been marked as a duplicate of this bug. ***
Comment 17 perry_yuan 2022-07-05 01:16:22 UTC
(In reply to Alex Maras from comment #13)
> Hey Perry, results below:
> 
> [I] alex@alex-laptop ~> sudo cpupower frequency-set -g userspace
> [sudo] password for alex:
> Setting cpu: 0
> Setting cpu: 1
> Setting cpu: 2
> Setting cpu: 3
> Setting cpu: 4
> Setting cpu: 5
> Setting cpu: 6
> Setting cpu: 7
> Setting cpu: 8
> Setting cpu: 9
> Setting cpu: 10
> Setting cpu: 11
> Setting cpu: 12
> Setting cpu: 13
> Setting cpu: 14
> Setting cpu: 15
> [I] alex@alex-laptop ~> systemctl suspend
> [I] alex@alex-laptop ~> cd /sys/devices/system/cpu/cpufreq/policy0
> [I] alex@alex-laptop /s/d/s/c/c/policy0> echo 1752589 | sudo tee
> scaling_setspeed
> 1752589
> [I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_setspeed
> 1752589
> [I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_governor
> userspace
> [I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_cur_freq
> 2796520
> [I] alex@alex-laptop /s/d/s/c/c/policy0>
> 
> 
> Looks like the same issue continuing. Do you want the same dmesg with pstate
> logging turned on?

The suspend issue has been fixed by below patch.
Please try the below patch for the suspend/resume issue.

https://patchwork.kernel.org/project/linux-pm/patch/20220623031509.555269-1-Jinzhou.Su@amd.com/
Comment 18 Alex Maras 2022-07-05 07:36:19 UTC
Thanks Perry - tested and working as expected. I'm now able to set the CPU freq with amd-pstate, suspend, resume and see that the cpu frequency limits are now respected correctly, and changing both the governor and the frequency limits works as expected. 

I've set this to resolved with a code fix.
Comment 19 perry_yuan 2022-07-05 09:28:57 UTC
(In reply to Alex Maras from comment #18)
> Thanks Perry - tested and working as expected. I'm now able to set the CPU
> freq with amd-pstate, suspend, resume and see that the cpu frequency limits
> are now respected correctly, and changing both the governor and the
> frequency limits works as expected. 
> 
> I've set this to resolved with a code fix.

HI Alex.

Good to hear that. 
Thanks for your confirm. 

Perry.
Comment 20 a 2022-07-06 06:00:40 UTC
Guys, have you get amd_pstate working with schedutil governor?  
On my system, it completely ignores frequency limits and boost state. 

```
~  sudo cpupower frequency-set -g schedutil -d 1200MHz -u 1600MHz                                                                                            
~  sudo cpupower frequency-info                                                                                                                              
analyzing CPU 0:
  driver: amd-pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 131 us
  hardware limits: 400 MHz - 1.90 GHz
  available cpufreq governors: ondemand performance schedutil
  current policy: frequency should be within 1.20 GHz and 1.60 GHz.
                  The governor "schedutil" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 2.60 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: no
    AMD PSTATE Highest Performance: 166. Maximum Frequency: 4.51 GHz.
    AMD PSTATE Nominal Performance: 70. Nominal Frequency: 1.90 GHz.
    AMD PSTATE Lowest Non-linear Performance: 41. Lowest Non-linear Frequency: 1.11 GHz.
    AMD PSTATE Lowest Performance: 15. Lowest Frequency: 400 MHz.
```

Other governors (ondemand/performance) works as intended.  
Is it a bug or something else?
Comment 21 regulator 2022-07-07 22:36:50 UTC
(In reply to a from comment #20)
> Guys, have you get amd_pstate working with schedutil governor?  
> On my system, it completely ignores frequency limits and boost state. 

On my 5800HS using amd-pstate, the lowest clock speed schedutil goes to is 2500mhz compared to other governors which go down to 400mhz, we should probably log a separate bug report? This is any time not just when waking from suspend

Note You need to log in before you can comment on or make changes to this bug.