Bug 218705

Summary: amd_pstate fails to load on AMD 5950x with Asus ROG CROSSHAIR VIII DARK HERO x570
Product: Power Management Reporter: Andrei Amuraritei (andamu)
Component: cpufreqAssignee: linux-pm (linux-pm)
Status: RESOLVED CODE_FIX    
Severity: normal CC: nielsenb, Perry.Yuan, xiaojian.du
Priority: P3    
Hardware: AMD   
OS: Linux   
Kernel Version: Subsystem:
Regression: No Bisected commit-id:
Attachments: cppc check script
values from cppc_attr_values_check.sh script
cpufreq_grep
dmesg with amd_pstate.dyndbg
lscpu -ae
cpufreq_grep with amd_pstate=active
dmesg with amd_pstate.dyndbg and amd_pstate=active
lscpu -ae with amd_pstate=active
lscpu -ae with amd_pstate=passive
dmesg with amd_pstate.dyndbg and amd_pstate=passive
cpufreq_grep with amd_pstate=passive

Description Andrei Amuraritei 2024-04-10 16:52:17 UTC
Hi!

Sorry if the formatting is broken or this is unreadable. Description preview is broken it seems.

amd_pstate fails to load on this hardware:
CPU: AMD 5950x
Mainboard: Asus ROG CROSSHAIR VIII DARK HERO x570 running UEFI version 4702
OS: Fedora 39 Wayland with kernel Linux 6.8.4-200.fc39.x86_64

amd_pstate does not get automatically enabled. dmesg -T | grep pstate says: 

[Tue Apr  9 17:24:36 2024] amd_pstate: driver load is disabled, boot with specific mode to enable this

No cppc found in output from lscpu. In UEFI the following are activated:

Advanced > AMD CBS > NBIO Common Options > SMU Common Options

CPPC (Auto / Enabled / Disabled): Enabled
CPPC Preferred Cores: Enabled
Comment 1 Andrei Amuraritei 2024-04-10 16:54:46 UTC
Reading though 218171, I have checked if my ACPI tables have the _CPC entries and they do:

ssdt10.dsl:        Name (_CPC, Package (0x17)  // _CPC: Continuous Performance Control
Comment 2 Brandon Nielsen 2024-04-15 16:04:39 UTC
Looking at the source[0] I think this is by design.

I am not sure what the current roadmap for enablement looks like, but I am also awaiting it.

[0] - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/cpufreq/amd-pstate.c#n55
Comment 3 Perry Yuan(AMD) 2024-04-22 08:04:46 UTC
Created attachment 306192 [details]
cppc check script
Comment 4 Perry Yuan(AMD) 2024-04-22 08:06:31 UTC
(In reply to Andrei Amuraritei from comment #0)
> Hi!
> 
> Sorry if the formatting is broken or this is unreadable. Description preview
> is broken it seems.
> 
> amd_pstate fails to load on this hardware:
> CPU: AMD 5950x
> Mainboard: Asus ROG CROSSHAIR VIII DARK HERO x570 running UEFI version 4702
> OS: Fedora 39 Wayland with kernel Linux 6.8.4-200.fc39.x86_64
> 
> amd_pstate does not get automatically enabled. dmesg -T | grep pstate says: 
> 
> [Tue Apr  9 17:24:36 2024] amd_pstate: driver load is disabled, boot with
> specific mode to enable this
> 
> No cppc found in output from lscpu. In UEFI the following are activated:
> 
> Advanced > AMD CBS > NBIO Common Options > SMU Common Options
> 
> CPPC (Auto / Enabled / Disabled): Enabled
> CPPC Preferred Cores: Enabled

Hi Andrei
Could you use the script to check the CPPC capabilities?
As I know, the 5950x is in the CPPC supported list, I need more information to check what the system is missing.

Perry.
Comment 5 Andrei Amuraritei 2024-04-22 11:07:07 UTC
Created attachment 306195 [details]
values from cppc_attr_values_check.sh script

Hi Perry,

I've attached 20240422-cppc_check.txt with the output from the script.

pstate did not load automatically for me on this system as long as I remember (across Debian or Fedora with kernels 6.x).

Let me know what other info you require please.

Thanks,
Andrei Amuraritei
Comment 6 Perry Yuan(AMD) 2024-04-25 03:36:51 UTC
I have confirmed that the CPPC capabilities are valid to enable pstate driver,

1) sudo nano /etc/default/grub
2) add ``amd_pstate.dyndbg=+p`` to kernel command line.
3) sudo update-grub
4) sudo reboot

then share below logs to me.

# dmesg
# lscpu -ae
# cd /sys/devices/system/cpu/cpu0/cpufreq
# grep -R .

Perry.
Comment 7 Perry Yuan(AMD) 2024-04-25 03:36:52 UTC
I have confirmed that the CPPC capabilities are valid to enable pstate driver,

1) sudo nano /etc/default/grub
2) add ``amd_pstate.dyndbg=+p`` to kernel command line.
3) sudo update-grub
4) sudo reboot

then share below logs to me.

# dmesg
# lscpu -ae
# cd /sys/devices/system/cpu/cpu0/cpufreq
# grep -R .

Perry.
Comment 8 Perry Yuan(AMD) 2024-04-25 05:27:17 UTC
BTW, which kernel version you used?
Have you added "amd_pstate=active" to command line?
Comment 9 Andrei Amuraritei 2024-04-25 15:25:27 UTC
Created attachment 306207 [details]
cpufreq_grep
Comment 10 Andrei Amuraritei 2024-04-25 15:26:13 UTC
Created attachment 306208 [details]
dmesg with amd_pstate.dyndbg
Comment 11 Andrei Amuraritei 2024-04-25 15:26:37 UTC
Created attachment 306209 [details]
lscpu -ae
Comment 12 Andrei Amuraritei 2024-04-25 15:27:07 UTC
Created attachment 306210 [details]
cpufreq_grep with amd_pstate=active
Comment 13 Andrei Amuraritei 2024-04-25 15:27:36 UTC
Created attachment 306211 [details]
dmesg with amd_pstate.dyndbg and amd_pstate=active
Comment 14 Andrei Amuraritei 2024-04-25 15:27:54 UTC
Created attachment 306212 [details]
lscpu -ae with amd_pstate=active
Comment 15 Andrei Amuraritei 2024-04-25 15:33:16 UTC
Hi Perry,

Attached the required output of the commands specified. First files are after adding amd_pstate.dyndbg=+p but there amd_pstate still did not load automatically.

Then I also added amd_pstate=active to the kernel param list and attached the outputs from commands, with "active" in names.

Kernel when opened the bug was 6.8.4-200.fc39.x86_64 and right now it is 6.8.6-200.fc39.x86_64 as specified in dmesg output.

"Talk" soon.
Andrei Amuraritei
Comment 16 xiaojian.du 2024-04-25 17:51:54 UTC
From the dmesg log:
......
[Do Apr 25 17:18:53 2024] device-mapper: uevent: version 1.0.3
[Do Apr 25 17:18:53 2024] device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: dm-devel@redhat.com
[Do Apr 25 17:18:53 2024] amd_pstate: AMD CPPC **shared memory** based functionality is supported
[Do Apr 25 17:18:53 2024] amd_pstate: policy_max =5084000, policy_min=550000
[Do Apr 25 17:18:53 2024] amd_pstate: set_policy: cpuinfo.max 5084000 policy->max 5084000
[Do Apr 25 17:18:53 2024] amd_pstate: policy_max =5084000, policy_min=550000
[Do Apr 25 17:18:53 2024] amd_pstate: set_policy: cpuinfo.max 5084000 policy->max 5084000
[Do Apr 25 17:18:53 2024] amd_pstate: policy_max =5084000, policy_min=550000
[Do Apr 25 17:18:53 2024] amd_pstate: set_policy: cpuinfo.max 5084000 policy->max
......

R9-5950X is ZEN3 Arch CPU and has half support for CPPC.
Would you try to use "amd_pstate=passive" in the grub config?
Comment 17 Andrei Amuraritei 2024-04-26 17:24:52 UTC
Created attachment 306222 [details]
lscpu -ae with amd_pstate=passive
Comment 18 Andrei Amuraritei 2024-04-26 17:25:14 UTC
Created attachment 306223 [details]
dmesg with amd_pstate.dyndbg and amd_pstate=passive
Comment 19 Andrei Amuraritei 2024-04-26 17:25:35 UTC
Created attachment 306224 [details]
cpufreq_grep with amd_pstate=passive
Comment 20 Andrei Amuraritei 2024-04-26 17:27:22 UTC
Hi Xiaojian,

Booted with amd_passive and ran the commands. Output attached in files with passive in name.

Friendly greetings
Andrei Amuraritei
Comment 21 Andrei Amuraritei 2024-04-26 17:40:58 UTC
Not sure if relevant, but I found out that Fedora also has amd-pstate-ut module, and loading this, i see the following in dmesg:


boot kernel with amd_pstate=passive

[Fr Apr 26 19:37:29 2024] amd_pstate_ut: 1    amd_pstate_ut_acpi_cpc_valid	 success!
[Fr Apr 26 19:37:29 2024] amd_pstate_ut: 2    amd_pstate_ut_check_enabled	 success!
[Fr Apr 26 19:37:29 2024] amd_pstate_ut: amd_pstate_ut_check_perf cpu0 highest=221 166 nominal=111 111 lowest_nonlinear=57 57 lowest=19 19, they should be equal!
[Fr Apr 26 19:37:29 2024] amd_pstate_ut: 3    amd_pstate_ut_check_perf	 fail!
[Fr Apr 26 19:37:29 2024] amd_pstate_ut: 4    amd_pstate_ut_check_freq	 success!

boot kernel with amd_pstate=active

[Fr Apr 26 19:35:07 2024] amd_pstate_ut: 1    amd_pstate_ut_acpi_cpc_valid	 success!
[Fr Apr 26 19:35:07 2024] amd_pstate_ut: 2    amd_pstate_ut_check_enabled	 success!
[Fr Apr 26 19:35:07 2024] amd_pstate_ut: amd_pstate_ut_check_perf cpu0 highest=221 166 nominal=111 111 lowest_nonlinear=57 57 lowest=19 19, they should be equal!
[Fr Apr 26 19:35:07 2024] amd_pstate_ut: 3    amd_pstate_ut_check_perf	 fail!
[Fr Apr 26 19:35:07 2024] amd_pstate_ut: 4    amd_pstate_ut_check_freq	 success!
Comment 22 Perry Yuan(AMD) 2024-04-27 00:49:56 UTC
Hi Andrei,

I saw the pstate passive mode driver was loaded on your system, how do you check the pstate dricer loaded or not?

# sudo  cpupower frequency-info
you could use cpupower to see if the driver loaded gernerally. 


[Fr Apr 26 19:20:02 2024] amd_pstate: AMD CPPC shared memory based functionality is supported.


root@fedora:/sys/devices/system/cpu/cpu0/cpufreq# grep -R .
amd_pstate_highest_perf:166
scaling_min_freq:550000
scaling_available_governors:conservative ondemand userspace powersave performance schedutil 
scaling_governor:schedutil
cpuinfo_max_freq:5084000
boost:0
amd_pstate_lowest_nonlinear_freq:1743000
amd_pstate_max_freq:5084000
related_cpus:0
scaling_cur_freq:2880470
scaling_setspeed:<unsupported>
affected_cpus:0
scaling_max_freq:5084000
cpuinfo_transition_latency:20000
scaling_driver:amd-pstate<<<<<<<<<<<<<<<<<<<<<<<<<
cpuinfo_min_freq:550000

Perry.
Comment 23 Andrei Amuraritei 2024-04-27 06:38:47 UTC
Hi Perry,

The driver loads when manually specifying amd_pstate kernel param. My issue is that it does not get automatically enabled for the system, as stated in the first message.

The issue is why is not enabled automatically, if the hardware support is there?

Thanks.
Andrei Amuraritei
Comment 24 Perry Yuan(AMD) 2024-04-27 13:25:55 UTC
Hi Andrei,

"pstate did not load automatically for me on this system as long as I remember (across Debian or Fedora with kernels 6.x)."

Ok, I thought your system cannot load pstate driver, not the problem is simple to resolve.

Have you configured the kernel config to enable the default mode 3(EPP) or set to disabled?

Could you check your kernel config if you have enabled the AMD pstate driver?

Here is my kernel config option, you can update config if you have not enabled it to default 3. 

# grep -rsn  AMD_PSTATE .config
693:CONFIG_X86_AMD_PSTATE=y
694:CONFIG_X86_AMD_PSTATE_DEFAULT_MODE=3
695:# CONFIG_X86_AMD_PSTATE_UT is not set

Perry.
Comment 25 Andrei Amuraritei 2024-04-27 22:58:55 UTC
Hi again,

I meant, amd_pstate did not load automatically, without me specifying a kernel param for it.

The .config values are (Fedora default config):

grep -rsn  AMD_PSTATE /boot/config-6.8.7-200.fc39.x86_64 
700:CONFIG_X86_AMD_PSTATE=y
701:CONFIG_X86_AMD_PSTATE_DEFAULT_MODE=3
702:CONFIG_X86_AMD_PSTATE_UT=m

Thanks.
Comment 26 Perry Yuan(AMD) 2024-04-30 06:48:53 UTC
(In reply to Andrei Amuraritei from comment #25)
> Hi again,
> 
> I meant, amd_pstate did not load automatically, without me specifying a
> kernel param for it.
> 
> The .config values are (Fedora default config):
> 
> grep -rsn  AMD_PSTATE /boot/config-6.8.7-200.fc39.x86_64 
> 700:CONFIG_X86_AMD_PSTATE=y
> 701:CONFIG_X86_AMD_PSTATE_DEFAULT_MODE=3
> 702:CONFIG_X86_AMD_PSTATE_UT=m
> 
> Thanks.

Thank you help to share the info, I have found the root cause, working on the fix patch, will send it out after we finish the testing.


Perry.
Comment 27 Perry Yuan(AMD) 2024-05-07 08:40:47 UTC
Hi, Please try this patchset for the issue

https://lore.kernel.org/lkml/cover.1715065568.git.perry.yuan@amd.com/
Comment 28 Brandon Nielsen 2024-05-13 19:04:45 UTC
(In reply to Perry Yuan(AMD) from comment #27)
> Hi, Please try this patchset for the issue
> 
> https://lore.kernel.org/lkml/cover.1715065568.git.perry.yuan@amd.com/

I see the same issue on my 5800x3d and Gigabyte X570S Aorus Elite and would like to test this fix. What kernel does this patchset apply to? I get conflicts on both 6.9 and linux-next.
Comment 29 Perry Yuan(AMD) 2024-05-14 02:30:58 UTC
 Hi Brandon, 

V2 patchset had been sent out,

https://lore.kernel.org/lkml/cover.1715356532.git.perry.yuan@amd.com/


the patches need to be applied to linux-pm/bleeding-ege branch,

https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git/?h=bleeding-edge

Perry.
Comment 30 Brandon Nielsen 2024-05-14 18:25:37 UTC
(In reply to Perry Yuan(AMD) from comment #29)
>  Hi Brandon, 
> 
> V2 patchset had been sent out,
> 
> https://lore.kernel.org/lkml/cover.1715356532.git.perry.yuan@amd.com/
> 
> 
> the patches need to be applied to linux-pm/bleeding-ege branch,
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git/
> ?h=bleeding-edge
> 
> Perry.

I can confirm with the V2 patchset applied to linux-pm/bleeding-ege, the AMD P-state driver loads by default on my system with no kernel arguments necessary.
Comment 31 Andrei Amuraritei 2024-05-15 13:44:13 UTC
Hi Perry,

I can also confirm the module loads automatically now, without specifying a kernel parameter, with V2 patchset on the mentioned kernel branch:

uname -r ; cat /sys/devices/system/cpu/amd_pstate/{prefcore,status} ; git remote -v
6.9.0-rc7+
enabled
active
origin	https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git/ (fetch)
origin	https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git/ (push)

cpupower frequency-info gives driver: amd-pstate-epp

Waiting for V3 patchset. Do you think this will be merged for kernel 6.10?

Thanks.