Currently AMD Ryzen CPUs which are not the maximum core state and probably binned down CPUs are reported as offline. I don't think this is intended to be like this. This is reproduceable on 5900X and 5800X (and probably others). This issue was for a short time fixed in 6.2 or 6.3 (im not sure) This can be checked with following command: ``` cat /sys/devices/system/cpu/online 0-23 cat /sys/devices/system/cpu/offline 24-31 ``` Example cpupower frequency set -g: ``` sudo cpupower frequency-set -g performance [sudo] password for ptr1337: Setting cpu: 0 Setting cpu: 1 Setting cpu: 2 Setting cpu: 3 Setting cpu: 4 Setting cpu: 5 Setting cpu: 6 Setting cpu: 7 Setting cpu: 8 Setting cpu: 9 Setting cpu: 10 Setting cpu: 11 Setting cpu: 12 Setting cpu: 13 Setting cpu: 14 Setting cpu: 15 Setting cpu: 16 Setting cpu: 17 Setting cpu: 18 Setting cpu: 19 Setting cpu: 20 Setting cpu: 21 Setting cpu: 22 Setting cpu: 23 Following CPUs are offline: 24-31 cpupower set operation was not performed on them ``` RCU does also the restrict the CPUs to 32, instead of 24: ``` [ 0.095488] rcu: Preemptible hierarchical RCU implementation. [ 0.095488] rcu: RCU restricting CPUs from NR_CPUS=320 to nr_cpu_ids=32. [ 0.095489] rcu: RCU priority boosting: priority 1 delay 500 ms. [ 0.095489] Trampoline variant of Tasks RCU enabled. [ 0.095490] Rude variant of Tasks RCU enabled. [ 0.095490] Tracing variant of Tasks RCU enabled. [ 0.095490] rcu: RCU calculated value of scheduler-enlistment delay is 50 jiffies. [ 0.095491] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=32 ``` Same counts for pcpu-alloc: ``` [ 0.042056] pcpu-alloc: [0] 00 01 02 03 04 05 06 07 [0] 08 09 10 11 12 13 14 15 [ 0.042060] pcpu-alloc: [0] 16 17 18 19 20 21 22 23 [0] 24 25 26 27 28 29 30 31 ``` If this could be an regression at the RCU Subsystem or others needs to be checked, im not sure how this correctly works. System Information: Operating System: CachyOS Linux KDE Plasma Version: 5.27.9 KDE Frameworks Version: 5.111.0 Qt Version: 5.15.11 Kernel Version: 6.6.0-4-cachyos-sched-ext-lto (64-bit) Graphics Platform: Wayland Processors: 24 × AMD Ryzen 9 5900X 12-Core Processor Memory: 31,2 GiB of RAM Graphics Processor: NVIDIA GeForce GTX 1070 Ti/PCIe/SSE2 Manufacturer: Gigabyte Technology Co., Ltd. Product Name: B550 AORUS ELITE AX V2
Would you mind sharing an acpidump for your system? I think this is actually a BIOS bug not a kernel bug.
Created attachment 305373 [details] acpidump acpidump
Hi Mario, I have attached you one. Since we know a bunch of people, which have this issue on their Zen3 CPU it could then also affect several BIOS.
> [008h 0008 1] Revision : 04 It appears that the MADT in this BIOS conforms to FADT Major version 04, no minor version specified. https://uefi.org/htmlspecs/ACPI_Spec_6_4_html/05_ACPI_Software_Programming_Model/ACPI_Software_Programming_Model.html#fixed-acpi-description-table-fadt > $ grep "Runtime Online Capable : 0" apic.dsl | wc -l 32 None of the processors are declared as runtime online capable (hotpluggable) > $ grep "Enabled : 1" apic.dsl | wc -l 24 > $ grep "Enabled : 0" apic.dsl | wc -l 8 You can see that your /proc/cpuinfo matches the 12 cores, but the BIOS has entries for another 4 cores that makes Linux advertise this incorrectly. The way this stuff all works is that there is a function acpi_is_processor_usable() which looks at the FADT version to decide whether to trust the values in the MADT. This is because the bit for "Runtime Online Capable" wasn't introduced until FADT major 6 minor 3 https://github.com/torvalds/linux/blob/d2f51b3516dade79269ff45eae2a7668ae711b25/arch/x86/kernel/acpi/boot.c#L200 Here is the ACPI spec location for it: https://uefi.org/htmlspecs/ACPI_Spec_6_4_html/05_ACPI_Software_Programming_Model/ACPI_Software_Programming_Model.html#processor-local-apic-structure > I have attached you one. Since we know a bunch of people, which have this issue on their Zen3 CPU it could then also affect several BIOS. Is this just a cosmetic reporting issue, or is there a functional Linux problem because of this BIOS bug?
> This issue was for a short time fixed in 6.2 or 6.3 (im not sure) The reason that this changed after kernel 6.3 is because there was a regression reported by forcing this to be ACPI 6.3 or later for hotplug CPUs to work. This is the commit that changed the behavior. https://github.com/torvalds/linux/commit/fed8d8773b8ea68ad99d9eee8c8343bef9da2c2c
Sorry I made one mistake. This is the field that has FADT major/minor. You have 6.2 not 4.0. [008h 0008 1] Revision : 06 [083h 0131 1] FADT Minor Revision : 02
Hi Mario, Thank you very much for your explanation and going into the details. I am not not sure, how much this could affect the general linux system. The only regression, which I found, which happened due of this issue was in sched-ext - but this is nothing in the offical kernel. You can follow the issue here: https://github.com/sched-ext/sched_ext/issues/69 Setting nr_cpus=24 fixed this issue and hetjun also has impleneted a fix for this for "scx_rusty". But the question is, how much this false reporting does affect the RCU Subsystem, Watchdog and percpualloc. (There could be maybe other also, these are which I know and report the threads as 32 instead of 24)
> Thank you very much for your explanation and going into the details. One more thing - You might want to read this original bug report to better understand how we got where we are. https://lore.kernel.org/all/20230327191026.3454-1-eric.devolder@oracle.com/ > The only regression, which I found, which happened due of this issue was in > sched-ext - but this is nothing in the offical kernel. You can follow the > issue here: I'll leave a comment. > But the question is, how much this false reporting does affect the RCU > Subsystem, Watchdog and percpualloc. (There could be maybe other also, these > are which I know and report the threads as 32 instead of 24) My expectation is that it shouldn't break other things and it's just cosmetic, but if you find otherwise please mention.
Hi Mario, I don't think I'd say this is just cosmetic. The kernel looks at the possible cpumap in a bunch of places, and does all sorts of allocations, work, etc based on what it thinks are CPUs that could someday come online. Wouldn't this affect any place where cpu_possible_mask is referenced?