Subject: ACPI: (UEFI?) Oops due to tables memmap wrong after S4 platform resume

Note: supporting files with complete logs, tables, and analysis are attached to the bug report (see end of text for detail).

With all versions tested from v6.9 going back to v5.10 (and beyond) - mainline and Debian kernels - this issue occurs on a Lenovo E495s on doing "platform" resume from S4. An OS "shutdown" resume is not affected. This bug manifests as an Oops in acpi_cpufreq but extensive debugging reveals it is due to the ACPI tables memmap changing (some tables move 'up' by one 4K page) - we find that on resume /sys/firmware/acpi/tables/SSDT5 (containing the per-CPU performance states that acpi_cpufreq relies upon) actually contains CDAT and therefore appears corrupt:

$ hexdump --length 16 -C 1-before-shutdown-hibernation/SSDT5
00000000  53 53 44 54 9c 11 00 00  01 5a 4c 45 4e 4f 56 4f  |SSDT.....ZLENOVO|
$ hexdump --length 16 -C 4-after-resume-from-shutdown/SSDT5
00000000  43 52 41 54 10 08 00 00  01 5f 4c 45 4e 4f 56 4f  |CRAT....._LENOVO|

[  267.784456] acpi_cpufreq: overriding BIOS provided _PSD data
[  267.787251] ACPI: button: Power Button [PWRB]
[  267.790096] input: Lid Switch as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0D:00/input/input101
[  267.791742] ACPI Error: Null stack entry at 00000000bc5560e1 (20220331/exresop-139)
[  267.792943] ACPI Error: AE_AML_INTERNAL, While resolving operands for [OpcodeName unavailable] (20220331/dswexec-431)
[  267.794270] ACPI Error: Aborting method \_PR.C003._PPC due to previous error (AE_AML_INTERNAL) (20220331/psparse-529)
[  267.795403] ACPI: \_PR_.C003: _PPC evaluation failed: AE_AML_INTERNAL
[  267.796803] BUG: kernel NULL pointer dereference, address: 0000000000000008
[  267.797993] #PF: supervisor read access in kernel mode
[  267.799224] #PF: error_code(0x0000) - not-present page
[  267.800323] PGD 0 P4D 0 
[  267.801382] Oops: 0000 [#1] PREEMPT SMP NOPTI
[  267.802551] CPU: 1 PID: 351 Comm: (udev-worker) Not tainted 6.1.0-21-amd64 #1  Debian 6.1.90-1
[  267.803753] Hardware name: LENOVO 20QKS0EQ0N/20QKS0EQ0N, BIOS R13ET56W(1.30 ) 03/01/2024
[  267.804806] RIP: 0010:acpi_ex_resolve_multiple+0x2d/0x2c0
[  267.805923] Code: 00 00 41 56 49 89 fe 41 55 49 89 d5 41 54 49 89 cc 55 53 48 89 f3 48 83 ec 18 65 48 8b 04 25 28 00 00 00 48 89 44 24 10 31 c0 <0f> b6 46 08 48 89 34 24 48 89 74 24 08 3c 0e 0f 84 28 01 00 00 3c
[  267.807081] RSP: 0018:ffffb5ae41147958 EFLAGS: 00010246
[  267.808016] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffb5ae411479a8
[  267.808978] RDX: ffffb5ae411479a4 RSI: 0000000000000000 RDI: ffff8eb084d9e800
[  267.809953] RBP: 0000000000000000 R08: ffffb5ae411479a8 R09: ffff8eb08591beb0
[  267.810071] ACPI: button: Lid Switch [LID]
[  267.811148] R10: 000000000000000f R11: ffffb5ae41147a00 R12: ffffb5ae411479a8
[  267.811151] R13: ffffb5ae411479a4 R14: ffff8eb084d9e800 R15: ffffd5ae3fb35f48
[  267.811154] FS:  00007f64c7196900(0000) GS:ffff8eb330a40000(0000) knlGS:0000000000000000
[  267.815792] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  267.816957] CR2: 0000000000000008 CR3: 0000000100d46000 CR4: 00000000003506e0
[  267.818133] Call Trace:
[  267.819255]  <TASK>
[  267.820397]  ? __die_body.cold+0x1a/0x1f
[  267.821538]  ? page_fault_oops+0xd2/0x2b0
[  267.822434]  ? __alloc_pages+0x1dc/0x330
[  267.823514]  ? exc_page_fault+0x70/0x170
[  267.824577]  ? asm_exc_page_fault+0x22/0x30
[  267.826045]  ? acpi_ex_resolve_multiple+0x2d/0x2c0
[  267.827495]  acpi_ex_opcode_1A_0T_1R+0x241/0x5a0
[  267.828655]  acpi_ds_exec_end_op+0x27e/0x510
[  267.829865]  acpi_ps_complete_final_op+0xa4/0x190
[  267.831062]  acpi_ps_parse_loop+0x2ed/0x6a0
[  267.832320]  acpi_ps_parse_aml+0x7c/0x3d0
[  267.833382]  acpi_ps_execute_method+0x13b/0x270
[  267.834435]  acpi_ns_evaluate+0x1ee/0x2d0
[  267.835479]  acpi_evaluate_object+0x149/0x2f0
[  267.836517]  acpi_evaluate_integer+0x6b/0xf0
[  267.837544]  acpi_processor_get_platform_limit+0x43/0x140
[  267.838564]  acpi_processor_register_performance+0x5d/0xd0
[  267.839566]  acpi_cpufreq_cpu_init+0x13c/0x8e0 [acpi_cpufreq]

MS Windows 10 can successfully resume from S4 platform; we confirmed it was using platform mode, and S4 not S5, because a lid-open event wakes the PC. The firmware specifically supports Linux (OSI=Linux) and changelogs for firmware mention Linux support (v.130 change-log states "Fixed an issue that Linux TSC is unstable and system will hang after exit from Boot Menu" ).

The diagnosis workflow consists of capturing ACPI tables and kernel log at each stage of:

  1. Cold boot
  2. S4 platform resume into initialramfs before loading hibernation snapshot
  3. After OS hibernation exit

In each case the kernel command line contains:

  resume=/dev/sda3 modprobe.blacklist=acpi_cpufreq acpi_osi=! acpi_osi="Windows 2016"

acpi_osi is set to ensure the same (optimum) code path will be taken in DSDT code during testing. DSDT also supports OSI="Linux" which was being used initially (by default) and suffered in the same way.

Analysis of the kernel log shows that the cause of the problem is the mapping of IVRS, SSDT(5), CDAT, CDIT moves 'up' by one 4K page during S4 platform resume:

 ACPI: VFCT 0x00000000B8CD3000 00D484 (v01 LENOVO TP-R13   000011E0 PTEC 00000002)
-ACPI: IVRS 0x00000000B8CD1000 00013E (v02 LENOVO TP-R13   000011E0 PTEC 00000002)
-ACPI: SSDT 0x00000000B8CCF000 00119C (v01 LENOVO TP-R13   00000001 AMD  00000001)
-ACPI: CRAT 0x00000000B8CCE000 000810 (v01 LENOVO TP-R13   000011E0 PTEC 00000002)
-ACPI: CDIT 0x00000000B8CCD000 000029 (v01 LENOVO TP-R13   000011E0 PTEC 00000002)
+ACPI: IVRS 0x00000000B8CD2000 00013E (v02 LENOVO TP-R13   000011E0 PTEC 00000002)
+ACPI: SSDT 0x00000000B8CD0000 00119C (v01 LENOVO TP-R13   00000001 AMD  00000001)
+ACPI: CRAT 0x00000000B8CCF000 000810 (v01 LENOVO TP-R13   000011E0 PTEC 00000002)
+ACPI: CDIT 0x00000000B8CCE000 000029 (v01 LENOVO TP-R13   000011E0 PTEC 00000002)
 ACPI: FPDT 0x00000000B8CCC000 000034 (v01 LENOVO TP-R13   000011E0 PTEC 00000002)

We assume the tables are mapped in a stack-like fashion (grows towards lower addresses). On cold boot IVRS is at B8CD1000 but after S4 "platform" resume it is at B8CD2000 (one 4K page higher). However, the original cold-boot table pointers are used after hibernation exit and this results with incorrect data being read. After detailed investigation we find that IVRS now only apparently contains 12 bytes, not 318 as it should, and we further see that those bytes are actually from offset 0x1000 (4K) into SSDT5 and the table length value of 12 is being interpreted from the 0x0000000c at offset 4:

$ hexdump --length 16 --skip 4096 --format '"%07.7_ax  " 16/1 "%02x " "\n"' 2-before-platform-hibernate/SSDT5
0001000  fd 04 00 00 0c 00 00 00 00 0c 00 00 00 00 0c 02

$ hexdump --format '"%07.7_ax  " 16/1 "%02x " "\n"' 4-after-resume-from-shutdown/IVRS
0000000  fd 04 00 00 0c 00 00 00 00 0c 00 00

This makes sense since IVRS, SSDT(5), CRAT, CDIT have all moved 'up' by one 4K page but the table pointers retain the original cold-boot addresses.

What appears to happen here is that at cold boot firmware uses a page between VFCT (video BIOS image) and IVRS but not for a table, but during S4 "platform" resume that page is not used so IVRS gets mapped there, resulting in it and the following tables all being 4K above where they are expected to be.

To discover if anything significant is in the regular boot 'hidden' page (0xb8cd2000) we built v6.9 with CONFIG_STRICT_DEVMEM=n and then dumped the page via /dev/mem. We did this for a 'warm' reboot after an S4 resume and a 'cold' boot. In both cases the page appears to contain much the same data - namely what looks like a structure starting at offset 0 with what could be a signature - "hapt" - and what could plausibly be remains of a stack containing addresses and data. If we widen the analysis to include the page immediately preceeding it (0xb8cd1000) and those following we see IVRS and discarded data (in the 'warm' case including recognisible parts of SSDT(5) from when it was mapped at 0xb8cd1000). In both cases though the "hapt" structure does not look to be spill-over from the previous page.

We include logs of a test run with v6.9 having CONFIG_EFI_PGT_DUMP=y. We also experimented with reserving the 'missing' page using memmap=4K$0xb8cd2000.

Tests were done with firmwares v1.25 (R13ET51W - 2022/02/24), v1.28 (R13ET54W - 2023/02/27), v1.29 (R13ET55W - 2023/12/26) and the latest v1.30 (R13ET56W - 2024/04/12).

Report archive contents:

$ find data -maxdepth 2 -type d
data
data/v6.1.90_debian
data/v6.1.90_debian/1-boot
data/v6.1.90_debian/2-platform-S4-resume-initrd
data/v6.1.90_debian/3-platform-S4-resume-after-hibernation_exit
data/v6.9+EFI_PGT_DUMP
data/v6.9+EFI_PGT_DUMP/1-boot
data/v6.9+EFI_PGT_DUMP/2-platform-S4-resume-initrd
data/v6.9+EFI_PGT_DUMP/3-platform-S4-resume-after-hibernation_exit

Each directory contains contents of /sys/firmware/acpi/tables plus the kernel dmesg log. Some contain analysis results of memory-maps and dmesg differences that lead to identifying the 'hidden' page cause.

v6.9+EFI_PGT_DUMP also contains the files:

ACPI-hapt_0xb8cd1000+16384_boot.hexdump.txt
ACPI-hapt_0xb8cd1000+16384_platform-S4_resume.hexdump.txt
ACPI-hapt_0xb8cd2000_boot.hexdump.txt
ACPI-hapt_0xb8cd2000_platform-S4_resume.hexdump.txt

taken from /dev/mem