Bug 213047 - VMD created PCI bridge lacks ACPI companion device - no d3cold support for NVMe
Summary: VMD created PCI bridge lacks ACPI companion device - no d3cold support for NVMe
Status: NEEDINFO
Alias: None
Product: Power Management
Classification: Unclassified
Component: Other (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: Rafael J. Wysocki
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-05-12 19:27 UTC by David Box
Modified: 2021-07-19 20:49 UTC (History)
5 users (show)

See Also:
Kernel Version: 5.12
Tree: Mainline
Regression: No


Attachments
DSDT (3.60 MB, text/x-csrc)
2021-05-12 19:28 UTC, David Box
Details
SSDT containing VMD power resource objects (370.16 KB, text/x-csrc)
2021-05-12 19:29 UTC, David Box
Details
PCI: VMD: Set ACPI companion for the VMD host bridge (953 bytes, patch)
2021-05-14 13:55 UTC, Rafael J. Wysocki
Details | Diff

Description David Box 2021-05-12 19:27:31 UTC
When operating under VMD, the root port for the NVMe doesn't have an ACPI companion device and therefore lacks support do D3 cold.

Under AHCI mode we see the following:

/sys/devices/pci0000:00/0000:00:1d.0/firmware_node/adr:0x001d0000
/sys/devices/pci0000:00/0000:00:1d.0/firmware_node/path:\_SB_.PC00.RP13
/sys/devices/pci0000:00/0000:00:1d.0/0000:01:00.0/firmware_node/adr:0x00000000
/sys/devices/pci0000:00/0000:00:1d.0/0000:01:00.0/firmware_node/path:\_SB_.PC00.RP13.PXSX

Under VMD mode however, we only see this:

/sys/devices/pci0000:00/0000:00:0e.0/firmware_node/adr:0x000e0000
/sys/devices/pci0000:00/0000:00:0e.0/firmware_node/path:\_SB_.PC00.VMD0


The VMD driver creates a new host bridge starting on bus 10000:
10000:e0:17.0 SATA controller: Intel Corporation Device 7ae2 (rev 10)
10000:e0:1d.0 System peripheral: Intel Corporation Device 09ab
10000:e0:1d.4 PCI bridge: Intel Corporation Device 7ab4 (rev 10)
10000:e1:00.0 Non-Volatile memory controller: SK hynix Device 1627


But none of these devices have ACPI objects associated with the, despite their appearance in the acpidump.
Comment 1 David Box 2021-05-12 19:28:44 UTC
Created attachment 296731 [details]
DSDT
Comment 2 David Box 2021-05-12 19:29:33 UTC
Created attachment 296733 [details]
SSDT containing VMD power resource objects
Comment 3 Rafael J. Wysocki 2021-05-14 13:55:50 UTC
Created attachment 296747 [details]
PCI: VMD: Set ACPI companion for the VMD host bridge

If I'm not mistaken, this may be as simple as the attached patch.

The _PS0 and _PS3 objects under the ACPI device in question are a bit of a concern, but AFAICS the PCI layer doesn't do any PM of the host bridge devices, so they should not be evaluated in that context.

Please test and let me know how it goes.
Comment 4 David Box 2021-05-15 20:17:21 UTC
The patch did add a companion device to the VMD host bridge but this was not enough to find companions for the devices on the bridge. I was able to determine that this is because the _ADRs for these devices don't match the expected format. The VMD folks did say that they use the following convention:

For PCIE: 8xyzFFFF, where xyz is the BDF 0Bbbbb'ddddd'fff. 
For SATA: 0xyxFFFt, where t is the port and xyz is the BDF as above.

But this doesn't match the ACPI spec. I enabled the ACPI debug messages (ADBG) to print the _ADRs for the devices remapped under VMD. They are in the "VMD _ADR" column below.

                | Bus Location  | ACPI _ADR  | VMD _ADR
PCIe Root Port  | 10000:e0:6.0  | 0x60000    | 0x8030FFFF 
SATA Controller	| 10000:e0:17.0 | 0x17000    | 0XB8FFF0

Because of this mismatch, the devices are not found by pci_acpi_find_companion(). The ACPI _ADR does convert the the VMD _ADR using the VMD convention above. When I hacked the function to do this conversion for devices on domain 0x10000, they were able to find their ACPI companions devices. And the root port was able to go to D3 cold on suspend. But this is of course not standard. We will need to get documentation for this. The VMD driver may need to implement its own acpi_bus_type in order to do the expected _ADR search.
Comment 5 Adam Štrauch 2021-07-18 20:49:44 UTC
Hi,

would be possible to post here the mentioned hack? Unfortunately my notebook is affected by this bug and HP does't allow me to disable VMD from BIOS setup so that hack is probably the only option I could use until it's properly fixed.
Comment 6 David Box 2021-07-19 20:11:26 UTC
(In reply to Adam Štrauch from comment #5)
> Hi,
> 
> would be possible to post here the mentioned hack? Unfortunately my notebook
> is affected by this bug and HP does't allow me to disable VMD from BIOS
> setup so that hack is probably the only option I could use until it's
> properly fixed.

This particular VMD issue should not affect notebooks. Can you open a new bug report for your HP system? If you think it's VMD related assign it to Power Management
Comment 7 Adam Štrauch 2021-07-19 20:49:00 UTC
I already did and I was pointed into this bug report there :-)

https://bugzilla.kernel.org/show_bug.cgi?id=213717

Note You need to log in before you can comment on or make changes to this bug.