Bug 216440

Is this a regression? Did kernel 5.18.x work?

Would be great if you could perform regression testing using git bisect.

Possibly related: bug 216438

Please have a try with iommu=pt on kernel command line.

Created attachment 301732 [details]
dmesg with resume issue using iommu=pt

Requeste dmesg with iommu-pt parameter

(In reply to Artem S. Tashkinov from comment #1)
> Is this a regression? Did kernel 5.18.x work?
> 
> Would be great if you could perform regression testing using git bisect.

No regression as the issue also happened on kernel 5.17.x from a fresh installation. git bisect returned nothing

(In reply to Artem S. Tashkinov from comment #1)
> Is this a regression? Did kernel 5.18.x work?
> 
> Would be great if you could perform regression testing using git bisect.

No regression as the issue also happened on kernel 5.17.x from a fresh installation meaning all kernel version is impacted. git bisect returned nothing

Created attachment 301733 [details]
updated dmesg with iommu=pt on kernel 5.19.6

Please share an acpidump. I suspect a bios issue and would need to check something to confirm.

Created attachment 301735 [details]
acpidump from kernel 5.19.6 on Dell Inspiron 14 7425 2-in-1

As requested, here is the acipdump from kernel 5.19.6

The kernel isn't detecting StorageD3Enable for the NVME storage in your system.

From your kernel you're not getting this message:
> nvme 0000:BB:DD.F: platform quirk: setting simple suspend

Did you change your hardware at all from how it was shipped?

Can you please confirm that
# cat /sys/bus/pci/drivers/nvme/0000:02:00.0/firmware_node/path

returns

\_SB_.PCI0.GPP1.NVME

Yes, I replaced the original 512GB NVME from Dell with a 1TB Samsung 970 Evo Plus NVME. 

# cat /sys/bus/pci/drivers/nvme/0000\:02\:00.0/firmware_node/path
  
returns

\_SB_.PCI0.GPP1.DEV0

It is odd that result is
\_SB_.PCI0.GPP1.DEV0

as fdisk -l reads

Disk /dev/nvme0n1: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: Samsung SSD 970 EVO Plus 1TB            
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt

> Yes, I replaced the original 512GB NVME from Dell with a 1TB Samsung 970 Evo
> Plus NVME. 

Did the original NVME have this problem too?  I would think so.

Do things work properly in Windows?

> \_SB_.PCI0.GPP1.DEV0

It seems to me that the NVME is not getting assigned to the proper ACPI companion from the firmware.  It should be set to GPP1.NVME.

Maybe it's due to the ambiguity between both nodes .DEV0 and .NVME both having _ADR of 0.

Hans, any ideas?

(In reply to Mario Limonciello (AMD) from comment #13)
> > Yes, I replaced the original 512GB NVME from Dell with a 1TB Samsung 970
> Evo
> > Plus NVME. 
> 
> Did the original NVME have this problem too?  I would think so.

Yes it does.
> 
> Do things work properly in Windows?
> 
Yes, suspend/resume runs smoothly on Windows even after swapping NVME drives. The issue appears specific to Linux kernel.

To me it seems it's a spec violation and best effort from the kernel side to handle the case of two devices sharing an _ADR.  Neither has _STA, and neither has a _HID, so the existing best effort from the kernel is helpless for your case.


Perhaps we can try to go off the fact that the NVME device has device properties? Can you see if below helps to link the right ACPI companion?

diff --git a/drivers/acpi/glue.c b/drivers/acpi/glue.c
index 204fe94c7e45..8346ad564904 100644
--- a/drivers/acpi/glue.c
+++ b/drivers/acpi/glue.c
@@ -96,8 +96,12 @@ static int find_child_checks(struct acpi_device *adev, bool check_children)
                return -ENODEV;
 
        status = acpi_evaluate_integer(adev->handle, "_STA", NULL, &sta);
-       if (status == AE_NOT_FOUND)
+       if (status == AE_NOT_FOUND) {
+               /* Make this more appealing if it has properties declared */
+               if (acpi_dev_has_props(adev))
+                       return FIND_CHILD_MAX_SCORE;
                return FIND_CHILD_MIN_SCORE;
+       }
 
        if (ACPI_FAILURE(status) || !(sta & ACPI_STA_DEVICE_ENABLED))
                return -ENODEV;

Sure, I applied the patch and will return the result after the build is complete.

After applying patch from #15

# cat /sys/bus/pci/drivers/nvme/0000\:02\:00.0/firmware_node/path
\_SB_.PCI0.GPP1.DEV0

Same result as tested on 6.0.0-0.rc4.20220909git506357871c18.34

Created attachment 301800 [details]
Patch to add a quirk

That's unfortunate that didn't help.  Let's try the quirk approach instead for now.

Give this patch a try, and if it fails can you please share both an updated dmesg log and dmidecode output (so I can get the strings right).

Sadly, the patch has no effect:

 # cat /sys/bus/pci/drivers/nvme/0000\:02\:00.0/firmware_node/path\_SB_.PCI0.GPP1.DEV0

I include both dmesg and dmidecode shortly

Created attachment 301803 [details]
dell inspiron 7425dmesg via journalctl

Here is included the dmesg report running on kernel 6.0.0.rc5

Created attachment 301804 [details]
dell inspiron 7425 dmidecode

dmidecide from Dell Inspiron running kernel 6.0.0.rc5

Sep 13 10:14:12 kernel: nvme 0000:02:00.0: platform quirk: setting simple suspend

It shouldn't change device node, only set quirk. It looks like it's setting policy properly from your log. Are you still hitting functional but with it?

Created attachment 301810 [details]
dmesg showing successful speeding resume

I confirm the resume comes quicker with the patch as highlighted in the attached dmesg:

[   52.991877] PM: resume devices took 1.030 seconds
[...]
[  109.103704] PM: resume devices took 1.035 seconds

(In reply to Mario Limonciello (AMD) from comment #22)
> Sep 13 10:14:12 kernel: nvme 0000:02:00.0: platform quirk: setting simple
> suspend
> 
> It shouldn't change device node, only set quirk. It looks like it's setting
> policy properly from your log. Are you still hitting functional but with it?

Yes, the quirk functions as intended and resume is definitely quicker instead of 60 seconds long. See comment #23 for the full dmesg log.

OK, the solution is queued up for kernel 6.1.

https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git/commit/?h=bleeding-edge&id=018d6711c26e4bd26e20a819fcc7f8ab902608f3

Thank you Mario for the quick response. For Hans, will it possible to copy the fix as patch for the Fedora kernel? Thanks in advance.

(In reply to Luya Tshimbalanga from comment #26)
> Thank you Mario for the quick response. For Hans, will it possible to copy
> the fix as patch for the Fedora kernel? Thanks in advance.

I'm sorry but my workload lately has been too high for me to be able to cherry-pick individual fixes into the Fedora kernels, so you will have to wait for this to get upstream through the stable series.

This patch should find its way into the 5.19.x / 6.0.x stable series as soon as 6.1-rc1 is out.

(In reply to Hans de Goede from comment #27)
> (In reply to Luya Tshimbalanga from comment #26)
> > Thank you Mario for the quick response. For Hans, will it possible to copy
> > the fix as patch for the Fedora kernel? Thanks in advance.
> 
> I'm sorry but my workload lately has been too high for me to be able to
> cherry-pick individual fixes into the Fedora kernels, so you will have to
> wait for this to get upstream through the stable series.
> 
> This patch should find its way into the 5.19.x / 6.0.x stable series as soon
> as 6.1-rc1 is out.

Understood.

(In reply to Mario Limonciello (AMD) from comment #25)
> OK, the solution is queued up for kernel 6.1.
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git/commit/
> ?h=bleeding-edge&id=018d6711c26e4bd26e20a819fcc7f8ab902608f3

Every issue Luya has described is also happening on my Inspiron 16 5625.

Will the solution also work for me?
I'm wondering because the product name that is hardcoded is different.

Can you please share your dmidecode and acpidump?  We can certainly try to add your system to the workaround list.

Created attachment 302987 [details]
Inspiron 16 5625 dmidecode

Created attachment 302988 [details]
acpidump from kernel 5.19.14 on Inspiron 16 5625

Created attachment 302989 [details]
dmidecode-inspiron-16-5625.txt

> bugzilla-daemon@kernel.org hat am 13.10.2022 12:38 IST geschrieben:
> 
>  
> https://bugzilla.kernel.org/show_bug.cgi?id=216440
> 
> --- Comment #30 from Mario Limonciello (AMD) (mario.limonciello@amd.com) ---
> Can you please share your dmidecode and acpidump?  We can certainly try to
> add
> your system to the workaround list.
> 
> -- 
> You may reply to this email to add a comment.
> 
> You are receiving this mail because:
> You are on the CC list for the bug.

Created attachment 302990 [details]
acpidump-inspiron-16-5625.txt

Created attachment 302994 [details]
Patch for Inspiron 16 526

OK, yeah it does look the same.  Have a try with this patch.

Worked like a charm, thank you.

If you don't mind a newbie ask some questions:

Can I somehow keep this patch until it is integrated into the fedora kernel? Is this advisable?

And out of curiosity, does this change need to be hardcoded or is this just a temporary solution?

> Can I somehow keep this patch until it is integrated into the fedora kernel?
> Is this advisable?

You can talk to Fedora kernel guys about trying to carry it in the Fedora kernel sooner than release, but otherwise yes you need to keep compiling and using your own kernels until landed.

> And out of curiosity, does this change need to be hardcoded or is this just a
> temporary solution?

The BIOS offers two ACPI nodes with the same _ADR and ACPI spec doesn't define what to do here.  So you end up with kernel taking a guess and picking wrong one.  If Dell issues a BIOS update that fixes this you would no longer need this hardcoded change.

Since this fix seems to be a hardcoded list of exceptions, I wanted to mention that my device (Dell Inspiron 14 5425) is also affected. I would happily share more device details to ensure my machine will be covered by this patch.

I had been troubleshooting in the following ask.fedoraproject post until I found this bug report. Anyone interested can find many logs and device details there.

https://ask.fedoraproject.org/t/fedora-36-slow-wake-time-dell-inspiron-ryzen-7-radeon-graphics/28903

Created attachment 303353 [details]
Output of `dmidecode` on Inspiron 14 5425

Kernel: 6.0.10-300.fc37.x86_64

Created attachment 303354 [details]
Output of `acpidump` on Inspiron 14 5425

Kernel: 6.0.10-300.fc37.x86_64

This bug report is closed.

Although it is *likely* the same type of solution; please open your own bug, place those attachments there and we can track it there.  You can CC me to that bug.

Sounds good. Sorry, I missed that the issue was closed.

Here is the bug report I opened: https://bugzilla.kernel.org/show_bug.cgi?id=216773