Bug 209711 - Ethernet: r8169: ACPI runtime GPE enabled for D3cold only works once. It stops working on second time GPE is enabled - Dell Wyse 5070
Summary: Ethernet: r8169: ACPI runtime GPE enabled for D3cold only works once. It stop...
Status: RESOLVED WILL_NOT_FIX
Alias: None
Product: ACPI
Classification: Unclassified
Component: Config-Interrupts (show other bugs)
Hardware: Intel Linux
: P1 normal
Assignee: acpi_config-interrupts
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-10-16 09:23 UTC by Kai-Heng Feng
Modified: 2020-12-10 13:32 UTC (History)
2 users (show)

See Also:
Kernel Version: v5.9 mainline, pm/bleeding-edge
Subsystem:
Regression: No
Bisected commit-id:


Attachments
ACPI GPEs at boot, can detect RJ45 plugging (5.93 KB, text/plain)
2020-10-16 09:28 UTC, Kai-Heng Feng
Details
ACPI GPEs when RJ45 is plugged (5.93 KB, text/plain)
2020-10-16 09:29 UTC, Kai-Heng Feng
Details
ACPI GPEs when RJ45 is unplugged. It can no longer detects RJ45 cable plugging (5.93 KB, text/plain)
2020-10-16 09:29 UTC, Kai-Heng Feng
Details
dmesg (73.62 KB, text/plain)
2020-10-16 09:30 UTC, Kai-Heng Feng
Details
lspci -tv (939 bytes, text/plain)
2020-10-16 09:30 UTC, Kai-Heng Feng
Details
lspci -vv (20.74 KB, text/plain)
2020-10-16 09:30 UTC, Kai-Heng Feng
Details
acpidump (463.36 KB, text/plain)
2020-10-16 09:46 UTC, Kai-Heng Feng
Details
lspci -vv (32.42 KB, text/plain)
2020-11-04 07:27 UTC, Kai-Heng Feng
Details
firmware_nodes (1.09 KB, text/plain)
2020-11-04 07:28 UTC, Kai-Heng Feng
Details

Description Kai-Heng Feng 2020-10-16 09:23:46 UTC
On Dell Wyse 5070, ethernet controller (r8169) is connected to a root port with D3cold support (i.e. _PR3 present).

Plugging RJ45 cable can wake the device from D3cold, but just for once. The second time the device gets put to D3cold, plugging RJ45 can no longer wake the device up.

The ethernet controller is using GPE 8, and the root port is using GPE 9.
Here's the GPE status after boot:
gpe07:       0         invalid      unmasked
gpe08:       0  EN     enabled      unmasked
gpe09:       0  EN     enabled      unmasked

Here's the GPE status after RJ45 cable is plugged to ethernet port, or we manually resume the device using `lspci`:
gpe07:       0     STS invalid      unmasked
gpe08:       0         disabled     unmasked
gpe09:       0         disabled     unmasked

And after RJ45 cable is unplugged, device and root are suspended again:
gpe07:       0     STS invalid      unmasked
gpe08:       0         disabled     unmasked
gpe09:       0         disabled     unmasked

And plugging RJ45 doesn't work anymore. I wonder if STS on GPE 7 causes this.
Comment 1 Kai-Heng Feng 2020-10-16 09:27:22 UTC
Blacklisting the ethernet driver (r8169) to make sure GPE 8 is not in use.
Resume the root port, we can still observe the GPE 7 status bit is flagged.
Comment 2 Kai-Heng Feng 2020-10-16 09:28:39 UTC
Created attachment 293013 [details]
ACPI GPEs at boot, can detect RJ45 plugging
Comment 3 Kai-Heng Feng 2020-10-16 09:29:12 UTC
Created attachment 293015 [details]
ACPI GPEs when RJ45 is plugged
Comment 4 Kai-Heng Feng 2020-10-16 09:29:52 UTC
Created attachment 293017 [details]
ACPI GPEs when RJ45 is unplugged. It can no longer detects RJ45 cable plugging
Comment 5 Kai-Heng Feng 2020-10-16 09:30:13 UTC
Created attachment 293019 [details]
dmesg
Comment 6 Kai-Heng Feng 2020-10-16 09:30:29 UTC
Created attachment 293021 [details]
lspci -tv
Comment 7 Kai-Heng Feng 2020-10-16 09:30:46 UTC
Created attachment 293023 [details]
lspci -vv
Comment 8 Kai-Heng Feng 2020-10-16 09:46:11 UTC
Created attachment 293027 [details]
acpidump
Comment 9 Heiner Kallweit 2020-10-18 12:56:32 UTC
Speaking for r8169 and RTL8168h:
Chip initialization includes the following:
RTL_W8(tp, MISC_1, RTL_R8(tp, MISC_1) & ~PFM_D3COLD_EN);
Due to missing chip documentation I can't say what this actually does, but it wouldn't be a surprise if D3cold doesn't work properly.

How is it if add a call to pci_d3cold_disable() to rtl_init_one() in r8169?
Is waking from D3hot working properly?
Comment 10 Kai-Heng Feng 2020-10-19 12:38:00 UTC
(In reply to Heiner Kallweit from comment #9)
> Speaking for r8169 and RTL8168h:
> Chip initialization includes the following:
> RTL_W8(tp, MISC_1, RTL_R8(tp, MISC_1) & ~PFM_D3COLD_EN);
> Due to missing chip documentation I can't say what this actually does, but
> it wouldn't be a surprise if D3cold doesn't work properly.

I've seen r8169 works with D3cold without any issue.
So if we really need to disable D3cold, we should do it at pcieport driver.

> 
> How is it if add a call to pci_d3cold_disable() to rtl_init_one() in r8169?
> Is waking from D3hot working properly?

PCI PME on D3hot works properly.
Comment 11 Zhang Rui 2020-11-02 03:26:01 UTC
So the problem is caused by disabled GPE 08 and 09?

does the problem still exist if you run "echo enable > gpe08" and "echo enable > gpe09" to enable them manually?
Comment 12 Kai-Heng Feng 2020-11-03 15:22:32 UTC
Sorry, it was a copy error.

Here's the status when it's suspended again:
gpe07:       0     STS invalid      unmasked
gpe08:       0  EN     enabled      unmasked
gpe09:       0  EN     enabled      unmasked
Comment 13 Kai-Heng Feng 2020-11-03 15:25:47 UTC
And I cannot clear gpe07:

# echo clear > gpe07
bash: echo: write error: Invalid argument

[ 1870.556493] ACPI: Can not change Invalid GPE/Fixed Event status
Comment 14 Zhang Rui 2020-11-04 02:05:37 UTC
(In reply to Kai-Heng Feng from comment #12)
> Sorry, it was a copy error.
> 
> Here's the status when it's suspended again:
> gpe07:       0     STS invalid      unmasked
> gpe08:       0  EN     enabled      unmasked
> gpe09:       0  EN     enabled      unmasked

So, it works as following?

The ethernet controller is using GPE 8, and the root port is using GPE 9.
Here's the GPE status after boot:
gpe07:       0         invalid      unmasked
gpe08:       0  EN     enabled      unmasked
gpe09:       0  EN     enabled      unmasked

Here's the GPE status after RJ45 cable is plugged to ethernet port, or we manually resume the device using `lspci`:
gpe07:       0     STS invalid      unmasked
gpe08:       0         disabled     unmasked
gpe09:       0         disabled     unmasked

And after RJ45 cable is unplugged, device and root are suspended again:
gpe07:       0     STS invalid      unmasked
gpe08:       0         enabled      unmasked
gpe09:       0         enabled      unmasked
Comment 15 Zhang Rui 2020-11-04 02:19:14 UTC
can you attach the output of lspci -v as well as "grep . /sys/bus/pci/devices/*/firmware_node/path"?
Comment 16 Kai-Heng Feng 2020-11-04 07:27:56 UTC
Created attachment 293427 [details]
lspci -vv
Comment 17 Kai-Heng Feng 2020-11-04 07:28:27 UTC
Created attachment 293429 [details]
firmware_nodes
Comment 18 Kai-Heng Feng 2020-11-04 07:44:05 UTC
Using this diff to do some test:
diff --git a/drivers/acpi/sysfs.c b/drivers/acpi/sysfs.c
index a5cc4f3bb..492259f2a 100644
--- a/drivers/acpi/sysfs.c
+++ b/drivers/acpi/sysfs.c
@@ -755,11 +755,13 @@ static ssize_t counter_set(struct kobject *kobj,
        if (result)
                goto end;
 
+       /*
        if (!(status & ACPI_EVENT_FLAG_HAS_HANDLER)) {
                printk(KERN_WARNING PREFIX
                       "Can not change Invalid GPE/Fixed Event status\n");
                return -EINVAL;
        }
+       */
 
        if (index < num_gpes) {
                if (!strcmp(buf, "disable\n") &&

When the issue happened:
gpe07:       0     STS invalid      unmasked
gpe08:       0         enabled      unmasked
gpe09:       0         enabled      unmasked

Clear gpe07:
# echo clear > gpe07
gpe07:       0         invalid      unmasked
gpe08:       0  EN     enabled      unmasked
gpe09:       0  EN     enabled      unmasked

Plug RJ45:
gpe07:       0     STS invalid      unmasked
gpe08:       0  EN     enabled      unmasked
gpe09:       0  EN     enabled      unmasked

Feels like the GPE shifted to GPE07?
Comment 19 Zhang Rui 2020-11-04 15:14:20 UTC
does the problem still exist when GPE07 cleared manually?
Comment 20 Kai-Heng Feng 2020-11-04 15:18:17 UTC
Yes. Both gpe08 and gpe09 don't work even with gpe07 gets cleared prior.
Comment 21 Kai-Heng Feng 2020-11-10 05:05:10 UTC
Zhang Rui,
Do you think it's a firmware/hardware bug? If so I'll write a quirk to disable D3cold for the root port.
Comment 22 Kai-Heng Feng 2020-11-30 13:47:35 UTC
Turns out the first wake is from native PME. ACPI GPE never works.
I'll ask if ODM is willing to fix is in DSDT/SSDT. Otherwise I'll write a quirk instead.
Comment 23 Kai-Heng Feng 2020-12-10 13:32:03 UTC
Looks like vendor will fixed it in BIOS, by changing _S0W from 4 to 3.

Note You need to log in before you can comment on or make changes to this bug.