Bug 218383

Summary: WWAN Intel XMM7560 LTE wakes system from s2idle
Product: Drivers Reporter: Martin (mwolf)
Component: network-wireless-intelAssignee: Default virtual assignee for network-wireless-intel (drivers_network-wireless-intel)
Status: NEW ---    
Severity: normal CC: dennis.lissov, kai.heng.feng, kolAflash, mario.limonciello, mkchetan, shaneparslow808
Priority: P3    
Hardware: All   
OS: Linux   
Kernel Version: Subsystem:
Regression: No Bisected commit-id:
Attachments: dmesg after the problem occured again
extracted "PM:" lines from dmesg for better visibility
I guess I am not on the proper firmware(?)
s2idle with recent version + wakeup after a short time

Description Martin 2024-01-16 18:55:19 UTC
I own a HP 845 G9 for almost a year now.
It has a 01:00.0 Wireless controller [0d40]: Intel Corporation XMM7560 LTE Advanced Pro Modem (rev 01) installed.

Since the beginning I experienced wakeups from s2idle, but I was not able to pinpoint it.
Thanks to an AMD Developer on github it seems the XMM7560 wakes up my device:
https://gitlab.freedesktop.org/drm/amd/-/issues/3082#note_2240758

2023-12-31 00:18:24,898 DEBUG:	                Case (0x11)
2023-12-31 00:18:24,898 DEBUG:	                {
2023-12-31 00:18:24,898 DEBUG:	                    M000 (0x3911)
2023-12-31 00:18:24,898 DEBUG:	                    M460 ("    Notify (\\_SB.PCI0.GP13, 0x02)\n", Zero, Zero, Zero, Zero, Zero, Zero)
2023-12-31 00:18:24,898 DEBUG:	                    Notify (\_SB.PCI0.GP13, 0x02) // Device Wake
2023-12-31 00:18:24,898 DEBUG:	                }

=> 

2023-12-31 00:18:24,850 DEBUG:	├─ 0000:01:00.0 : Intel Corporation  [8086:7560] : \_SB_.PCI0.GP13.PWAN

Can someone please take a look at it?
Comment 1 Mario Limonciello (AMD) 2024-01-17 16:39:18 UTC
To add extra credence to the above comment, Martin did confirm that disabling WWAN in BIOS fixes the issue.

So something in XMM7560 driver or firmware configuration is causing the GPIO to be asserted while the system is in s0i3 which will wake it up.
Comment 2 Martin 2024-02-16 03:03:14 UTC
Is there really no one at intel, that cares about this problem?
Comment 3 Kai-Heng Feng 2024-02-20 05:30:41 UTC
(In reply to Martin from comment #2)
> Is there really no one at intel, that cares about this problem?

I am trying to reproduce the issue on 865 G9 (can't find any 845 G9 w/ 7560).

Does the issue require the lid remain closed?
Comment 4 Martin 2024-02-20 10:38:21 UTC
Yes, occurred when the lid was closed.
Comment 5 Kai-Heng Feng 2024-02-22 01:33:21 UTC
(In reply to Martin from comment #4)
> Yes, occurred when the lid was closed.

I wonder how the issue was spotted when lid remained closed?
Comment 6 Martin 2024-02-22 09:24:59 UTC
Battery drain, fan spinning up, warm notebook and dmesg
Comment 7 Kai-Heng Feng 2024-02-26 08:31:39 UTC
OK, I can reproduce the issue on 865 G9. And seems like the issue is fixed by latest mainline kernel (v6.8-rc6). Is it possible for you to also give that kernel a try?
Comment 8 Martin 2024-02-28 00:28:15 UTC
yes, it looks fixed.
Comment 9 Mario Limonciello (AMD) 2024-02-28 00:46:45 UTC
So probably fixed by https://github.com/torvalds/linux/commit/1db34aa58d80988f5ee99d2fd9d8f7489c3b0681
?
Comment 10 Martin 2024-02-28 00:48:26 UTC
no, that is from an other bug of mine ;)
https://bugzilla.kernel.org/show_bug.cgi?id=217200
Comment 11 Martin 2024-02-28 00:50:09 UTC
no, sorry that one: https://bugzilla.kernel.org/show_bug.cgi?id=217996
Comment 12 Mario Limonciello (AMD) 2024-02-28 00:53:06 UTC
Could you bisect what actually fixed it then? I would like to understand this.
Comment 13 Martin 2024-02-28 00:54:49 UTC
Actually, I am not sure, when the problem started.
Also as you notice, there are quite a few bugs in the iosm device, that cause various issues. 
How would I take these fixes / bugs into account?
Comment 14 Martin 2024-02-28 00:56:50 UTC
Or should I basically say "bad" e.g. Kernel 6.7 and good Kernel 6.8v6?
Comment 15 Kai-Heng Feng 2024-02-29 12:38:12 UTC
(In reply to Mario Limonciello (AMD) from comment #12)
> Could you bisect what actually fixed it then? I would like to understand
> this.

I am doing the bisection. Since I use not waking up for 24 hours as "good" so it can take a while.
Comment 16 Martin 2024-03-08 04:36:49 UTC
I am not sure if it is really fixed.
(I updated to -rc7 now) 

Mär 07 15:57:37 HP845G9 systemd[1]: systemd-suspend.service: Deactivated successfully.
Mär 07 15:57:37 HP845G9 systemd[1]: Finished systemd-suspend.service - System Suspend.
Mär 07 15:58:06 HP845G9 systemd[1]: Starting systemd-suspend.service - System Suspend...
Mär 07 15:58:06 HP845G9 systemd-sleep[409553]: Entering sleep state 'suspend'...
Mär 07 16:01:12 HP845G9 systemd-sleep[409553]: System returned from sleep state.
Mär 07 16:01:12 HP845G9 systemd[1]: systemd-suspend.service: Deactivated successfully.
Mär 07 16:01:12 HP845G9 systemd[1]: Finished systemd-suspend.service - System Suspend.
Mär 07 16:01:41 HP845G9 systemd[1]: Starting systemd-suspend.service - System Suspend...
Mär 07 16:01:41 HP845G9 systemd-sleep[410010]: Entering sleep state 'suspend'...
Mär 07 16:02:12 HP845G9 systemd-sleep[410010]: System returned from sleep state.
Mär 07 16:02:12 HP845G9 systemd[1]: systemd-suspend.service: Deactivated successfully.
Mär 07 16:02:12 HP845G9 systemd[1]: Finished systemd-suspend.service - System Suspend.
Mär 07 16:02:41 HP845G9 systemd[1]: Starting systemd-suspend.service - System Suspend...
Mär 07 16:02:41 HP845G9 systemd-sleep[410437]: Entering sleep state 'suspend'...
Mär 07 16:40:14 HP845G9 systemd-sleep[410437]: System returned from sleep state.
Mär 07 16:40:14 HP845G9 systemd[1]: systemd-suspend.service: Deactivated successfully.
Mär 07 16:40:14 HP845G9 systemd[1]: Finished systemd-suspend.service - System Suspend.
Mär 07 16:40:43 HP845G9 systemd[1]: Starting systemd-suspend.service - System Suspend...
Mär 07 16:40:43 HP845G9 systemd-sleep[410934]: Entering sleep state 'suspend'...
Mär 07 16:41:13 HP845G9 systemd-sleep[410934]: System returned from sleep state.
Mär 07 16:41:14 HP845G9 systemd[1]: systemd-suspend.service: Deactivated successfully.
Mär 07 16:41:14 HP845G9 systemd[1]: Finished systemd-suspend.service - System Suspend.
Mär 07 16:41:43 HP845G9 systemd[1]: Starting systemd-suspend.service - System Suspend...
Mär 07 16:41:43 HP845G9 systemd-sleep[411382]: Entering sleep state 'suspend'...
Mär 07 16:45:34 HP845G9 systemd-sleep[411382]: System returned from sleep state.
Mär 07 16:45:34 HP845G9 systemd[1]: systemd-suspend.service: Deactivated successfully.
Mär 07 16:45:34 HP845G9 systemd[1]: Finished systemd-suspend.service - System Suspend.
Mär 07 16:46:03 HP845G9 systemd[1]: Starting systemd-suspend.service - System Suspend...
Mär 07 16:46:03 HP845G9 systemd-sleep[411864]: Entering sleep state 'suspend'...
Mär 07 16:46:33 HP845G9 systemd-sleep[411864]: System returned from sleep state.
Mär 07 16:46:34 HP845G9 systemd[1]: systemd-suspend.service: Deactivated successfully.
Mär 07 16:46:34 HP845G9 systemd[1]: Finished systemd-suspend.service - System Suspend.
Mär 07 16:47:03 HP845G9 systemd[1]: Starting systemd-suspend.service - System Suspend...
Mär 07 16:47:03 HP845G9 systemd-sleep[412307]: Entering sleep state 'suspend'...
Mär 07 16:56:36 HP845G9 systemd-sleep[412307]: System returned from sleep state.
Mär 07 16:56:37 HP845G9 systemd[1]: systemd-suspend.service: Deactivated successfully.
Mär 07 16:56:37 HP845G9 systemd[1]: Finished systemd-suspend.service - System Suspend.
Mär 07 16:57:06 HP845G9 systemd[1]: Starting systemd-suspend.service - System Suspend...
Mär 07 16:57:06 HP845G9 systemd-sleep[412764]: Entering sleep state 'suspend'...
Mär 07 16:57:38 HP845G9 systemd-sleep[412764]: System returned from sleep state.
Mär 07 16:57:38 HP845G9 systemd[1]: systemd-suspend.service: Deactivated successfully.
Mär 07 16:57:38 HP845G9 systemd[1]: Finished systemd-suspend.service - System Suspend.
Mär 07 16:58:07 HP845G9 systemd[1]: Starting systemd-suspend.service - System Suspend...
Mär 07 16:58:07 HP845G9 systemd-sleep[413181]: Entering sleep state 'suspend'...
Mär 07 16:58:44 HP845G9 systemd-sleep[413181]: System returned from sleep state.
Mär 07 16:58:45 HP845G9 systemd[1]: systemd-suspend.service: Deactivated successfully.
Mär 07 16:58:45 HP845G9 systemd[1]: Finished systemd-suspend.service - System Suspend.
Comment 17 Kai-Heng Feng 2024-03-15 01:21:47 UTC
Turns out it's quite hard to bisect. The system can't boot or freezes on suspend on many of the commits in bisection.
Comment 18 Kai-Heng Feng 2024-03-15 01:23:00 UTC
(In reply to Martin from comment #16)
> I am not sure if it is really fixed.
> (I updated to -rc7 now) 

Can you please run `echo 1 > /sys/power/pm_debug_messages`, reproduce the issue and attach the dmesg here?

Thanks!
Comment 19 Martin 2024-04-07 09:39:57 UTC
Created attachment 306102 [details]
dmesg after the problem occured again

dmesg with echo 1 > /sys/power/pm_debug_messages
Comment 20 Martin 2024-04-07 09:41:09 UTC
sorry, that it took so long, but the problem is not easy to reproduce. I am still not 100% sure when the problem returns. Is it possible that it has something to do with the removal / plugging in of the power cord?
Comment 21 Martin 2024-04-07 14:37:29 UTC
Created attachment 306104 [details]
extracted "PM:" lines from dmesg for better visibility

extracted "PM:" lines from dmesg for better visibility
Comment 22 Kai-Heng Feng 2024-04-09 04:11:19 UTC
Mario, can you please confirm it's still the same issue? "GPIO 17 is active: 0x30157a00" the GPIO value seems to be different to the older dmesg.

And I still couldn't figure out how to find the connection between GPIO 17 and GP 13, is there anyway to determine that from dmesg?
Comment 23 Mario Limonciello (AMD) 2024-04-09 11:16:08 UTC
Using https://gitlab.freedesktop.org/drm/amd/-/blob/master/scripts/amd_s2idle.py is better because it captures more system state information about active GPIO, IRQ and mapping out what they are.
Comment 24 Mario Limonciello (AMD) 2024-04-09 11:19:56 UTC
GPIO 17 is active comes from pinctrl-amd driver. GPIO values are set up from _AEI. So looking at _AEI/_EVT can see what ACPI devices are notified when it's active.

ACPI GP13 maps to PCI WWAN. As GP13 is notified it's pin for WWAN.
Comment 25 Mario Limonciello (AMD) 2024-04-09 12:07:17 UTC
Regarding GPIO values being different the difference is bit 29.

This indicates whether the GPIO was the very first thing to start the wakeup sequence.

GPIO bit 28 and 29 being active means that the interrupt is active and it's the first thing to wake the system.

GPIO bit 28 only being active means that the system already woke from hardware sleep from a different reason (will capture it in that report) but this GPIO is ALSO active.
Comment 26 Martin 2024-04-09 14:03:18 UTC
Created attachment 306113 [details]
I guess I am not on the proper firmware(?)

I am on Fedora 40 (beta) with all Updates installed. Shouldnt the firmware be already in Stable?
Comment 27 Mario Limonciello (AMD) 2024-04-09 14:44:13 UTC
I *think* that's output from an older version of the script that didn't know about some WLAN firmware versions for the WCN6855.
Comment 28 Martin 2024-04-09 16:19:51 UTC
yeah, you are right. 
I used the old one, when I tested for the error for the first time. I am sorry. I will run it again.

Am I allowed to close the lid, after I started it?
Comment 29 Mario Limonciello (AMD) 2024-04-09 16:33:08 UTC
Sure. But be cognizant it might be an easy way to trip this issue to close the lid after in suspend.
Comment 30 Kai-Heng Feng 2024-04-10 02:10:02 UTC
(In reply to Mario Limonciello (AMD) from comment #24)
> GPIO 17 is active comes from pinctrl-amd driver. GPIO values are set up from
> _AEI. So looking at _AEI/_EVT can see what ACPI devices are notified when
> it's active.
> 
> ACPI GP13 maps to PCI WWAN. As GP13 is notified it's pin for WWAN.

I wonder how the dots are connected without ACPI tables?
Comment 31 Mario Limonciello (AMD) 2024-04-10 03:39:30 UTC
You'll need a schematic for that.

I've used this technique on a number of bugs, it's been used to find floating pins, missing drivers, buggy drivers.
Comment 32 kolAflash 2024-04-12 11:30:58 UTC
Could this be related?

first hibernate to disk fails on HP EliteBook 845 G8 when Intel 7360 WWAN enabled in BIOS
https://bugzilla.kernel.org/show_bug.cgi?id=216647#c18

Hint:
Start reading that bug from the linked recent comment number 18 (2024-04-10).
The older comments are mostly obsolete.
Comment 33 Martin 2024-04-14 07:06:24 UTC
Created attachment 306148 [details]
s2idle with recent version + wakeup after a short time

I hope this is what you are looking for.
Comment 34 Kai-Heng Feng 2024-04-15 04:28:16 UTC
(In reply to Martin from comment #33)
> Created attachment 306148 [details]
> s2idle with recent version + wakeup after a short time
> 
> I hope this is what you are looking for.

Does this issue happen when `iosm` module is not loaded?
Comment 35 Martin 2024-04-17 21:35:46 UTC
I think you are right. I have blacklisted 'iosm' and my system still stays in standby as it should.
Comment 36 Martin 2024-09-26 21:20:29 UTC
Do you plan to fix this?
I am not able to use the LTE Modem for almost half a year now.