Bug 46421

Summary: tg3 (bcm57781): Immediate wakeup from S3 after 1st use of WOL - ASRock Z77 Extreme4 mainboard
Product: Drivers Reporter: Harald Judt (h.judt)
Component: NetworkAssignee: drivers_network (drivers_network)
Status: ASSIGNED ---    
Severity: normal CC: aaron.lu, alan, cyberynight, h.judt, jed-kernel.org, lenb, szg00000
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 3.5.1 Subsystem:
Regression: No Bisected commit-id:
Bug Depends on:    
Bug Blocks: 56331    
Attachments: Debug message to show which stage of suspend failed

Description Harald Judt 2012-08-24 19:01:42 UTC
I have an ASRock Z77 Extreme4 mainboard and set it up to wake it using Wake on LAN (WOL).

The same setup worked fine with my older mainboard ASRock P67 Extreme4.

Steps to reproduce:
1) Enable Wake on LAN via magic packet:
   ethtool -s eth0 wol g
2) Suspend computer (S3)
3) Wake up the computer via magic packet.
4) Suspend computer (S3)

Expected results:
Computer should be suspended.

Actual results:
Computer immediately wakes up again (POWER LED blinks a few times, then computer starts resuming). After it has resumed, you can successfully suspend it, without it waking up.

Disabling WOL after waking up with ethtool does not change anything.


04:00.0 Ethernet controller: Broadcom Corporation NetLink BCM57781 Gigabit Ethernet PCIe (rev 10)
        Subsystem: ASRock Incorporation Device 96b1
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 16
        Region 0: Memory at f0010000 (64-bit, prefetchable) [size=64K]
        Region 2: Memory at f0000000 (64-bit, prefetchable) [size=64K]
        Expansion ROM at f7a00000 [disabled] [size=2K]
        Capabilities: [48] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [a0] MSI-X: Enable+ Count=5 Masked-
                Vector table: BAR=2 offset=00000000
                PBA: BAR=2 offset=00000120
        Capabilities: [ac] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <4us, L1 <64us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr+ NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <2us, L1 <64us
                        ClockPM+ Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 14, GenCap+ CGenEn- ChkCap+ ChkEn-
        Capabilities: [13c v1] Device Serial Number 00-00-00-00-00-00-00-00
        Capabilities: [150 v1] Power Budgeting <?>
        Capabilities: [160 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed- WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=01
                        Status: NegoPending- InProgress-

Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Half 1000baseT/Full 
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Half 1000baseT/Full 
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Link partner advertised link modes:  10baseT/Half 10baseT/Full 
                                             100baseT/Half 100baseT/Full 
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Speed: 100Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: on
        Supports Wake-on: g
        Wake-on: g
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: yes

I'll attach dmesg in the next comment.
Comment 1 Harald Judt 2012-08-24 19:05:56 UTC
dmesg from 2 S3 attempts. After the first one, wake up via WOL. After the second one, system wakes again without any user interaction.

PM: Syncing filesystems ... done.
Freezing user space processes ... (elapsed 0.02 seconds) done.
Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
Suspending console(s) (use no_console_suspend to debug)
sd 2:0:0:0: [sdb] Synchronizing SCSI cache
sd 0:0:0:0: [sda] Synchronizing SCSI cache
sd 2:0:0:0: [sdb] Stopping disk
serial 00:0c: disabled
serial 00:0c: wake-up capability disabled by ACPI
i8042 kbd 00:0b: wake-up capability enabled by ACPI
sd 0:0:0:0: [sda] Stopping disk
ACPI handle has no context!
ACPI handle has no context!
PM: suspend of devices complete after 1121.778 msecs
PM: late suspend of devices complete after 0.111 msecs
pcieport 0000:00:1c.7: wake-up capability enabled by ACPI
pcieport 0000:00:1c.4: wake-up capability enabled by ACPI
ehci_hcd 0000:00:1d.0: wake-up capability enabled by ACPI
ehci_hcd 0000:00:1a.0: wake-up capability enabled by ACPI
xhci_hcd 0000:00:14.0: wake-up capability enabled by ACPI
PM: noirq suspend of devices complete after 54.927 msecs
ACPI: Preparing to enter system sleep state S3
PM: Saving platform NVS memory
Disabling non-boot CPUs ...
CPU 1 is now offline
CPU 2 is now offline
CPU 3 is now offline
CPU 4 is now offline
CPU 5 is now offline
CPU 6 is now offline
CPU 7 is now offline
Extended CMOS year: 2000
ACPI: Low-level resume complete
PM: Restoring platform NVS memory
Extended CMOS year: 2000
Enabling non-boot CPUs ...
Booting Node 0 Processor 1 APIC 0x2
CPU1 is up
Booting Node 0 Processor 2 APIC 0x4
CPU2 is up
Booting Node 0 Processor 3 APIC 0x6
CPU3 is up
Booting Node 0 Processor 4 APIC 0x1
CPU4 is up
Booting Node 0 Processor 5 APIC 0x3
CPU5 is up
Booting Node 0 Processor 6 APIC 0x5
CPU6 is up
Booting Node 0 Processor 7 APIC 0x7
CPU7 is up
ACPI: Waking up from system sleep state S3
xhci_hcd 0000:00:14.0: wake-up capability disabled by ACPI
ehci_hcd 0000:00:1a.0: wake-up capability disabled by ACPI
ehci_hcd 0000:00:1d.0: wake-up capability disabled by ACPI
pcieport 0000:00:1c.7: wake-up capability disabled by ACPI
PM: noirq resume of devices complete after 1.366 msecs
PM: early resume of devices complete after 0.028 msecs
xhci_hcd 0000:00:14.0: setting latency timer to 64
ehci_hcd 0000:00:1a.0: setting latency timer to 64
pcieport 0000:00:1c.4: wake-up capability disabled by ACPI
ehci_hcd 0000:00:1d.0: setting latency timer to 64
ahci 0000:00:1f.2: setting latency timer to 64
pci 0000:05:00.0: setting latency timer to 64
usb usb3: root hub lost power or was reset
usb usb6: root hub lost power or was reset
snd_hda_intel 0000:01:00.1: irq 57 for MSI/MSI-X
tg3 0000:04:00.0: eth0: Link is down
i8042 kbd 00:0b: wake-up capability disabled by ACPI
xhci_hcd 0000:07:00.0: irq 49 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 50 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 51 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 52 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 53 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 54 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 55 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 56 for MSI/MSI-X
serial 00:0c: activated
[drm] PCIE GART of 512M enabled (table at 0x0000000000141000).
radeon 0000:01:00.0: WB enabled
radeon 0000:01:00.0: fence driver on ring 0 use gpu addr 0x0000000080000c00 and cpu addr 0xffff88041becdc00
radeon 0000:01:00.0: fence driver on ring 1 use gpu addr 0x0000000080000c04 and cpu addr 0xffff88041becdc04
radeon 0000:01:00.0: fence driver on ring 2 use gpu addr 0x0000000080000c08 and cpu addr 0xffff88041becdc08
[drm] ring test on 0 succeeded in 0 usecs
[drm] ib test on ring 0 succeeded in 0 usecs
ata7: SATA link down (SStatus 0 SControl 300)
ata8: SATA link down (SStatus 0 SControl 300)
ata4: SATA link down (SStatus 0 SControl 300)
ata5: SATA link down (SStatus 0 SControl 300)
ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata6.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata6.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata6.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata2: SATA link down (SStatus 0 SControl 300)
ata6.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata6.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata6.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata6.00: configured for UDMA/100
tg3 0000:04:00.0: eth0: Link is up at 100 Mbps, full duplex
tg3 0000:04:00.0: eth0: Flow control is off for TX and off for RX
tg3 0000:04:00.0: eth0: EEE is disabled
ata3: link is slow to respond, please be patient (ready=0)
ata1: link is slow to respond, please be patient (ready=0)
ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata3.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata3.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata3.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata3.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata3.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata3.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata3.00: configured for UDMA/133
sd 2:0:0:0: [sdb] Starting disk
ata1: COMRESET failed (errno=-16)
ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata1.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata1.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata1.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata1.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata1.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata1.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata1.00: configured for UDMA/133
sd 0:0:0:0: [sda] Starting disk
PM: resume of devices complete after 12434.676 msecs
Restarting tasks ... done.
Adding 4194300k swap on /dev/sdb2.  Priority:-1 extents:1 across:4194300k 
PM: Syncing filesystems ... done.
Freezing user space processes ... (elapsed 0.01 seconds) done.
Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
Suspending console(s) (use no_console_suspend to debug)
sd 2:0:0:0: [sdb] Synchronizing SCSI cache
sd 0:0:0:0: [sda] Synchronizing SCSI cache
sd 2:0:0:0: [sdb] Stopping disk
serial 00:0c: disabled
serial 00:0c: wake-up capability disabled by ACPI
i8042 kbd 00:0b: wake-up capability enabled by ACPI
sd 0:0:0:0: [sda] Stopping disk
ACPI handle has no context!
ACPI handle has no context!
PM: suspend of devices complete after 994.760 msecs
PM: late suspend of devices complete after 0.111 msecs
pcieport 0000:00:1c.7: wake-up capability enabled by ACPI
pcieport 0000:00:1c.4: wake-up capability enabled by ACPI
ehci_hcd 0000:00:1d.0: wake-up capability enabled by ACPI
ehci_hcd 0000:00:1a.0: wake-up capability enabled by ACPI
xhci_hcd 0000:00:14.0: wake-up capability enabled by ACPI
PM: noirq suspend of devices complete after 54.935 msecs
ACPI: Preparing to enter system sleep state S3
ata6.00: configured for UDMA/100
tg3 0000:04:00.0: eth0: Link is up at 100 Mbps, full duplex
tg3 0000:04:00.0: eth0: Flow control is off for TX and off for RX
tg3 0000:04:00.0: eth0: EEE is disabled
ata3: link is slow to respond, please be patient (ready=0)
ata1: link is slow to respond, please be patient (ready=0)
ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata3.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata3.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata3.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata3.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata3.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata3.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata3.00: configured for UDMA/133
sd 2:0:0:0: [sdb] Starting disk
ata1: COMRESET failed (errno=-16)
ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata1.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata1.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata1.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata1.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata1.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata1.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata1.00: configured for UDMA/133
sd 0:0:0:0: [sda] Starting disk
PM: resume of devices complete after 12434.676 msecs
Restarting tasks ... done.
Adding 4194300k swap on /dev/sdb2.  Priority:-1 extents:1 across:4194300k 
PM: Syncing filesystems ... done.
Freezing user space processes ... (elapsed 0.01 seconds) done.
Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
Suspending console(s) (use no_console_suspend to debug)
sd 2:0:0:0: [sdb] Synchronizing SCSI cache
sd 0:0:0:0: [sda] Synchronizing SCSI cache
sd 2:0:0:0: [sdb] Stopping disk
serial 00:0c: disabled
serial 00:0c: wake-up capability disabled by ACPI
i8042 kbd 00:0b: wake-up capability enabled by ACPI
sd 0:0:0:0: [sda] Stopping disk
ACPI handle has no context!
ACPI handle has no context!
PM: suspend of devices complete after 994.760 msecs
PM: late suspend of devices complete after 0.111 msecs
pcieport 0000:00:1c.7: wake-up capability enabled by ACPI
pcieport 0000:00:1c.4: wake-up capability enabled by ACPI
ehci_hcd 0000:00:1d.0: wake-up capability enabled by ACPI
ehci_hcd 0000:00:1a.0: wake-up capability enabled by ACPI
xhci_hcd 0000:00:14.0: wake-up capability enabled by ACPI
PM: noirq suspend of devices complete after 54.935 msecs
ACPI: Preparing to enter system sleep state S3
PM: Saving platform NVS memory
Disabling non-boot CPUs ...
CPU 1 is now offline
CPU 2 is now offline
CPU 3 is now offline
CPU 4 is now offline
CPU 5 is now offline
CPU 6 is now offline
CPU 7 is now offline
Extended CMOS year: 2000
ACPI: Low-level resume complete
PM: Restoring platform NVS memory
Extended CMOS year: 2000
Enabling non-boot CPUs ...
Booting Node 0 Processor 1 APIC 0x2
CPU1 is up
Booting Node 0 Processor 2 APIC 0x4
CPU2 is up
Booting Node 0 Processor 3 APIC 0x6
CPU3 is up
Booting Node 0 Processor 4 APIC 0x1
CPU4 is up
Booting Node 0 Processor 5 APIC 0x3
CPU5 is up
Booting Node 0 Processor 6 APIC 0x5
CPU6 is up
Booting Node 0 Processor 7 APIC 0x7
CPU7 is up
ACPI: Waking up from system sleep state S3
xhci_hcd 0000:00:14.0: wake-up capability disabled by ACPI
ehci_hcd 0000:00:1a.0: wake-up capability disabled by ACPI
ehci_hcd 0000:00:1d.0: wake-up capability disabled by ACPI
pcieport 0000:00:1c.7: wake-up capability disabled by ACPI
PM: noirq resume of devices complete after 1.369 msecs
PM: early resume of devices complete after 0.028 msecs
xhci_hcd 0000:00:14.0: setting latency timer to 64
ehci_hcd 0000:00:1a.0: setting latency timer to 64
ehci_hcd 0000:00:1d.0: setting latency timer to 64
pcieport 0000:00:1c.4: wake-up capability disabled by ACPI
usb usb3: root hub lost power or was reset
pci 0000:05:00.0: setting latency timer to 64
usb usb6: root hub lost power or was reset
ahci 0000:00:1f.2: setting latency timer to 64
snd_hda_intel 0000:01:00.1: irq 57 for MSI/MSI-X
i8042 kbd 00:0b: wake-up capability disabled by ACPI
tg3 0000:04:00.0: eth0: Link is down
serial 00:0c: activated
xhci_hcd 0000:07:00.0: irq 49 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 50 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 51 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 52 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 53 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 54 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 55 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 56 for MSI/MSI-X
[drm] PCIE GART of 512M enabled (table at 0x0000000000141000).
radeon 0000:01:00.0: WB enabled
radeon 0000:01:00.0: fence driver on ring 0 use gpu addr 0x0000000080000c00 and cpu addr 0xffff88041becdc00
radeon 0000:01:00.0: fence driver on ring 1 use gpu addr 0x0000000080000c04 and cpu addr 0xffff88041becdc04
radeon 0000:01:00.0: fence driver on ring 2 use gpu addr 0x0000000080000c08 and cpu addr 0xffff88041becdc08
[drm] ring test on 0 succeeded in 2 usecs
[drm] ib test on ring 0 succeeded in 0 usecs
ata7: SATA link down (SStatus 0 SControl 300)
ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata8: SATA link down (SStatus 0 SControl 300)
ata4: SATA link down (SStatus 0 SControl 300)
ata2: SATA link down (SStatus 0 SControl 300)
ata5: SATA link down (SStatus 0 SControl 300)
ata6.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata6.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata6.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata6.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata6.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata6.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata6.00: configured for UDMA/100
tg3 0000:04:00.0: eth0: Link is up at 100 Mbps, full duplex
tg3 0000:04:00.0: eth0: Flow control is off for TX and off for RX
tg3 0000:04:00.0: eth0: EEE is disabled
ata1: link is slow to respond, please be patient (ready=0)
ata3: link is slow to respond, please be patient (ready=0)
ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata3.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata3.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata3.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata3.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata3.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata3.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata3.00: configured for UDMA/133
sd 2:0:0:0: [sdb] Starting disk
ata1: COMRESET failed (errno=-16)
ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata1.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata1.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata1.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata1.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata1.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata1.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata1.00: configured for UDMA/133
sd 0:0:0:0: [sda] Starting disk
PM: resume of devices complete after 11687.694 msecs
xhci_hcd 0000:07:00.0: irq 54 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 55 for MSI/MSI-X
xhci_hcd 0000:07:00.0: irq 56 for MSI/MSI-X
[drm] PCIE GART of 512M enabled (table at 0x0000000000141000).
radeon 0000:01:00.0: WB enabled
radeon 0000:01:00.0: fence driver on ring 0 use gpu addr 0x0000000080000c00 and cpu addr 0xffff88041becdc00
radeon 0000:01:00.0: fence driver on ring 1 use gpu addr 0x0000000080000c04 and cpu addr 0xffff88041becdc04
radeon 0000:01:00.0: fence driver on ring 2 use gpu addr 0x0000000080000c08 and cpu addr 0xffff88041becdc08
[drm] ring test on 0 succeeded in 2 usecs
[drm] ib test on ring 0 succeeded in 0 usecs
ata7: SATA link down (SStatus 0 SControl 300)
ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata8: SATA link down (SStatus 0 SControl 300)
ata4: SATA link down (SStatus 0 SControl 300)
ata2: SATA link down (SStatus 0 SControl 300)
ata5: SATA link down (SStatus 0 SControl 300)
ata6.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata6.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata6.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata6.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata6.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata6.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata6.00: configured for UDMA/100
tg3 0000:04:00.0: eth0: Link is up at 100 Mbps, full duplex
tg3 0000:04:00.0: eth0: Flow control is off for TX and off for RX
tg3 0000:04:00.0: eth0: EEE is disabled
ata1: link is slow to respond, please be patient (ready=0)
ata3: link is slow to respond, please be patient (ready=0)
ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata3.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata3.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata3.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata3.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata3.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata3.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata3.00: configured for UDMA/133
sd 2:0:0:0: [sdb] Starting disk
ata1: COMRESET failed (errno=-16)
ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata1.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata1.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata1.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata1.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
ata1.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
ata1.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
ata1.00: configured for UDMA/133
sd 0:0:0:0: [sda] Starting disk
PM: resume of devices complete after 11687.694 msecs
Restarting tasks ... done.
Adding 4194300k swap on /dev/sdb2.  Priority:-1 extents:1 across:4194300k
Comment 2 Harald Judt 2012-08-24 19:28:21 UTC
Additionally, it seems waking up from hibernation using WOL doesn't work at all with this board.

All works fine using Microsoft Windows and the same BIOS settings, so it's not a configuration problem.
Comment 3 Harald Judt 2012-08-24 19:36:18 UTC
This stays the same before and after waking with WOL:

for i in /sys/bus/pci/drivers/tg3/0000\:04\:00.0/power/wakeup*; do echo $i:$(cat $i); done
/sys/bus/pci/drivers/tg3/0000:04:00.0/power/wakeup:enabled
/sys/bus/pci/drivers/tg3/0000:04:00.0/power/wakeup_abort_count:0
/sys/bus/pci/drivers/tg3/0000:04:00.0/power/wakeup_active:0
/sys/bus/pci/drivers/tg3/0000:04:00.0/power/wakeup_active_count:0
/sys/bus/pci/drivers/tg3/0000:04:00.0/power/wakeup_count:0
/sys/bus/pci/drivers/tg3/0000:04:00.0/power/wakeup_expire_count:0
/sys/bus/pci/drivers/tg3/0000:04:00.0/power/wakeup_last_time_ms:133550565
/sys/bus/pci/drivers/tg3/0000:04:00.0/power/wakeup_max_time_ms:0
/sys/bus/pci/drivers/tg3/0000:04:00.0/power/wakeup_total_time_ms:0

Here is /proc/acpi/wakeup, but isn't this deprecated now?
cat /proc/acpi/wakeup 
Device  S-state   Status   Sysfs node
PS2K      S4    *enabled   pnp:00:0b
UAR1      S4    *disabled  pnp:00:0c
P0P1      S4    *disabled  
USB1      S3    *disabled  
USB2      S3    *disabled  
USB3      S3    *disabled  
USB4      S3    *disabled  
USB5      S3    *disabled  
USB6      S3    *disabled  
USB7      S3    *disabled  
RP01      S4    *disabled  pci:0000:00:1c.0
RP02      S4    *disabled  
RP03      S4    *disabled  
RP04      S4    *disabled  pci:0000:00:1c.3
RP05      S4    *disabled  pci:0000:00:1c.4
RP06      S4    *disabled  pci:0000:00:1c.5
BR40      S4    *disabled  pci:0000:05:00.0
RP07      S4    *disabled  
RP08      S4    *disabled  pci:0000:00:1c.7
PEG0      S4    *disabled  pci:0000:00:01.0
PEGP      S4    *disabled  
PEG1      S4    *disabled  
PEG2      S4    *disabled  
PEG3      S4    *disabled  
GLAN      S4    *disabled  
EHC1      S4    *enabled   pci:0000:00:1d.0
EHC2      S4    *enabled   pci:0000:00:1a.0
XHC       S4    *enabled   pci:0000:00:14.0
HDEF      S4    *disabled

BIOS settings:
Suspend to RAM: Auto
Check Ready Bit: Enabled
ACPI HPET Table: Enabled

PS/2 Keyboard Power On: Disabled
PCI Devices Power On: Enabled
Ring-In Power On: Disabled
RTC Alarm Power On: By OS
USB Keyboard/Remote Power On: Disabled
USB Mouse Power On: Disabled
Comment 4 Harald Judt 2012-08-24 20:04:56 UTC
GLAN      S4    *disabled

Is this supposed to be the network adapter? It doesn't have any pci assigned. "echo GLAN > /proc/acpi/wakeup" doesn't work, it stays disabled. However, PS2K can be changed to disabled.
Comment 5 Len Brown 2013-01-30 20:49:03 UTC
Re: /proc/acpi/wakeup

Actually, it should still be effective.
The problem with is is that the ACPI name is arbitrary,
so unless you've got the PCI name on that line, it can
be confusing what device is what -- unless you dig into the AML.

Re: no wake from hibernate

Note that none of the wake devices in /proc/acpi/wakeup
advertise wake from S5.  /sys/power/disk will tell you which
is being used when you hibernate to disk.  "platform" means
you will enter ACPI S4.  "shutdown" will be S5.

In any case
04:00.0 Ethernet controller: Broadcom Corporation NetLink BCM57781 Gigabit
Ethernet PCIe

Does not appear in /proc/acpi/wakeup

it is apparently an add-in PCIe device that ACPI doesn't know about.

It appears that this is a tg3 bug.
Comment 6 Aaron Lu 2013-02-20 03:09:04 UTC
Hello Harald,

Is it possible for you to test on 3.8 and see if you still have this problem?

From the log, it is highly possible the 2nd suspend failed due to prepare_late failed if the log is complete. It would be good if you can attach the full dmesg instead of posting it here, thanks.
Comment 7 Aaron Lu 2013-02-20 03:12:00 UTC
Created attachment 93641 [details]
Debug message to show which stage of suspend failed

In case you still have the same problem in v3.8, please apply this patch and then attach the dmesg after suspend failed, let's see why it failed, thanks.
Comment 8 Harald Judt 2013-02-20 08:30:00 UTC
Thanks for your suggestions. Yes, I will try with 3.8, if necessary with your patch applied.

Some questions though:
First, if suspend fails, why does it only fail when the machine has been awoken by a WOL package before? And why is suspend successful on the next try?

And question number two: The computer still seems to go into S3 (the power LED is blinking, fans spin down, devices seem to turn off), but awakes again after 1 or 2 seconds. Observing this behaviour makes me believe suspend happens successfully, because it wouldn't shut down otherwise, right?

I tend to claim the tg3 driver is at fault here, but there is one strange thing: If I turn off PCIe wakeup events (is that possible via OS?), shouldn't the NIC - as an add-on card - be unable to wake up the machine by WOL?

Anyway, maybe 3.8 fixes this, I will try later this day.
Comment 9 Aaron Lu 2013-02-20 09:05:13 UTC
(In reply to comment #8)
> Some questions though:
> First, if suspend fails, why does it only fail when the machine has been
> awoken
> by a WOL package before? And why is suspend successful on the next try?

That's why I added some debug message to see what the error code is, if the 2nd suspend indeed failed.

> 
> And question number two: The computer still seems to go into S3 (the power
> LED
> is blinking, fans spin down, devices seem to turn off), but awakes again
> after
> 1 or 2 seconds. Observing this behaviour makes me believe suspend happens
> successfully, because it wouldn't shut down otherwise, right?

Right. But from the dmesg you have posted, it seems to me the suspend is aborted due to some error in the process.

Normally, after the below message:
ACPI: Preparing to enter system sleep state S3
It should be followed by:
PM: Saving platform NVS memory
Disabling non-boot CPUs ...

But in the 2nd suspend, it is followed by the sata port being resumed.
So I thought the suspend failed somewhere and kernel decides to resume back the system.

> 
> I tend to claim the tg3 driver is at fault here, but there is one strange
> thing: If I turn off PCIe wakeup events (is that possible via OS?), shouldn't
> the NIC - as an add-on card - be unable to wake up the machine by WOL?

I need to verify this.

> 
> Anyway, maybe 3.8 fixes this, I will try later this day.

Thanks!
Comment 10 Harald Judt 2013-02-20 10:04:16 UTC
Thanks for the explanation, I did not know the kernel would be able to resume at that point. Unfortunately it came to my mind that I cannot test suspend & resume with kernels >= 7.0rc2 because of an unfixed radeon drm bug which crashes the machine after resume (reported here: https://bugs.freedesktop.org/show_bug.cgi?id=60439). Until that other issue gets fixed, I will try to apply your patch to 3.6.11, maybe it will still reveal something.
Comment 11 Harald Judt 2013-02-20 19:17:59 UTC
I applied the patch to 3.7-rc2 and performed the steps as done previously, but not one of the debugging messages from your patch did show up.

What's more, I tried enabling and disabling all PCIe stuff in /proc/acpi/wakeup:
RP01      S4    *disabled  pci:0000:00:1c.0
RP04      S4    *disabled  pci:0000:00:1c.3
RP05      S4    *disabled  pci:0000:00:1c.4
RP06      S4    *disabled  pci:0000:00:1c.5
BR40      S4    *disabled  pci:0000:05:00.0
RP08      S4    *disabled  pci:0000:00:1c.7

...except "PEG0      S4    *disabled  pci:0000:00:01.0", which I forgot but which was disabled anyway.

If I do not enable RP05 and/or BR40, WOL won't work. However, disabling them and disabling WOL with "ethtool -s eth0 d" doesn't have any effect, even though ethtool reports "Wake-on d".
Comment 12 Harald Judt 2013-02-20 19:18:25 UTC
Sorry, of course I meant 3.7-rc1, not rc2.
Comment 13 Aaron Lu 2013-02-21 02:55:37 UTC
(In reply to comment #11)
> I applied the patch to 3.7-rc2 and performed the steps as done previously,
> but
> not one of the debugging messages from your patch did show up.

That means the suspend succeeded, because the debug message I added will only be printed out when some operation failed in the process of suspending.

> 
> What's more, I tried enabling and disabling all PCIe stuff in
> /proc/acpi/wakeup:
> RP01      S4    *disabled  pci:0000:00:1c.0
> RP04      S4    *disabled  pci:0000:00:1c.3
> RP05      S4    *disabled  pci:0000:00:1c.4
> RP06      S4    *disabled  pci:0000:00:1c.5
> BR40      S4    *disabled  pci:0000:05:00.0
> RP08      S4    *disabled  pci:0000:00:1c.7
> 
> ...except "PEG0      S4    *disabled  pci:0000:00:01.0", which I forgot but
> which was disabled anyway.
> 
> If I do not enable RP05 and/or BR40, WOL won't work. However, disabling them
> and disabling WOL with "ethtool -s eth0 d" doesn't have any effect, even
> though
> ethtool reports "Wake-on d".

Do you mean after a suspend-resume cycle, if you disable those wakeup devices in /proc/acpi/wakeup and disable WOL with ethtool for the Ethernet card, the 2nd suspend will still be followed by an automatic resume?
Comment 14 Harald Judt 2013-02-21 06:51:57 UTC
Yes. All those RP0x and BR40 devices are disabled in /proc/acpi/wakeup by default actually, and I have to enable them or WOL won't work at all. Only specific devices might need to be enabled, but I forgot which ones they would be, so I simply enabled all of them for the last tests. Then suspend, wake up by WOL, disable all stuff, deactivate WOL with ethtool, suspend => computer is off for 3 secs => automatic resume => suspend => computer off.

[I erroneously thought that setting WOL to "d" with ethtool wouldn't work (I thought it would stay at "g"), but in fact this works even if "d" is not listed as a supported option, in contrast to how it is done by other network cards. However, it has no influence on the automatic resume so _I guess_ we can neglect that here.]
Comment 15 Aaron Lu 2013-02-22 05:26:05 UTC
Hi Harald,

I'm not aware of a way to identify which device caused a system resume through software, or we will be able to decide if it is the ethernet card that caused the automatic resume.

Perhaps we can involve some drivers guy to see if they have some idea.
Comment 16 Harald Judt 2013-03-17 21:24:57 UTC
FYI, the problem is still reproducible with 3.8.0.

It seems to suffice to enable wakeup only for RP05 (pci:0000:00:1c.4) and turn on WOL using ethtool. 0000:04:00.0 is the network card and it appears below

/sys/devices/pci0000:00/0000:00:1c.4 # ls -1
0000:00:1c.4:pcie01
0000:04:00.0
...

I will test again if disabling wakeup for RP05 after the first resume will prohibit the next WOL. It would not be a solution, however; Turning it off _and_ back on again before the second suspend would not prevent the undesired wakeup and if it did, WOL would not be possible anymore.
Comment 17 Harald Judt 2013-03-18 12:38:51 UTC
Ok, as announced I did the following test:

1) Enable wakeup for RP05.
2) Activate WOL using ethtool.
3) Suspend.
4) Resume by WOL packet.
5) Disable RP05 using "echo RP05 > /proc/acpi/wakeup", verified => state "Disabled".
6) Disable WOL using ethtool, verified => state "d".
7) Suspend.

Result: Computer suspends but wakes up after ~3 seconds without user interaction.
Comment 18 Harald Judt 2014-01-08 18:53:08 UTC
Problem still exists with 3.12.0. Does no one have any idea how to narrow it down at least to a certain subsystem?
Comment 19 Harald Judt 2014-03-15 16:17:08 UTC
Problem still exists with 3.13.5.

I've built tg3 as a module and unloaded it before the second suspend. Still, the machine did wake up again.
Comment 20 Harald Judt 2014-04-25 11:29:11 UTC
Interestingly, with 3.14.1 I now have this problem after every hibernation even if I don't use WOL. I'll try to bisect, maybe it gives some clue what's going wrong here.
Comment 21 Hubert 2014-11-24 00:28:23 UTC
Hello Harald,

I have exactly the same problem. My motherboard is ASRock E3C226D2I with 2 x IntelĀ® i210 (Linux driver igb). Debian Jessy.

The system wakes up immediately after the second suspend or hibernate. Even unloading the driver with modprove -r igb and reloading didn't help. I guess it is a hardware/firmware problem?

Did you find any workaround?
Comment 22 Harald Judt 2014-11-24 11:39:29 UTC
(In reply to Harald Judt from comment #20)
> Interestingly, with 3.14.1 I now have this problem after every hibernation
> even if I don't use WOL. I'll try to bisect, maybe it gives some clue what's
> going wrong here.

I have to revert this observation. This happened because the new mouse, simply when being moved, made the machine wake up, in contrast to the old mouse which would only cause that on a mouse click.

(In reply to Hubert from comment #21)
> Hello Harald,
> 
> I have exactly the same problem. My motherboard is ASRock E3C226D2I with 2 x
> IntelĀ® i210 (Linux driver igb). Debian Jessy.

This probably means it is not a problem in any network drivers, but might be rather a firmware problem or - because it's only a problem with linux and not with windows - a problem with ACPI.

And no, I haven't found any workaround.
Comment 23 Hubert 2014-11-26 13:50:06 UTC
For me this happens with pm-suspend, pm-hibernate or even "echo -n mem > /sys/power/state"; no difference.

My "workaround" is, I put a reboot command into the pm-suspend resume script, so the server now reboots each time directly after it is woken with WoL.

The problem is that I have six hard drives connected to the server and the need to suspend it "two times in a row" to finally suspend makes those hard drives spin-down / spin-up once more which is very bad (spinning-up and down is most stressful for mechanical hard drives.)


So we have to identify first, what causes the problem. As Harald suggested, it seems the "WoL" package flag does not get cleared after the first wake-up. Does anybody with technical knowledge know where it is being stored? Can't we just "overwrite" or "reset" that part somewhere hidden in RAM/ACPI after a wake-up? ;-)

As I already tried, removing the driver and unloading eth with ifconfig didn't help...