Created attachment 66622 [details] dmesg output [1.] One line summary of the problem: Kernel boot hangs, usually for about a minute, sometimes apparently for good. [2.] Full description of the problem/report: Since about 2.6.38, boot times have been getting steadily longer on my machine, with occasionally the boot appearing to hang altogether. linux-3.0.0 reports an RCU stall, which seems to be associated with the hang in the dmesg, which is attached. Hopefully "scheduler" is the right place to report this: I'm sure which section RCU bugs fall under in general, but this particular one hits tick_sched_timer in the backtrace so it seems a reasonable first stab. Apologies if I've got it wrong. [3.] Keywords (i.e., modules, networking, kernel): RCU, kernel [4.] Kernel version (from /proc/version): Linux version 3.0.0-1-amd64 (Debian 3.0.0-1) (ben@decadent.org.uk) (gcc version 4.5.3 (Debian 4.5.3-4) ) #1 SMP Sun Jul 24 02:24:44 UTC 2011 [5.] Output of Oops.. message (if applicable) with symbolic information resolved (see Documentation/oops-tracing.txt) See dmesg attached. [6.] A small shell script or example program which triggers the problem (if possible) Happens on boot. [7.] Environment [7.1.] Software (add the output of the ver_linux script here) Linux xanthus 3.0.0-1-amd64 #1 SMP Sun Jul 24 02:24:44 UTC 2011 x86_64 GNU/Linux Gnu C 4.6.1 Gnu make 3.81 binutils 2.21.53.20110720 1.11 util-linux 2.17.2 mount support module-init-tools 3.12 e2fsprogs 1.41.12 PPP 2.4.5 Linux C Library 2.13 Dynamic linker (ldd) 2.13 Procps 3.2.8 Net-tools 1.60 Kbd 1.15.2 Sh-utils 8.5 Modules Loaded ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables cpufreq_stats cpufreq_powersave ppdev cpufreq_conservative lp cpufreq_userspace powernow_k8 mperf bridge stp bnep rfcomm bluetooth rfkill binfmt_misc fuse ext2 dm_snapshot it87 hwmon_vid loop firewire_sbp2 kvm_amd kvm snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_pcm radeon ttm drm_kms_helper snd_seq snd_timer snd_seq_device drm snd edac_core sp5100_tco i2c_algo_bit soundcore tpm_tis tpm i2c_piix4 k10temp i2c_core button processor edac_mce_amd tpm_bios snd_page_alloc pcspkr evdev power_supply parport_pc wmi parport thermal_sys serio_raw ext4 mbcache jbd2 crc16 usbhid hid dm_mirror dm_region_hash dm_log dm_mod sg sr_mod sd_mod cdrom crc_t10dif ata_generic ohci_hcd ahci libahci pata_atiixp firewire_ohci firewire_core crc_itu_t libata ehci_hcd scsi_mod usbcore r8169 mii [7.2.] Processor information (from /proc/cpuinfo): processor : 0 vendor_id : AuthenticAMD cpu family : 16 model : 5 model name : AMD Athlon(tm) II X4 630 Processor stepping : 2 cpu MHz : 800.000 cache size : 512 KB physical id : 0 siblings : 4 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt npt lbrv svm_lock nrip_save bogomips : 5626.04 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate (+ 3 more cpus the same) [7.3.] Module information (from /proc/modules): ip6table_filter 12540 0 - Live 0xffffffffa0555000 ip6_tables 22263 1 ip6table_filter, Live 0xffffffffa054a000 ebtable_nat 12580 0 - Live 0xffffffffa0545000 ebtables 26250 1 ebtable_nat, Live 0xffffffffa0539000 ipt_MASQUERADE 12635 3 - Live 0xffffffffa052f000 iptable_nat 12928 1 - Live 0xffffffffa0534000 nf_nat 18306 2 ipt_MASQUERADE,iptable_nat, Live 0xffffffffa0529000 nf_conntrack_ipv4 18176 4 iptable_nat,nf_nat, Live 0xffffffffa051a000 nf_defrag_ipv4 12483 1 nf_conntrack_ipv4, Live 0xffffffffa0515000 xt_state 12503 1 - Live 0xffffffffa0520000 nf_conntrack 57034 5 ipt_MASQUERADE,iptable_nat,nf_nat,nf_conntrack_ipv4,xt_state, Live 0xffffffffa0506000 ipt_REJECT 12502 2 - Live 0xffffffffa0501000 xt_tcpudp 12570 4 - Live 0xffffffffa04fc000 iptable_filter 12536 1 - Live 0xffffffffa04f7000 ip_tables 22131 2 iptable_nat,iptable_filter, Live 0xffffffffa04e7000 x_tables 19024 10 ip6table_filter,ip6_tables,ebtables,ipt_MASQUERADE,iptable_nat,xt_state,ipt_REJECT,xt_tcpudp,iptable_filter,ip_tables, Live 0xffffffffa04f1000 cpufreq_stats 12862 0 - Live 0xffffffffa04e2000 cpufreq_powersave 12454 0 - Live 0xffffffffa04dd000 ppdev 12763 0 - Live 0xffffffffa04d8000 cpufreq_conservative 13147 0 - Live 0xffffffffa04c8000 lp 17270 0 - Live 0xffffffffa04d2000 cpufreq_userspace 12576 0 - Live 0xffffffffa04cd000 powernow_k8 17688 0 - Live 0xffffffffa03d6000 mperf 12453 1 powernow_k8, Live 0xffffffffa04c3000 bridge 66612 0 - Live 0xffffffffa04b1000 stp 12435 1 bridge, Live 0xffffffffa0395000 bnep 17615 2 - Live 0xffffffffa03d0000 rfcomm 33848 0 - Live 0xffffffffa04a7000 bluetooth 114376 10 bnep,rfcomm, Live 0xffffffffa048a000 rfkill 19080 3 bluetooth, Live 0xffffffffa0385000 binfmt_misc 13040 1 - Live 0xffffffffa0280000 fuse 62334 3 - Live 0xffffffffa0479000 ext2 63732 1 - Live 0xffffffffa0468000 dm_snapshot 32809 3 - Live 0xffffffffa038b000 it87 30739 0 - Live 0xffffffffa037c000 hwmon_vid 12388 1 it87, Live 0xffffffffa029b000 loop 22711 0 - Live 0xffffffffa025a000 firewire_sbp2 18062 0 - Live 0xffffffffa0271000 kvm_amd 51180 0 - Live 0xffffffffa03c2000 kvm 268970 1 kvm_amd, Live 0xffffffffa0425000 snd_hda_codec_hdmi 26321 1 - Live 0xffffffffa0293000 snd_hda_codec_realtek 278027 1 - Live 0xffffffffa03e0000 snd_hda_intel 26140 2 - Live 0xffffffffa0234000 snd_hda_codec 72699 3 snd_hda_codec_hdmi,snd_hda_codec_realtek,snd_hda_intel, Live 0xffffffffa03af000 snd_hwdep 13186 1 snd_hda_codec, Live 0xffffffffa0217000 snd_pcm 68104 3 snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec, Live 0xffffffffa039d000 radeon 648372 2 - Live 0xffffffffa02dc000 ttm 52979 1 radeon, Live 0xffffffffa0285000 drm_kms_helper 27216 1 radeon, Live 0xffffffffa0278000 snd_seq 45208 0 - Live 0xffffffffa0264000 snd_timer 22581 2 snd_pcm,snd_seq, Live 0xffffffffa021c000 snd_seq_device 13137 1 snd_seq, Live 0xffffffffa0207000 drm 163231 4 radeon,ttm,drm_kms_helper, Live 0xffffffffa02b3000 snd 52823 13 snd_hda_codec_hdmi,snd_hda_codec_realtek,snd_hda_intel,snd_hda_codec,snd_hwdep,snd_pcm,snd_seq,snd_timer,snd_seq_device, Live 0xffffffffa024c000 edac_core 35343 0 - Live 0xffffffffa01f4000 sp5100_tco 12900 0 - Live 0xffffffffa01e2000 i2c_algo_bit 12850 1 radeon, Live 0xffffffffa01e8000 soundcore 13152 1 snd, Live 0xffffffffa023e000 tpm_tis 13152 0 - Live 0xffffffffa0247000 tpm 17819 1 tpm_tis, Live 0xffffffffa01bd000 i2c_piix4 12536 0 - Live 0xffffffffa02a8000 k10temp 12611 0 - Live 0xffffffffa02ae000 i2c_core 23909 5 radeon,drm_kms_helper,drm,i2c_algo_bit,i2c_piix4, Live 0xffffffffa02a1000 button 12930 0 - Live 0xffffffffa01ef000 processor 27942 1 powernow_k8, Live 0xffffffffa022c000 edac_mce_amd 17103 0 - Live 0xffffffffa0226000 tpm_bios 12944 1 tpm, Live 0xffffffffa0212000 snd_page_alloc 13043 2 snd_hda_intel,snd_pcm, Live 0xffffffffa01dd000 pcspkr 12579 0 - Live 0xffffffffa01d5000 evdev 17558 7 - Live 0xffffffffa01b7000 power_supply 13475 1 radeon, Live 0xffffffffa01ae000 parport_pc 22395 1 - Live 0xffffffffa01ce000 wmi 13243 0 - Live 0xffffffffa020d000 parport 31929 3 ppdev,lp,parport_pc, Live 0xffffffffa01fe000 thermal_sys 17949 1 processor, Live 0xffffffffa01c4000 serio_raw 12846 0 - Live 0xffffffffa013e000 ext4 313917 2 - Live 0xffffffffa0160000 mbcache 13066 2 ext2,ext4, Live 0xffffffffa00ef000 jbd2 62574 1 ext4, Live 0xffffffffa014f000 crc16 12343 2 bluetooth,ext4, Live 0xffffffffa00d5000 usbhid 40516 0 - Live 0xffffffffa0144000 hid 73172 1 usbhid, Live 0xffffffffa012b000 dm_mirror 17643 0 - Live 0xffffffffa00e9000 dm_region_hash 13506 1 dm_mirror, Live 0xffffffffa00af000 dm_log 13577 2 dm_mirror,dm_region_hash, Live 0xffffffffa005e000 dm_mod 63164 22 dm_snapshot,dm_mirror,dm_log, Live 0xffffffffa011a000 sg 25985 0 - Live 0xffffffffa00e1000 sr_mod 21811 0 - Live 0xffffffffa00da000 sd_mod 36259 4 - Live 0xffffffffa00cb000 cdrom 35093 1 sr_mod, Live 0xffffffffa00bc000 crc_t10dif 12348 1 sd_mod, Live 0xffffffffa00c6000 ata_generic 12479 0 - Live 0xffffffffa0013000 ohci_hcd 26576 0 - Live 0xffffffffa00b4000 ahci 25089 3 - Live 0xffffffffa0070000 libahci 22767 1 ahci, Live 0xffffffffa0065000 pata_atiixp 12697 0 - Live 0xffffffffa004d000 firewire_ohci 35553 0 - Live 0xffffffffa0054000 firewire_core 48371 2 firewire_sbp2,firewire_ohci, Live 0xffffffffa0040000 crc_itu_t 12347 1 firewire_core, Live 0xffffffffa0039000 libata 149043 4 ata_generic,ahci,libahci,pata_atiixp, Live 0xffffffffa00f4000 ehci_hcd 40090 0 - Live 0xffffffffa00a4000 scsi_mod 162442 5 firewire_sbp2,sg,sr_mod,sd_mod,libata, Live 0xffffffffa007b000 usbcore 124242 4 usbhid,ohci_hcd,ehci_hcd, Live 0xffffffffa0019000 r8169 42411 0 - Live 0xffffffffa0007000 mii 12675 1 r8169, Live 0xffffffffa0000000 [7.4.] Loaded driver and hardware information (/proc/ioports, /proc/iomem) $ cat /proc/ioports 0000-0cf7 : PCI Bus 0000:00 0000-001f : dma1 0020-0021 : pic1 0040-0043 : timer0 0050-0053 : timer1 0060-0060 : keyboard 0064-0064 : keyboard 0070-0073 : rtc0 0080-008f : dma page reg 00a0-00a1 : pic2 00c0-00df : dma2 00f0-00ff : fpu 0170-0177 : 0000:00:14.1 0170-0177 : pata_atiixp 01f0-01f7 : 0000:00:14.1 01f0-01f7 : pata_atiixp 0220-0225 : pnp 00:01 0228-022f : pnp 00:02 022d-022e : it87 022d-022e : it87 0238-023f : pnp 00:02 0290-0294 : pnp 00:01 0376-0376 : 0000:00:14.1 0376-0376 : pata_atiixp 0378-037a : parport0 03c0-03df : vga+ 03f6-03f6 : 0000:00:14.1 03f6-03f6 : pata_atiixp 03f8-03ff : serial 040b-040b : pnp 00:02 04d0-04d1 : pnp 00:01 04d6-04d6 : pnp 00:02 0b00-0b0f : pnp 00:02 0b00-0b07 : piix4_smbus 0b10-0b1f : pnp 00:02 0b20-0b3f : pnp 00:02 0c00-0c01 : pnp 00:02 0c14-0c14 : pnp 00:02 0c50-0c52 : pnp 00:02 0c6c-0c6d : pnp 00:02 0c6f-0c6f : pnp 00:02 0cd0-0cd1 : pnp 00:02 0cd2-0cd3 : pnp 00:02 0cd4-0cdf : pnp 00:02 0cf8-0cff : PCI conf1 0d00-ffff : PCI Bus 0000:00 4000-40fe : pnp 00:02 4000-4003 : ACPI PM1a_EVT_BLK 4004-4005 : ACPI PM1a_CNT_BLK 4008-400b : ACPI PM_TMR 4010-4015 : ACPI CPU throttle 4020-4027 : ACPI GPE0_BLK 4050-4050 : ACPI PM2_CNT_BLK 4100-411f : pnp 00:02 4210-4217 : pnp 00:02 c000-cfff : PCI Bus 0000:03 d000-dfff : PCI Bus 0000:02 de00-deff : 0000:02:00.0 de00-deff : r8169 e000-efff : PCI Bus 0000:01 ee00-eeff : 0000:01:00.0 fa00-fa0f : 0000:00:14.1 fa00-fa0f : pata_atiixp fb00-fb0f : 0000:00:11.0 fb00-fb0f : ahci fc00-fc03 : 0000:00:11.0 fc00-fc03 : ahci fd00-fd07 : 0000:00:11.0 fd00-fd07 : ahci fe00-fe03 : 0000:00:11.0 fe00-fe03 : ahci ff00-ff07 : 0000:00:11.0 ff00-ff07 : ahci $ cat /proc/iomem 00000000-0000ffff : reserved 00010000-000983ff : System RAM 00098400-0009ffff : reserved 000a0000-000bffff : PCI Bus 0000:00 000c0000-000dffff : PCI Bus 0000:00 000c0000-000cf7ff : Video ROM 000f0000-000fffff : reserved 000f0000-000fffff : System ROM 00100000-cfd80fff : System RAM 01000000-0133f2e2 : Kernel code 0133f2e3-0168a07f : Kernel data 0171b000-017f7fff : Kernel bss c4000000-c7ffffff : GART cfd81000-cfddffff : RAM buffer cfde0000-cfde2fff : ACPI Non-volatile Storage cfde3000-cfdeffff : ACPI Tables cfdf0000-cfdfffff : reserved cfdf0000-cfdfffff : pnp 00:0b cfe00000-cfefffff : RAM buffer cff00000-febfffff : PCI Bus 0000:00 d0000000-dfffffff : PCI Bus 0000:01 d0000000-dfffffff : 0000:01:00.0 e0000000-efffffff : PCI MMCONFIG 0000 [bus 00-ff] e0000000-efffffff : reserved e0000000-efffffff : pnp 00:0a fdb00000-fdbfffff : PCI Bus 0000:02 fdb00000-fdb0ffff : 0000:02:00.0 fdbe0000-fdbeffff : 0000:02:00.0 fdbe0000-fdbeffff : r8169 fdbff000-fdbfffff : 0000:02:00.0 fdbff000-fdbfffff : r8169 fdc00000-fdcfffff : PCI Bus 0000:03 fdd00000-fddfffff : PCI Bus 0000:03 fddf8000-fddfbfff : 0000:03:0e.0 fddff000-fddff7ff : 0000:03:0e.0 fddff000-fddff7ff : firewire_ohci fde00000-fdefffff : PCI Bus 0000:02 fdf00000-fdffffff : PCI Bus 0000:01 fdf00000-fdf1ffff : 0000:01:00.0 fdfc0000-fdfdffff : 0000:01:00.0 fdffc000-fdffffff : 0000:01:00.1 fdffc000-fdffffff : ICH HD audio fe024000-fe027fff : 0000:00:14.2 fe024000-fe027fff : ICH HD audio fe028000-fe028fff : 0000:00:14.5 fe028000-fe028fff : ohci_hcd fe029000-fe0290ff : 0000:00:13.2 fe029000-fe0290ff : ehci_hcd fe02a000-fe02afff : 0000:00:13.1 fe02a000-fe02afff : ohci_hcd fe02b000-fe02bfff : 0000:00:13.0 fe02b000-fe02bfff : ohci_hcd fe02c000-fe02c0ff : 0000:00:12.2 fe02c000-fe02c0ff : ehci_hcd fe02d000-fe02dfff : 0000:00:12.1 fe02d000-fe02dfff : ohci_hcd fe02e000-fe02efff : 0000:00:12.0 fe02e000-fe02efff : ohci_hcd fe02f000-fe02f3ff : 0000:00:11.0 fe02f000-fe02f3ff : ahci fec00000-ffffffff : reserved fec00000-fec003ff : IOAPIC 0 fed00000-fed003ff : HPET 0 fee00000-fee00fff : Local APIC fee00400-fee00fff : pnp 00:02 fff80000-fffeffff : pnp 00:0b ffff0000-ffffffff : pnp 00:0b 100000000-12fffffff : System RAM [7.5.] PCI information ('lspci -vvv' as root) root@xanthus:/home/phil# lspci -vvv 00:00.0 Host bridge: Advanced Micro Devices [AMD] RS780 Host Bridge Subsystem: Advanced Micro Devices [AMD] RS780 Host Bridge Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx- Latency: 32 Region 3: Memory at <ignored> (64-bit, non-prefetchable) Capabilities: [c4] HyperTransport: Slave or Primary Interface Command: BaseUnitID=0 UnitCnt=12 MastHost- DefDir- DUL- Link Control 0: CFlE- CST- CFE- <LkFail- Init+ EOC- TXO- <CRCErr=0 IsocEn+ LSEn- ExtCTL- 64b- Link Config 0: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=16bit DwFcInEn- LWO=16bit DwFcOutEn- Link Control 1: CFlE- CST- CFE- <LkFail+ Init- EOC+ TXO+ <CRCErr=0 IsocEn- LSEn- ExtCTL- 64b- Link Config 1: MLWI=8bit DwFcIn- MLWO=8bit DwFcOut- LWI=8bit DwFcInEn- LWO=8bit DwFcOutEn- Revision ID: 3.00 Link Frequency 0: [b] Link Error 0: <Prot+ <Ovfl- <EOC- CTLTm- Link Frequency Capability 0: 200MHz+ 300MHz- 400MHz+ 500MHz- 600MHz+ 800MHz+ 1.0GHz+ 1.2GHz- 1.4GHz- 1.6GHz- Vend- Feature Capability: IsocFC- LDTSTOP+ CRCTM- ECTLT- 64bA- UIDRD- Link Frequency 1: 200MHz Link Error 1: <Prot- <Ovfl- <EOC- CTLTm- Link Frequency Capability 1: 200MHz- 300MHz- 400MHz- 500MHz- 600MHz- 800MHz- 1.0GHz- 1.2GHz- 1.4GHz- 1.6GHz- Vend- Error Handling: PFlE- OFlE- PFE- OFE- EOCFE- RFE- CRCFE- SERRFE- CF- RE- PNFE- ONFE- EOCNFE- RNFE- CRCNFE- SERRNFE- Prefetchable memory behind bridge Upper: 00-00 Bus Number: 00 Capabilities: [54] HyperTransport: UnitID Clumping Capabilities: [40] HyperTransport: Retry Mode Capabilities: [9c] HyperTransport: #1a Capabilities: [f8] HyperTransport: #1c 00:02.0 PCI bridge: Advanced Micro Devices [AMD] RS780 PCI to PCI bridge (ext gfx port 0) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 0000e000-0000efff Memory behind bridge: fdf00000-fdffffff Prefetchable memory behind bridge: 00000000d0000000-00000000dfffffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA- VGA+ MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [50] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [58] Express (v2) Root Port (Slot+), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag+ RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep+ BwNot+ LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #2, PowerLimit 75.000W; Interlock- NoCompl+ SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock- Changed: MRL- PresDet+ LinkState+ RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Not Supported, TimeoutDis- ARIFwd- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -3.5dB Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit- Address: fee0f00c Data: 4141 Capabilities: [b0] Subsystem: Advanced Micro Devices [AMD] Device 9600 Capabilities: [b8] HyperTransport: MSI Mapping Enable+ Fixed+ Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [110 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- Kernel driver in use: pcieport 00:0a.0 PCI bridge: Advanced Micro Devices [AMD] RS780 PCI to PCI bridge (PCIE port 5) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Bus: primary=00, secondary=02, subordinate=02, sec-latency=0 I/O behind bridge: 0000d000-0000dfff Memory behind bridge: fde00000-fdefffff Prefetchable memory behind bridge: 00000000fdb00000-00000000fdbfffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [50] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [58] Express (v2) Root Port (Slot+), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag+ RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep+ BwNot+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #10, PowerLimit 25.000W; Interlock- NoCompl+ SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock- Changed: MRL- PresDet+ LinkState+ RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Not Supported, TimeoutDis- ARIFwd- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -3.5dB Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit- Address: fee0f00c Data: 4149 Capabilities: [b0] Subsystem: Advanced Micro Devices [AMD] Device 9600 Capabilities: [b8] HyperTransport: MSI Mapping Enable+ Fixed+ Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [110 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- Kernel driver in use: pcieport 00:11.0 SATA controller: ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode] (prog-if 01 [AHCI 1.0]) Subsystem: Giga-byte Technology Device b002 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 32, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 22 Region 0: I/O ports at ff00 [size=8] Region 1: I/O ports at fe00 [size=4] Region 2: I/O ports at fd00 [size=8] Region 3: I/O ports at fc00 [size=4] Region 4: I/O ports at fb00 [size=16] Region 5: Memory at fe02f000 (32-bit, non-prefetchable) [size=1K] Capabilities: [60] Power Management version 2 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [70] SATA HBA v1.0 InCfgSpace Kernel driver in use: ahci 00:12.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0 Controller (prog-if 10 [OHCI]) Subsystem: Giga-byte Technology Device 5004 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 32, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 16 Region 0: Memory at fe02e000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd 00:12.1 USB Controller: ATI Technologies Inc SB700 USB OHCI1 Controller (prog-if 10 [OHCI]) Subsystem: Giga-byte Technology Device 5004 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 32, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 16 Region 0: Memory at fe02d000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd 00:12.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller (prog-if 20 [EHCI]) Subsystem: Giga-byte Technology Device 5004 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 32, Cache Line Size: 64 bytes Interrupt: pin B routed to IRQ 17 Region 0: Memory at fe02c000 (32-bit, non-prefetchable) [size=256] Capabilities: [c0] Power Management version 2 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Bridge: PM- B3+ Capabilities: [e4] Debug port: BAR=1 offset=00e0 Kernel driver in use: ehci_hcd 00:13.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0 Controller (prog-if 10 [OHCI]) Subsystem: Giga-byte Technology Device 5004 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 32, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 18 Region 0: Memory at fe02b000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd 00:13.1 USB Controller: ATI Technologies Inc SB700 USB OHCI1 Controller (prog-if 10 [OHCI]) Subsystem: Giga-byte Technology Device 5004 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 32, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 18 Region 0: Memory at fe02a000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd 00:13.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller (prog-if 20 [EHCI]) Subsystem: Giga-byte Technology Device 5004 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 32, Cache Line Size: 64 bytes Interrupt: pin B routed to IRQ 19 Region 0: Memory at fe029000 (32-bit, non-prefetchable) [size=256] Capabilities: [c0] Power Management version 2 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Bridge: PM- B3+ Capabilities: [e4] Debug port: BAR=1 offset=00e0 Kernel driver in use: ehci_hcd 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3a) Subsystem: Giga-byte Technology Device 4385 Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Capabilities: [b0] HyperTransport: MSI Mapping Enable- Fixed+ Kernel driver in use: piix4_smbus 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller (prog-if 8a [Master SecP PriP]) Subsystem: Giga-byte Technology Device 5002 Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 32, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 16 Region 0: I/O ports at 01f0 [size=8] Region 1: I/O ports at 03f4 [size=1] Region 2: I/O ports at 0170 [size=8] Region 3: I/O ports at 0374 [size=1] Region 4: I/O ports at fa00 [size=16] Capabilities: [70] MSI: Enable- Count=1/1 Maskable- 64bit- Address: 00000000 Data: 0000 Kernel driver in use: pata_atiixp 00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) Subsystem: Giga-byte Technology Device a102 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 32, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 16 Region 0: Memory at fe024000 (64-bit, non-prefetchable) [size=16K] Capabilities: [50] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: HDA Intel 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller Subsystem: ATI Technologies Inc SB700/SB800 LPC host controller Control: I/O+ Mem+ BusMaster+ SpecCycle+ MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (prog-if 01 [Subtractive decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop+ ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 64 Bus: primary=00, secondary=03, subordinate=03, sec-latency=64 I/O behind bridge: 0000c000-0000cfff Memory behind bridge: fdd00000-fddfffff Prefetchable memory behind bridge: fdc00000-fdcfffff Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- 00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2 Controller (prog-if 10 [OHCI]) Subsystem: Giga-byte Technology Device 5004 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 32, Cache Line Size: 64 bytes Interrupt: pin C routed to IRQ 18 Region 0: Memory at fe028000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd 00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor HyperTransport Configuration Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Capabilities: [80] HyperTransport: Host or Secondary Interface Command: WarmRst+ DblEnd- DevNum=0 ChainSide- HostHide+ Slave- <EOCErr- DUL- Link Control: CFlE- CST- CFE- <LkFail- Init+ EOC- TXO- <CRCErr=0 IsocEn+ LSEn+ ExtCTL- 64b- Link Config: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=16bit DwFcInEn- LWO=16bit DwFcOutEn- Revision ID: 3.00 Link Frequency: [b] Link Error: <Prot- <Ovfl- <EOC- CTLTm- Link Frequency Capability: 200MHz+ 300MHz- 400MHz+ 500MHz- 600MHz+ 800MHz+ 1.0GHz+ 1.2GHz+ 1.4GHz- 1.6GHz- Vend- Feature Capability: IsocFC+ LDTSTOP+ CRCTM- ECTLT- 64bA+ UIDRD- ExtRS- UCnfE- 00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor Address Map Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- 00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor DRAM Controller Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- 00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor Miscellaneous Control Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Capabilities: [f0] Secure device <?> Kernel driver in use: k10temp 00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor Link Control Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- 01:00.0 VGA compatible controller: ATI Technologies Inc Juniper [Radeon HD 5750 Series] (prog-if 00 [VGA controller]) Subsystem: PC Partner Limited Device e138 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 43 Region 0: Memory at d0000000 (64-bit, prefetchable) [size=256M] Region 2: Memory at fdfc0000 (64-bit, non-prefetchable) [size=128K] Region 4: I/O ports at ee00 [size=256] [virtual] Expansion ROM at fdf00000 [disabled] [size=128K] Capabilities: [50] Power Management version 3 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+ Address: 00000000fee0f00c Data: 4179 Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [150 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Kernel driver in use: radeon 01:00.1 Audio device: ATI Technologies Inc Juniper HDMI Audio [Radeon HD 5700 Series] Subsystem: PC Partner Limited Device aa58 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin B routed to IRQ 44 Region 0: Memory at fdffc000 (64-bit, non-prefetchable) [size=16K] Capabilities: [50] Power Management version 3 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+ Address: 00000000fee0f00c Data: 4181 Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [150 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Kernel driver in use: HDA Intel 02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02) Subsystem: Giga-byte Technology GA-EP45-DS5 Motherboard Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 42 Region 0: I/O ports at de00 [size=256] Region 2: Memory at fdbff000 (64-bit, prefetchable) [size=4K] Region 4: Memory at fdbe0000 (64-bit, prefetchable) [size=64K] [virtual] Expansion ROM at fdb00000 [disabled] [size=64K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable+ Count=1/2 Maskable- 64bit+ Address: 00000000fee0f00c Data: 4151 Capabilities: [70] Express (v1) Endpoint, MSI 01 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <8us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 4096 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <64us ClockPM+ Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- Capabilities: [b0] MSI-X: Enable- Count=2 Masked- Vector table: BAR=4 offset=00000000 PBA: BAR=4 offset=00000800 Capabilities: [d0] Vital Product Data Unknown small resource type 00, will not decode more. Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 14, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [140 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- Capabilities: [160 v1] Device Serial Number 12-34-56-78-12-34-56-78 Kernel driver in use: r8169 03:0e.0 FireWire (IEEE 1394): Texas Instruments TSB43AB23 IEEE-1394a-2000 Controller (PHY/Link) (prog-if 10 [OHCI]) Subsystem: Giga-byte Technology GA-EP45-DS5 Motherboard Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 32 (500ns min, 1000ns max), Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 22 Region 0: Memory at fddff000 (32-bit, non-prefetchable) [size=2K] Region 1: Memory at fddf8000 (32-bit, non-prefetchable) [size=16K] Capabilities: [44] Power Management version 2 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME+ Kernel driver in use: firewire_ohci
*** Bug 40082 has been marked as a duplicate of this bug. ***
> You can suppress the RCU CPU stall checking by adding > "rcu_tree.rcu_cpu_stall_suppress=1" to the kernel boot parameters. > If you do this, the stall would likely still be happening, but > RCU would refrain from reporting it. Still, this is useful in case > the problem is in the RCU stall checking itself. OK, I've tried this & received another oops at boot. (dmesg attached if it makes any difference. >From the look of the stack traces, I would guess that you have frame >pointers disabled. If so, could you please try reproducing this with >frame pointers enabled? >Could you please also try rebuilding the kernel with CONFIG_RCU_FAST_NO_HZ=n? Will do.
Created attachment 66642 [details] dmesg output with rcu_tree.rcu_cpu_stall_supress=1
On Tue, Jul 26, 2011 at 09:12:39AM +0100, Phil Armstrong wrote: > On 26/07/11 01:15, Paul E. McKenney wrote: > >Hello, Phil, > > > >Bugzilla seems less than responsive, so I am falling back on email. > > It seems to have recovered again now... I CCed it so that this message should show up automatically. > >You can suppress the RCU CPU stall checking by adding > >"rcu_tree.rcu_cpu_stall_suppress=1" to the kernel boot parameters. > >If you do this, the stall would likely still be happening, but > >RCU would refrain from reporting it. Still, this is useful in case > >the problem is in the RCU stall checking itself. > > Will try this. > > Looking at a boot log from this morning, I got the delayed boot but > without the RCU stall check oops, so perhaps there are really two > problems :( Or perhaps the stall is happening and delaying your boot whether or not RCU is complaining about it. Ah, and notice that CPU 3 is not responding to the NMI until a full minute after the stall was detected in https://bugzilla.kernel.org/attachment.cgi?id=66642: [ 3.976349] [<ffffffff8132f594>] ? start_secondary+0x1de/0x1e5 [ 67.293217] NMI backtrace for cpu 3 [ 67.293217] CPU 3 But wait... We also have the following, earlier in that same dmesg: [ 67.293213] [<ffffffff8100827e>] ? cpu_idle+0xa2/0xc4 [ 67.293213] [<ffffffff8132f594>] ? start_secondary+0x1de/0x1e5 [ 3.976363] NMI backtrace for cpu 0 [ 3.976363] CPU 0 Different CPUs seem to have rather different ideas about what time it is. Which could cause the system to wait a minute, if one CPU waited for an absolute time set by the other CPU. The different ideas about time can also provoke RCU CPU stalls. And you got a stall despite rcu_tree.rcu_cpu_stall_suppress=1. Not sure why that would be. Could you please double-check this? If worst comes to worst, I can send you a patch that hard-disables CPU stall checking. Thanx, Paul > > From the look of the stack traces, I would guess that you have frame > >pointers disabled. If so, could you please try reproducing this with > >frame pointers enabled? > >Could you please also try rebuilding the kernel with > CONFIG_RCU_FAST_NO_HZ=n? > > Will do. > > Phil
Looks like I mis-spelt "suppress" in the kernel command line for the second dmesg, so scratch the second report. I'll try again tonight with the correct option. Sorry about that. Having left the machine in question downloading the full git clone I'll try a frame-pointer enabled build.
Comment on attachment 66642 [details] dmesg output with rcu_tree.rcu_cpu_stall_supress=1 Ignore this, the kernel command line is wrong.
OK. I think I got the kernel command line correct, but I'm still getting an oops from the RCU code. See the attached dmesg output: syslog-3.0.0-no_rcu_stall2.txt
Created attachment 66762 [details] RCU oops with rcu_tree.rcu_cpu_stall_suppress=1
Created attachment 66782 [details] Mutiple RCU oops with CONFIG_RCU_FAST_NO_HZ=y, CONFIG_FRAME_POINTER=y
Setting CONFIG_RCU_FAST_NO_HZ=n results in a kernel that boots to [ 0.503236] Switched to NOHz mode on CPU#3 [ 0.503241] Switched to NOHz mode on CPU#1 [ 0.503247] Switched to NOHz mode on CPU#2 [ 0.503250] Switched to NOHz mode on CPU#0 and then hangs. After fifteen minutes I poked the keyboard & it promptly jumped ahead again, before hanging for good at: [ 92.899293] NET: Registered protocol family 1 I gave up after another half an hour. Note that the time on the log (92s) bears no relation to wall clock time at all. I'll try this kernel again & then give up on it!
Does nohz=off change the system behavior? Also, does tapping frequently on the keyboard (ie: ctrl key) during boot make the issue go away?
Tapping on the keyboard does seem to make the boot go quicker. I'll try a few times & see whether it kills the RCU oops: they don't appear every boot.
nohz=off eliminates the delays for both CONFIG_RCU_FAST_NO_HZ=y and allows the CONFIG_RCU_FAST_NO_HZ=n kernel to boot successfully.
Tapping on the keyboard doesn't seem to help with CONFIG_RCU_FAST_NO_HZ=y. I also have a kernel dmesg from that boot where the timestamps appear to go backwards as the oops is polled from the different CPUs. Odd.
Ok. If nohz=off resolves it, it sounds like it then a problem with the oneshot clockevent mode. Possibly the device that triggers the timer irq is halting in a low-power mode. Usually keyboard interrupts force the system out of low-power, allowing the system to continue more normally. That said, it does seem configuration dependent so there may be a few things going on. Could you also try "nolapic_timer" ?
nolapic_timer also fixes both kernels. It also eliminates some pauses (which could be curtailed by a keyboard press) in the shutdown process with showed up even with nohz=off.
Philip: Thanks for the debugging and very quick feedback! Thomas: Any thoughts on how to address this?
Thomas?
On Monday, August 15, 2011, Paul E. McKenney wrote: > On Sun, Aug 14, 2011 at 09:02:27PM +0200, Rafael J. Wysocki wrote: ... > > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=40092 > > Subject : RCU stall in linux-3.0.0 > > Submitter : Philip Armstrong <phil@kantaka.co.uk> > > Date : 2011-07-25 21:44 (21 days old) > > This one was due the the CPU's idea of the current time getting way > out of sync (as in more than a minute's worth of disagreement, which is > pretty impressive when you think about it). So what happened is that > one CPU decided that the current grace period was a full minute old > immediately, and thus started screaming piteously. John Stultz provided > a configuration workaround, and he and Thomas Gleixner would be working > out the long-term fix.